What's this about?
A revisit of training strategies when you have no chain-of-thought data, questioning whether next-chunk reasoning RL genuinely beats plain supervised fine-tuning. 19 upvotes.
What is SFT versus RL here?
Supervised fine-tuning shows the model correct examples and trains it to reproduce them — cheap, stable, limited by the examples you have. Reinforcement learning lets the model generate its own attempts and rewards the good ones, which can exceed the demonstration data but is expensive and finicky. RL is currently fashionable, so papers that carefully check whether a well-tuned SFT baseline matches it are valuable and unglamorous.
Why does this matter to you?
- "Is the fashionable method actually better than the boring baseline" is a genre worth reading every time it appears.
- The no-CoT-data setting is the realistic one for most practitioners.
- Read next to TTPO and SWE-Prime in today's digest, both of which also question the scale-up-RL default.