What's this about?
Post-training methods like RL and on-policy self-distillation have driven rapid progress in mathematical reasoning, but they depend on ground truth. TTPO optimises the policy at test time, without that dependency.
What does "test time" mean here?
Training time is when you adjust the model's weights using labelled data. Test time is when you use it, on a problem whose answer nobody knows. Test-time optimisation means improving the model's behaviour on *this specific problem*, right now, using only signals available without an answer key — self-consistency across samples, verification of intermediate steps, or internal agreement. It is the difference between studying and thinking harder during the exam.
Why does this matter to you?
- The ground-truth dependency is the binding constraint on applying RL outside maths and code, where verification is cheap.
- Anything that improves reasoning without labels transfers to domains that will never have labelled data.
- One of three papers today arguing the post-training orthodoxy is over-fitted to its own conveniences.