What's this about?
Reinforcement learning has become the main lever for improving reasoning in language and vision-language models, but its strongest successes still depend heavily on ground-truth supervision — verifiable answers you can grade against. Co-RL shows reasoning emerging without that supervision, from a diverse cohort of agents trained together in multi-agent RL.
What exactly is the ground-truth problem?
Reinforcement learning needs a reward, and the cleanest reward is a verifiable answer: the maths problem has a correct result, the code passes the tests. That's why RL-with-verifiable-rewards has driven so much recent progress — and also why it's confined to domains where verification is cheap. Most valuable reasoning isn't like that: strategy, diagnosis, writing, judgment. Co-RL's approach is to substitute cohort agreement for a verifier — if a diverse population of agents independently converges on something, that convergence carries information about correctness. The obvious risk, which is the whole difficulty, is that a population can converge on being confidently wrong together, which is why "diverse" is doing the work in the title.
Why is this worth noticing?
- It targets the binding constraint on RL for reasoning, which is verifier availability rather than algorithm quality.
- It's the same shape as this week's Grading the Graders paper from the opposite direction: one asks how good verifiers are, this asks what to do without one.
- The failure mode is documented elsewhere in this month's papers — Physics of Agents modelled exactly the phase transition where a population flips from productive disagreement into uniform consensus.