What's this about?
When one model produces only failed rollouts for a task, GRAFT substitutes informative successes and failures already found by another model, without declaring either model the teacher.
What exactly is off-policy mismatch?
Training data came from a different policy, so its choices may be unnatural for the learner. Compatibility weights and clipping limit how strongly those foreign trajectories update the model.
Why does this matter?
12 HF upvotes. Both models improved across three pairs at unchanged rollout budgets, suggesting expensive exploration can be shared.