What's this about?
SEED proposes self-evolving on-policy distillation as an alternative to outcome-based reinforcement learning for training long-horizon agents — models that do multi-turn tasks with tools and environment feedback. (HF 3↑)
What exactly is "on-policy distillation"?
Distillation = a student model learning from a teacher's outputs. On-policy = the student learns on its *own* attempted trajectories (with the teacher correcting), not the teacher's — so it learns to fix its actual mistakes. "Self-evolving" loops this: the improved student generates better trajectories to learn from next round. Notably, DeepSeek V4's recipe (covered yesterday) leaned on the same family of techniques instead of RLHF.
Why does it matter?
Outcome-based RL for agents is expensive and sparse (one reward per long task). If distillation variants match it, training capable agents gets dramatically cheaper — a trend now visible across multiple labs.