What's this about?
On-policy distillation for flow matching models, without a teacher model. 71 upvotes.
How can you distil without a teacher?
Normally distillation needs a big model to imitate. Self-distillation instead has the model learn from a *more expensive version of itself* — typically its own output computed with many more sampling steps, which is slower but better. That high-quality output becomes the target for the cheap fast path. There is no external teacher because the teacher is the same weights given more compute, which is why it works for generative models where quality scales with sampling effort.
Why does this matter to you?
- Removing the teacher removes the main cost and the main licensing obstacle to distillation.
- It is the fourth on-policy-distillation paper to trend this month, after SecOPD, Open-MOPD and D³-MOPD — the technique is spreading fast.
- Flow matching is the generative-model family behind much current image and video work, so this is an efficiency result with immediate product consequences.