What's this about?
A training method where a model teaches itself without an external teacher model, engineering an information asymmetry so the "self-teacher" stays reliably stronger than the student it trains (43 upvotes).
What exactly is "self-distillation"?
Normal distillation needs a big teacher model; self-distillation makes one model both teacher and student — like grading your own homework, which only works if the grading-you somehow knows more than the answering-you. The trick is giving the teacher side extra context the student side lacks.
Why does it matter?
Removing the external-teacher requirement removes the biggest cost and licensing bottleneck in distillation — anyone's model can improve itself, which is also why the technique feeds directly into the industry's distillation-ethics arguments.