The paradigm also shortens model reasoning traces.
Self-distillation does not work for thinking models YET
https://arxiv.org/abs/2603.24472 https://openreview.net/forum?id=VhCJItwQHn https://arxiv.org/abs/2606.11709
ML Twitter: What's your favorite on-policy (self)-distillation paper / blogs from this year? Sharing your own work is totally fine!
If who want to learn more about LLM distillation, you can watch: https://youtu.be/O1AR4iL30mg?si=Zznk_BYCnjCAmhAz
Some users find the research on self-distillation failing for thinking models interesting because it controls model confidence in epistemic states and reduces OOD generation.
@agarwl_ Why do you think there is so much focus on the single-sample version? I'm confused by the "negative" token arguments on twitter.
Self-distillation does not work for thinking models YET
https://arxiv.org/abs/2603.24472 https://openreview.net/forum?id=VhCJItwQHn https://arxiv.org/abs/2606.11709
@agarwl_ I guess the more bitter-lesson pilled way to go would be to let a judge (maybe the model itself) decide which tokens to learn from and which not :)
@agarwl_ what do you think about this one
https://arxiv.org/abs/2606.11709
@hallerite The idea is smart (condition on different hints to capture "content" tokens) but didn't feel simple enough. Also we do care about pause tokens (hmm, wait, ..) that they call stylistic tokens to ignore
@agarwl_ but there's a nuance to it
https://arxiv.org/abs/2605.06188
@agarwl_ interesting that it seems to control how "confident" the model is in its epistemic state which reduces ood gen
@agarwl_ When you say self-distillation are you talking about on-policy self-distillation (OPSD)? Can’t you use non-thinking model as a student and the thinking version of the same model as teacher for OPSD?
@agarwl_ soon