Grouping equivalent stop tokens reportedly prevents runaway output in on-policy distillation
A user sharing a paper says teacher and student models can favor different end-of-sequence tokens—the signals that end generation—even when their declared stopping sets match.
TLDR
The post describes a failure mode in on-policy distillation: a teacher model penalizes the student's preferred stopping token without transferring its own alternative, encouraging overly long responses that exhaust the generation budget. Matching decoding stop sets alone does not fix it, according to the account. The user says treating functionally equivalent end-of-sequence tokens as one shared stopping action eliminated length explosions across Qwen3, Llama and Gemma rollouts without hurting reasoning accuracy.
Combined views
—
1 Source, first seen ago
— likes— comments— saves— reposts
