Announcement
Soft-MTP’s one-line training change is claimed to speed up speculative decoding
Jayden Teoh’s preprint says the method removes a noisy gradient without adding parameters or measurable overhead.
TLDR
In his October 28 preprint, Teoh describes Soft-MTP as a one-line change to multi-token prediction that recovers a self-distillation gradient using existing training calculations. He reports lower next-token prediction perplexity, faster speculative decoding and higher zero-shot accuracy than standard MTP at every tested scale, from 167 million to 1.2 billion parameters. The project’s GitHub repository provides code and links to pretrained model checkpoints.
Combined views
1.7K
3 Sources, first seen ago
