Announcement
Soft-MTPβs one-line training change is claimed to speed up speculative decoding
Jayden Teohβs preprint says the method removes a noisy gradient without adding parameters or measurable overhead.
TLDR
In his October 28 preprint, Teoh describes Soft-MTP as a one-line change to multi-token prediction that recovers a self-distillation gradient using existing training calculations. He reports lower next-token prediction perplexity, faster speculative decoding and higher zero-shot accuracy than standard MTP at every tested scale, from 167 million to 1.2 billion parameters. The projectβs GitHub repository provides code and links to pretrained model checkpoints.
Combined views
9.7K
3 Sources, first seen ago

Comments