A post reports that Jan Varho set a NanoGPT speedrun world record of 67.6 seconds, a 0.4-second improvement. The technique masks impossible token continuations. In the post’s example, “Hyperparameter” splits into “Hyper,” “param” and “eter,” yet multi-token prediction teaches “Hyper” to predict both later pieces—even though “eter” cannot directly follow it in the tokenized dataset. The post says masking that continuation helps improve validation loss and, during inference, prevents the model from generating token pairs never seen in training.