NITP Introduces Implicit Token Prediction for Advanced LLM Pretraining

this paper should have claim the title to be the first self distillation during pretraining😌It's kind of conceptually close to Next Latent Prediction Transformers from https://arxiv.org/pdf/2511.05963v1. The loss and setup seem to closely resemble siamese though.