Is this… Entropix? (I'm kidding, there are many ways of enhancing decoding or training that leverage entropy. But this looks almost implausibly good)
2/8 We uncover a persistent Guess-Refine-Perturb forward-pass dynamic. Intermediate layers rigorously refine core reasoning, but the absolute final layers often drag predictions back toward safe, generic common words. This creates a massive planning-pragmatics tradeoff.