Recirculation Technique Recycles Activations in Transformers
The method reuses top-layer activations in lower layers during inference on existing models.
Researcher Michael C. Mozer posted a paper describing recirculation, an enhancement for off-the-shelf foundation models. The approach passes activations from higher layers into lower layers during subsequent inference steps. It requires no retraining. The paper states the change markedly reduces perplexity and boosts accuracy on generation and reasoning tasks while adding near-zero latency. Several researchers shared the work on X and highlighted its simplicity.
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training?
Reactions from ranked influencers
15 postslike everything works in ai now you can literally do whatever
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training?
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training?
like everything works in ai now you can literally do whatever
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training?
What if a foundation model could tell us how to modify its architecture to boost inference and reasoning instantaneously—without retraining? What if that tweak incurred near zero latency cost during generation and supported indefinite state tracking?
What if a foundation model could tell us how to modify its architecture to boost inference and reasoning instantaneously—without retraining? What if that tweak incurred near zero latency cost during generation and supported indefinite state tracking?
like it wouldn't surprise me if OpenAI or Ant or both have something like this:
Recurrent transformer that injects activations from the top layers to the bottom layers at the next step (https://arxiv.org/abs/2608.08888). Why does this work without training?
