DeepMind Paper Introduces Recirculation for Transformers
Method feeds deep-layer activations back to shallow layers at inference time.
Posts on X discuss a Google DeepMind paper titled Recirculation. It describes an inference-time enhancement for existing foundation models. The approach recirculates a small fraction of deep-layer activations downward so shallow layers gain access to context already processed deeper in the network. The arXiv abstract states this change reduces perplexity and raises accuracy on generation and reasoning tasks. No retraining is required. Creators shared the paper and noted its unusually short title along with the potential benefit for models that lose state over long inputs.
Combined views
11K
3 posts, first seen 2d ago


