Reaction
Çağlar Gülçehre revisits 2016 Dynamic Neural Turing Machine paper, likens its NOP cell to transformer attention sinks
Çağlar Gülçehre says its NOP cell lets a controller attend to “nothing” when no meaningful read or write is needed.
TLDR
The arXiv record for the Dynamic Neural Turing Machine paper lists a June 2016 submission. On Oct. 22, 2026, co-author Çağlar Gülçehre pointed to a NOP memory cell he says the team introduced, describing it as an option when no meaningful memory read or write is needed. He argues that it serves the same purpose as transformer attention sinks, but the paper’s abstract does not describe the cell or establish that comparison.
Combined views
2.5K
2 Sources, first seen ago
