Sakana AI's Continuous Memory Machine separates short- and long-term memory

DAIR.AI says recurrent models often keep everything in one hidden vector, forcing short-term computation and long-term storage to share space. It describes Sakana AI's Continuous Memory Machine as using separate matrices for recent neuron activity and information needed later. DAIR.AI says the model beat LSTM, DNC, RMC and CTM baselines on several tasks and generalized to longer inputs than earlier memory-augmented networks.