Depth Memory framework has model layers read from and write to streams

A post introducing Depth Memory says mHC and AttnRes show that richer depth mixing improves model performance at scale. The framework has each layer read from a set of streams to form its input, then write back its output. The post says Linear Depth Memory equips models with a matrix-valued memory. Separately, another poster asks what the state of the art is for residual approaches.