Jonas Geiping Announces Recurrent Depth LLM Training

Jonas Geiping, a machine learning researcher, posted that his group trained an LLM with recurrent depth at scale. He described spending the last year, actually a bit longer, on the project. The model includes an internal latent space that lets it adaptively spend more compute to think longer. Geiping said he could finally discuss the work publicly and pointed to a tech report in the post.