Ofir Press Marks Ten Years in Language Modeling

Ofir Press posted that he reached ten years of language modeling experience a few weeks ago. He began the work in January 2016 and submitted a paper to EMNLP in June of that year. The paper showed that one matrix could serve as both the embedding and unembedding layer. It was rejected. His message includes a screenshot of an arXiv abstract page. The statement stands as his own account of the timeline.