Pedro Domingos Says LLMs Memorize Data, Not Compress It
Reply by Pedro Domingos to Christian Szegedy on maximum-likelihood training of LLMs.
TLDR
In a reply to Christian Szegedy, Pedro Domingos stated that likelihood is maximized by memorizing the data, which is largely what LLMs do. He called the description of this process as compression a travesty. Domingos is Professor Emeritus of Computer Science at the University of Washington and author of The Master Algorithm. The exchange appears among visible replies on X. No further details on the original post or responses are supplied in the packet.
Combined views
656
1 Source, first seen 29d ago