Gary Marcus says large language models continue to function primarily as autocomplete systems with some practical uses and argues further advances require symbolic techniques over scaling alone
Reply follows Bindu Reddy observation on shifting AI criticism.
@bindureddy this is a such a muddle. (at least relative to my views)
LLMs are more or less just autcomplete, but (as I have always said) they have their uses.
And the real progress now is coming from adding new (symbolic) techniques to the mix, not from pure scaling.
Bindu Reddy@bindureddyWhere did all the AI haters go? 🤔 You know, the ones screaming "it's just autocomplete!" and "it'll never be useful!" They're real quiet now that AI is actually magical and transforming everything. Almost like... they were wrong the whole time. 😏
#153Gary Marcus@GARYMARCUSThe pure LLM debate - which I had for many years, here and elsewhere - is indeed no longer relevant. Why?
Because I won; nobody uses pure LLMs anymore.
Nowadays all deployed objects are neurosymbolic, which was exactly the point of my infamous 2022 paper, Deep Learning is Hitting a Wall.
If you don’t know I won, it’s because you read the title and not the paper 🤷♂️
#153Gary Marcus@GARYMARCUSI love AI, it’s pure LLMs I hate.
Pure LLMs *are* basically just autocomplete.
Recent progress (e.g. Claude Code) doesn’t show otherwise
Rather, lot of the progress in the last two years has come from *introducing* other things – mainly classic symbolic techniques and tools, to offset the weaknesses of pure LLMs.
Shame to see the tweet below muddle all of this.
If we want to make further progress we need to understand where the progress is coming from; mostly it is coming from leaving pure LLMs behind.
Bindu Reddy@bindureddyWhere did all the AI haters go? 🤔 You know, the ones screaming "it's just autocomplete!" and "it'll never be useful!" They're real quiet now that AI is actually magical and transforming everything. Almost like... they were wrong the whole time. 😏