More people need to listen to what Gary is saying here. Many, MANY of these talking heads who suck the oxygen out of the room with just their ego, say things that are so wildly wrong, that its misguiding an entire generation, and its all for ego and greed.

Literally, not a clue what they are talking about...

Yet the next idiot calls them an AI Expert 🤦‍♂️

@GaryMarcus It is clear enough to me Brin does not say Transformers alone.

His point is that the current architecture, with Transformers at the core, and whateer it takes bolted on, will be enough.

Which likely will. Claude is a wonder to behold.

LeCun, Fei-Fei Li etc will augment this.

@GaryMarcus Marcus isn't wrong. Everyone bolts tools onto transformers because they hit walls fast. Neurosymbolic AI is rising for a reason: transformers are a component, not the system.

Gary, you're right that transformers aren't sufficient. But neurosymbolic AI isn't the answer either. It's a better map of a map.

Meaning is not a capability. It's not emergent. It's the precondition for understanding to exist at all.

Every system today, transformer or hybrid, operates on artifacts *produced by* meaning-bearing humans. That's imitation of understanding, not understanding.

The substrate has to be different from the start.

@WinnHuab it was funnier the first time

I once heard a joke: A man goes to the doctor and says he’s depressed. He says life seems hard and cruel. He explains that he feels alone in a menacing world where large language models promise revolutions that never arrive, where “understanding” is just statistics in disguise, and where everyone applauds the hype while true artificial intelligence remains a distant myth.

The doctor says: “The treatment is simple. The great Gary Marcus is in town — go watch his lecture on the limits of LLMs and why they will never truly understand the world. That should cheer you up.”

The man bursts into tears and says: “But doctor… I am Gary Marcus.”

@AndyXAndersen @GaryMarcus No, I don't think transformers at the core make sense for AGI either, but are a component for language. The real question, is there demand for AGI? I think AGI is only in demand, from the governments that want to nationalize it for political dominance.

@GaryMarcus yes, but what is agi? they haven't properly defined it yet, so it may indeed be enough to qualify as agi by "some definitions"..

@Ken_Granville read my 2020 Next Decade in AI for why I don’t believe NSAI is sufficient.

@GaryMarcus Transformers are to AGI what Broca's area is to the brain.

@P33RL3SS @GaryMarcus AGI is a marketing term. In practice we need automation that can reliably ingest messy world phenomena and come up with effective strategies for processing.

So, if not Transformers, something in the spirit of it. Large-scale data and methods.

@ecoezen see my essay dear elon musk, and agidefinition.agi.

@P33RL3SS @AndyXAndersen @GaryMarcus Of course there is demand for AGI. And If AGI truly exists govs wont be able to control it. If they can then it is not AI.

@GaryMarcus Needed or used maybe, for the lack of alternatives, but even "necessary" is too strong imo.

@andygrossberg no, not really. but it’s right to think of as a component rather than the whole thing.

@GaryMarcus interesting framing when meta just dropped joint embedding architecture that actively moves away from pure seq2seq

what does "neuro" look like in practice today?

@GaryMarcus gotta love when billionaires skip the last 3 years of research

can u really add tool use and call it "just transformers"?

@GaryMarcus Combining methods feels more human than betting on one architecture

@GaryMarcus neurosymbolic AI is fancy word of coding :)

@GaryMarcus the hard part is nobody agrees on what parts beyond attention actually count as the "architecture"