Gary Marcus Disputes AI Scaling Claims Citing Claude's Symbolic Code

@GaryMarcus the whole AI bubble depends on investors not understanding that the progress on coding harnesses DOES NOT GENERALIZE to other less-quantifiable tasks. AI companies are deliberately obscuring that massive limitation. most journalists do not understand or acknowledge it either

@GaryMarcus the whole AI bubble depends on investors not understanding that the progress on coding harnesses DOES NOT GENERALIZE to other less-quantifiable tasks. AI companies are deliberately obscuring that massive limitation. most journalists do not understand or acknowledge it either

@GaryMarcus It's very true ... for evidence, run tweakcc, it can extract the 250ish prompts built into Claude Code, all strung together with logic fragments and loops

https://github.com/Piebald-AI/tweakcc

@ArtKeller @GaryMarcus Examples please

@GaryMarcus You might be able to claim this in 2023. 2024 latest. He knows this is a lie.

@GaryMarcus Claude Code without the harness is just a hallucination generator. Same with trading bots without the execution rules.

@QuantCappers @GaryMarcus look up the articles about what actually powers Claude Code harness- it is a bunch of GOFAI. Which is fine-but that is domain-specific. to use it for a different thing, you need to build a different harness AND it has to be for a highly-verifiable application like coding.

@GaryMarcus TIL Gary doesn't know about the Pi harness

@GaryMarcus I'm not even a specialist in this field but I do understand that you can't reliably bruteforce your way to AGI with sheer compute unless you have a matroshka brain or something to do it with.

@GaryMarcus yes broski, kinda agree

Jack Clark underestimated AI

@LH a rewrite in python is also available. it’s 500k lines of symbolic code etc

@GaryMarcus the "just scale it" crowd won't like this

@GaryMarcus Most of it written by Claude Code itself. Given how much Gary Marcus has been a huge proponent of neurosymbolic AI for decades, should Claude Code be renamed Gary Code? Or should he be given a Turing Award? Or both?

@GaryMarcus the deeper point: capability is increasingly in the harness, not the weights. once Claude Code is 500k lines of scaffolding around the model, the next gains come from better scaffolding, memory, tools, observability, not a bigger base model. progress is moving up the stack.

@phibrowser @GaryMarcus This is a false dichotomy. You're saying it is either the harness matters or model size matters. The reality is that both matter. Better harnesses can extract more value from a model, and larger models unlock capabilities no harness can invent. Progress comes from both axes.

@GaryMarcus Synthetic Authority: Trust in AI systems grows with its fluency, not its understanding, and institutions drift toward whatever that system was built to optimize, not toward what is right or true.

@GaryMarcus Didn't Richard Sutton, the guy who coined the Bitter Lesson, come out and say the Bitter Lesson was falling apart?

@GaryMarcus Yes, it even uses the token "cymbal".

@GaryMarcus The production coding agents that actually ship are already hybrid by necessity. Pure neural scaling hits the same guardrail and determinism walls that made symbolic components non optional in the first place.

@ArtKeller @GaryMarcus Marcus keeps pointing at the scaffolding and calling it the building. those symbolic tools extend a capable model, they don't replace the need for one.