Gary Marcus Questions OpenAI Astra Math Claims
Cognitive scientist compares AI demo to historical math advances like calculus and algebra.
Gary Marcus posted multiple times questioning claims around OpenAI Astra math results. He called Fable's assertion of the most significant day in math history absurd and said no new techniques or theories appeared. Marcus listed past advances including calculus, algebra, logarithms, probability, the decimal system, information theory, and the concept of zero, asking whether Astra exceeds them. He added that math problems suit formal methods and data augmentation far better than most real-world tasks, so strong results there will likely not carry over to everyday problems at the same quality.
Combined views
767.7K
62 posts, first seen 23d ago
Reactions from ranked influencers
62 posts
the thing that is deeply misleading about this is that most problems are not as amenable to formal methods and data augmentation as math is. you will be able to “use” the model for “every problem [you] face in life”, but in most problems you will likely not get the quality of…
Everyone in the world will soon be able to use the model that made these breakthroughs for every problem they face in life, no matter how mundane, at a cost that will fall dramatically in a matter of months. I still struggle to get my head around this fact.…
What is deeply misleading about this — and many similar statements re Astra— is that *most problems are not as amenable to formal methods and data augmentation as math is.* you will be able to “use” the model for “every problem [you] face in life”, but in most problems you will…
Everyone in the world will soon be able to use the model that made these breakthroughs for every problem they face in life, no matter how mundane, at a cost that will fall dramatically in a matter of months. I still struggle to get my head around this fact.…
Many of the models I critiqued here are ones you too would laugh at now (after having worked with them) and realize in hindsight were not remotely godlike. But I am sure this one is different. 🤦♂️
Fascinating: OpenAI’s @deanwball is saying Astra can do anything, and it’s not even clear it can do “anything” in math (let alone anything in more or open-ended, less formalizable domains).
I dropped Noam’s tweet in Claude to see it’s reaction, and this was it’s final line (totally unprompted too, I did not ask it to neg): “The pattern worth noticing: almost everything here is a construction or an explicit bound — the kind of thing where a clever object, once…
By and large nobody here wants to actually look at what Astra’s technical limits might be. (And there may be a lot). Most just want to scream “Oh My God, My Doomerism/Optimism is Vindicated!” Confirmation bias is a terrible thing.
Fascinating: OpenAI’s @deanwball is saying Astra can do anything, and it’s not even clear it can do “anything” in math (let alone anything in more or open-ended, less formalizable domains). https://twitter.com/benjamin_horne/status/2083550923872145619
“Stochastic parrots” is not my term (it’s @emilymbender’s). But a lot of people today commenting on it are confused. To some extent (though I don’t think it’s a perfect metaphor, and have said that before), pure LLMs *are* like stochastic parrots. But Astra almost certainly…
These stochastic parrots are OUT OF CONTROL! https://twitter.com/ns123abc/status/2083505040224887272
