Gary Marcus argues that inference-time compute and reinforcement learning scaling validate his claims on neurosymbolic AI

GMGary Marcus@GaryMarcus

Neurosymbolic AI is saving deep learning from hitting the wall (*exactly* as I said my 2022 paper “deep learning is hitting a wall”) Really sad to see someone as smart @peterwildeford confusing the original argument to this degree.

PWPeter Wildeford🇺🇸🚀@peterwildeford

Once upon a time there was an Lead AI Developer who's AI was not getting impressive benchmark results. That evening, all of his neighbors came around to commiserate. They said, "We are so sorry to hear that deep learning is hitting a wall. This is most unfortunate." The Lead Developer said, "Maybe." The next day the LLM came back bringing seven massive benchmark scores and even got 90% on the LSAT. I the evening everybody came back and said, "Oh, isn’t that lucky. What a great turn of events. You now are really close to AGI!" The Lead AI Developer again said, "Maybe." The following day his son tried to train the next successor model, and while training it, he found that 10x'ing pre-training compute wasn't giving results anymore. The neighbors then said, "Oh dear, that’s too bad. Deep learning is hitting a wall." and the Lead AI Developer responded, “Maybe.” The day after, the Lead AI Developer announced they'd achieved breakthrough results by adding inference-time compute, RL scaling, and tool use. The neighbors came around and said, "Oh wow, AGI is soon!" The Lead AI Developer said, "Maybe."