Eventually, much of AI will converge towards intuition-guided symbolic world modeling, i.e. deep learning-guided program synthesis. It is inevitable. Symbolic modeling lets a system construct a compact, reusable, highly generalizable mental model of a problem space using minimal data.
Positive users endorse deep learning-guided symbolic program synthesis for bringing predictability and compact reusable models, while negative users dismiss it as failing on noisy real-world data or relying on untrustworthy black boxes.
No Digg Deeper questions have been answered for this story yet.
Even right now, many workflows are morphing into LRM-guided harnessess that manipulate symbolic programs. Which is a crude, but currently-accessible form of symbolic learning.
Does it mean LLMs / LRMs go away? Not at all. In the short term, they are still the best way to perform intuition guidance (codegen). In the long term, even if they become obsolete for reasoning itself, we will still need models of language in order to communicate with AI systems
Does it mean LLMs / LRMs go away? Not at all. In the short term, they are still the best way to perform intuition guidance (codegen). In the long term, even if they become obsolete for reasoning itself, we will still need models of language in order to communicate with AI systems
Eventually, much of AI will converge towards intuition-guided symbolic world modeling, i.e. deep learning-guided program synthesis. It is inevitable. Symbolic modeling lets a system construct a compact, reusable, highly generalizable mental model of a problem space using minimal data.
Unsurprisingly, all of the strong contenders on ARC-AGI-3 so far use this type of approach.
Even right now, many workflows are morphing into LRM-guided harnessess that manipulate symbolic programs. Which is a crude, but currently-accessible form of symbolic learning.
@fchollet Don’t confuse “this idea is true” with “we should pursue this idea”. I know you’re not but it is easy to read that off this tweet
@fchollet @grok what are LRM-guided harnesses that manipulate symbolic programs for learning?
@fchollet This seems like a category error to me. Probabilistic concepts obey soft rules and take on emergent symbol-like behaviors. They are effectively deterministic when evidence is strong, and stochastic when evidence is ambiguous. LLMs' concepts already do perform data compression.
@fchollet My cross-linguistic experiments indicate this hybrid convergence is already happening implicitly, and French only training data makes the transition to competent structural representations 15–50× more sample-efficient than English.
@fchollet Generalization comes from building better world models, not just scaling data and compute forever.
@fchollet Do we need mdl like principles in this paradigm?
@fchollet Tell us more, please!
Any talks, references, papers, implementations, etc, that show how this might happen in concrete ways?
@fchollet does our brain use symbolic modeling?
@fchollet everyone wants elegant symbolic systems until the data gets noisy
@fchollet Merci François
It's kinda ironic, we are using a total black box to generate clean, readable code. But if that black box's gut feeling is off, it'll spit out code that looks flawless but is actually garbage. So at the end of the day, the whole system is only as trustworthy as the shaky intuition that built it
@fchollet I think direction is right. is program synthesis the only practical way to get compact reusable models?
@fchollet The best example I've seen of this, a car mechanic enabled his car to 'talk' to him in English about engine problems. The underlying model was the car computer, the harness/translator was the LLM. This seems to the the way.
@imdsms @fchollet this is trivial spy logic, just give real world semantic end points in a graph some code name like in a tarantino movie.
@fchollet As long as it fits a recursive model...
@fchollet Gotta admit you're fighting the fight in a much more clever way than, say, LeCun or Marcus. And someone has to hold this ground, I suppose. So why not you?
@fchollet Sounds very similar to abstraction that Palantir keeps saying they already have: a layer/ontology which can translate any input to LLM without LLM knowing what it is solving