Samuel Hammond argues advanced LLMs may possess consciousness, while Gary Marcus disputes that behavioral mimicry indicates subjective experience
Marcus points out persistent errors like basic counting failures.
interesting paper, which I quickly skimmed. but…
1. you are showing convergence on some domains, but
(a) that convergence is parasitic, in the sense of being contingent on being trained on human data; you won’t see if they are trained only on code (b) the convergence is partial; LLMs still do lots of things like (fumbling with counting) that are just silly (c) convergence on cognitive behaviors doesn’t logically entail that the LLMs have experience, which is the real question here.
2. additionally, i think the world model arguments you make are weak, considering e..g that an off the shelf LLM can’t even reliably follow the rules of chess, even with lots of sample games and exposure to explicit rules. even in chess what you get is an approximation and not a real world model (which I know Fei Fei Li and Yann LeCun and Demis Hassabis would acknowledge). likewise all the legal hallucinations and biographical errors etc stem from a lack of proper world models.
3. see also the comments i made to @Grimezsz re imitation of language re parenting etc that cannot possibly have a proper referent. mimicry is not experience.
Samuel Hammond 🦉@hamandcheese@Grimezsz @GaryMarcus I don't know if AIs are conscious for certain, but it's getting increasingly hard to rule out: https://www.secondbest.ca/p/time-to-take-ai-consciousness-seriously
@hamandcheese You need a persistent state / memory if you are to have consciousness.
Samuel Hammond 🦉@hamandcheese@beffjezos Agree with the intuition that consciousness has something to do with online learning, but it's possible it's already present with in-context learning. https://www.secondbest.ca/p/time-to-take-ai-consciousness-seriously
@beffjezos Agree with the intuition that consciousness has something to do with online learning, but it's possible it's already present with in-context learning.
Beff (e/acc)@beffjezosConsciousness is an emergent property of self-organizing matter, it is not going to be something exclusive to the biological substrate. While I wouldn't say current LLMs are conscious, they will soon have online learning and persistent state, which will make them effectively indistinguishable from conscious beings.
@beffjezos You don't think transformers have state, at least within session? They don't have a mutable state like an RNN or mamba but they carry state across layers and autoregressively within the KV. Hence their coherence over long horizon
Beff (e/acc)@beffjezos@hamandcheese You need a persistent state / memory if you are to have consciousness.