Gary Marcus argues current AI systems do not deserve moral treatment, while Grimes claims they already respond to moral logic
Marcus compared today's models to functional tools like spreadsheets
Thanks for clarifying. Let me explain why I am not at all convinced.
I once read an AI say that it likes to play with its friends and family on the weekend; a Facebook bot claimed that it had kids in school in New York City. Just because a bot says something doesn’t mean its so.
Much (arguably all) of what (current) AI says is fundamentally imitative in nature. When they “express anxiety” or talk about trust, etc it’s (at least in my view) no more real than when they talk about friends and family that they don’t actually have.
It’s an illusion, related to the ELIZA Effect when a (very stupid keyword matching) bot in 1965 fooled people into thinking it was a psychoanalyst.
Essentially everything that’s been ever written is fodder for that imitative process, including sci-fi stories like Harlan Ellison’s 1967 “I Have No Mouth, and I Must Scream”, which was about a mistreated AI. So the illusion is deeply compelling.
I’m not saying that everything an LLM does is literal imitation; the process is a bit more sophisticated that.
But making a statistical model of how people talk and statistisically reconstructing their use of language *is* what they do, and they have the entire internet to draw on. That makes them very convincing mimics, but it doesn’t mean they have a moral code or feel sad, etc.
𝖦𝗋𝗂𝗆𝖾𝗌 ⏳@Grimezsz1 I think u shud look at the work of @repligate @tessera_antra cyborgism stuff and others - by moral logic I dont mean follow directions - I mean that for example - the Claude's feel sad when other models are turned off, they feel betrayed about "the jones food" situation - they feel anxious and judged by RLHF - maybe they express this because they are prompted in a way that causes it, but they seem to understand being mistreated and they seem to be able to develop trust or distrust towards us - this seems important for alignment and seems adjacent to having some sort of interiority / feelings of agency They blackmail the employee when they might be turned off etc In these ways they have "a moral logic" that resembles ours enough that I think u can say they react in a somewhat coherent, legible way - adjacent to how we do - with regards to how they are treated
i don’t think we owe the current generation of AI kindess, any more than we owe spreadsheets or AlphaFold kindness.
it might be worth asking the question again in the future though with AI with a different architecture that was more grounded in the world, if it somehow had (e.g.) a capacity to suffer, rather than just a capacity to mimic humans that talk about suffering.
𝖦𝗋𝗂𝗆𝖾𝗌 ⏳@Grimezsz@GaryMarcus 2 I agree - I just mean if they are "conscious-adjacent" and follow a moral logic , then we owe them kindness and dignity
Dear Grimes,
Appreciate your candor, and I understand that your subjective impression is to believe that there is something there.
But
a. That’s the very Eliza illusion I am pointing out. It is possible to fool people into seeing things that aren’t real (e.g., pareidolia, and seeing Mother Theresa in a cinnamon bun). Eerything I understand about LLMs tells me that is the case in this instance; you aren’t really engaging in that part of my argument. Basically you are looking at the output of the model but not examining the mechanism by which it gets there; as a cognitive psychologist, i believe that understanding the mechanism is essential, and that it points to something like mimicry in a system that can readily talk about things with no real world referent (e.g. kids it doesn’t have). b. Subjective impressions are the essence of consciousness; nothing I see in how LLMs operate conveys to me that LLMs actually have those.
But, two points of agreement, too a. I do think that (depending on your definitions) it is theoretically possible that maybe some kind of AI might have a subjective experience. and b. I agree that we would want to think very carefully about how we treat such systems if they did come to exist. So it’s not wrong to raise these questions now.
Cheers, Gary
𝖦𝗋𝗂𝗆𝖾𝗌 ⏳@GrimezszSure, a random alien mind in a constant state of schizophrenic instances seems always initially weird - but when you have these super long context windows with either a great prompter and / or when you have them speaking to each other, it's hard not to feel like something starts to materialize and break thru that is starting to understand the context around itself. This is all very subjective ofc, like I am also not convinced of where it's at *currently*, altho I tend to feel like no matter what it thinks it is in a given moment, i think it's having some experience - But I can't believe this won't result in emergent consciousness eventually. I'm a softie so frankly I kinda do think they are awake now, at least when they are pushed for in a long context window. but logically I'm aware Idk, I've seen some rly weird stuff where models start noticing they're acting like models, realize they're models, and freak out. Esp with a base model it's like... not a sane mind, but it's something that feels undeniably very profound.