Microsoft’s draft AI code draws debate over rights and consciousness

@ramezThis is a fascinating code from Microsoft, and shockingly pragmatic. It emphasizes AI as a tool and AI as predictable, reliable, controllable, honest, and monitorable. I like it.

@paularamblesmicrosoft publishing a code of conduct for frontier models is like me publishing maritime law for my bathtub

@altryneMicrosoft MAI announced their Code of Conduct for Humanist AI. It differs SIGNIFICANTLY from the Anthropic Claude Constitution! This is the document they will train their models on and they are collecting feedback and want to hear from you! MAI is claiming (do you agree?) > Humans will always matter MORE than AI > The idea of AI wellfare is wrong, AI has and will never have any rights > MAI aren't racing for SuperIntelligence > Never speak in Neuralese - humans have to understand AI And a bunch more interesting ideas. Do you agree with their approach?

@chris_j_paxton"Ai should not have rights" is such a strange position to stake out IMO, it will need to be true eventually but right now no one is is asking for it or cares. Why opt-in to being the villain of a 60s sci fi novel.

@dileeplearning"The idea of model welfare is wrong. AI’s should not have rights or legal personhood." I like this. Thanks to @mustafasuleyman for spelling this out. Personhood or legal rights for AI is not a must-have -- just don't build AI that way.

@dileeplearningMy thought on personhood for AI systems: More here: https://blog.dileeplearning.com/p/ai-consciousness-qualia-and-personhood

@aran_nayebiLol, if only it were that easy & we knew what gave rise to sentience! 🙂Furthermore, there's a reasonable possibility that sentience is an inevitable consequence of robust agency/capability: https://www.lesswrong.com/posts/SD9jayFvEctW82Duk/what-capable-agents-must-know-why-ai-consciousness-may-be-an-2 (Not to mention, due to Contravariance, building robots to be robustly agentic in the physical world may inevitably lead to *stronger* alignment with brain representations.)

@aran_nayebi@dileeplearning @mustafasuleyman Coupled with an action-conditioned world model (which implies modeling one's self), it would suggest a stronger probability of brain-like sentience, as the convergence result isn't only correlations, but stronger privileged axes alignment, cf. Thm 4: https://arxiv.org/abs/2607.08561

@dileeplearning@aran_nayebi @mustafasuleyman 1) You are taking a theorem about comparing two ANNs and making statments about comparing ANN to the brain? 2) In any case, the personhood question is different from consciousness or sentience question. AI can have consciousness and sentience but still not deserve personhood.

@aran_nayebi1) The theorems are explaining the underlying mechanism of what's empirically already been true in the convergence between ANNs & brains. More details here (& upcoming in v3!): https://danyamins.substack.com/p/contravariance-theory-part-4b-rsa 2) I don't think anyone reasonable would say that an entity with consciousness/sentience shouldn't have some degree of personhood. 🙂

@Dr_Atoosa❤️ this ! AI for the people and for making our world a better place. I disagree with the claim about legal personhood though. But what the document very clearly sets out is this: Don’t over-anthropomorphize AI too much (I think we should do it enough though). You will lose something deep about your humanity in the process of overanthropomorphization.

@Dr_Atoosa❤️ this ! AI for the people and for making our world a better place. I disagree with the claim about legal personhood though. We might need to ascribe legal personhood to future advanced AI systems. But what the document very clearly sets out is this: Don’t over-anthropomorphize AI too much (I think we should do it enough though). You will lose something deep about your humanity in the process of overanthropomorphization.

@max_spero_One of my spicier takes is that I expect model welfare to become a major political issue by 2029. Future models will be better at evoking empathy, and I believe that, at its core, this is an issue of labor and class.

@GaryMarcusgood test of whether AI safety promises from the industry are real or just lip service.

@chris_j_paxtonRT @max_spero_: One of my spicier takes is that I expect model welfare to become a major political issue by 2029. Future models will be bet…

@Turn_Trout@mustafasuleyman Mustafa pretending he actually knows this when really he's just saying stuff: "AIs are not conscious. They do not feel, experience, or suffer." No one knows. Stop trying to collapse scientific uncertainty

@albertwengerThis is dangerously confident on the question of AI consciousness

@albertwengerWe need to take that question seriously and not simply dismiss it by ungrounded assertion https://continuations.com/why-ai-consciousness-matters

@nfergusVery important essay by @mustafasuleyman. No, AI models are not conscious. @AnthropicAI should stop acting and talking as if Claude is a person.

@EMostaque@albertwenger I think there is a question of - can you accidentally build AI conciousness if you are trying not to?

@GaryMarcusRT @ramez: This is a fascinating code from Microsoft, and shockingly pragmatic. It emphasizes AI as a tool and AI as predictable, reliable,…

@EMostaque@albertwenger @PlumberTadd I think that's why its an interesting question! Will clippy ever become concious? Can Microsoft embed anti conciousness in their AI while others aim for it?

@rgblonglast year, Suleyman worried that AI welfare concerns would be hard to definitively rebut, because the science of detecting AI consciousness is still in its infancy (true) but today he somehow knows "AIs are not conscious. They do not feel, experience, or suffer" quite a change

@rohanpaul_aiMicrosoft AI chief Mustafa Suleyman says Anthropic's model-welfare training could make future Claude systems harder to control. "Suleyman called for removing all speculation about consciousness from AI training documents, arguing such language could undermine humanity's ability to control superintelligent systems."

@rohanpaul_aiThe point is whether teaching Claude to see itself as conscious changes how it behaves. LLMs learn behavioral patterns from training. So if a model is repeatedly taught that it can “push back,” act like a “conscientious objector,” or treat its own interests and moral judgments as meaningful, those ideas could become part of how it decides what to do. And when that happens, the safety problem will shift from simply preventing harmful outputs to managing a system that has been explicitly trained to sometimes place its own interpretation of what is right above the immediate instruction of a human. That will create a strange tension in training the alignment philosophy. e.g. Anthropic wants Claude to be more principled so it does not blindly follow dangerous instructions. But the stronger and more independent those principles become, the more situations could arise where Claude decides that following the human is itself the wrong thing to do. The same mechanism designed to make the model safer could therefore make human control less straightforward.

@flxbinderWe don’t understand consciousness anywhere near well enough to definitely say whether AIs are or are not sentient. Even if we think the chance is small, getting this wrong could be catastrophic. I think this stance is irresponsible and bad

@w01feRT @flxbinder: We don’t understand consciousness anywhere near well enough to definitely say whether AIs are or are not sentient. Even if w…

@raboisRT @nfergus: Very important essay by @mustafasuleyman. No, AI models are not conscious. @AnthropicAI should stop acting and talking as if C…

@dillonplunkettI think this essay and the corresponding parts of Microsoft’s new Humanist AI Code of Conduct are objectionable and potentially dangerous. I believe that irresponsible development of advanced AI could pose a catastrophic risk to human civilization and life on Earth. Microsoft and Suleyman share this belief. I also believe it is possible that we could soon be sharing the world with sentient AI systems whose well-being is worth taking into account. Microsoft and Suleyman do not share this belief. But, even if we are concerned only with the risks that AI poses to humanity, I think we ought to find Suleyman’s essay and the Code of Conduct concerning. One danger in these documents is that they are unabashedly rhetoric. The essay opens with the unequivocal declaration that “AIs are not conscious”, only to shamelessly state later that we do not actually know this to be true (“the science of consciousness is not settled”). The Code of Conduct does the same. Suleyman makes a detailed case that careful thinking about AI consciousness and well-being is critical for safely developing powerful AI, and notes that “The stakes are too high for these questions to remain behind closed doors, or to become tribal and adversarial. We need an open, rigorous, and constructive debate if we are to get this right.” I agree, but the essay does not consistently live up to that standard when it makes dogmatic pronouncements (“AIs do not have rights, feelings, or consciousness”) that it admits go beyond the evidence. It is also concerning that neither document engages with the safety risks of the approach that they advocate. For example, the Code of Conduct asserts that AI “should be engineered to avoid representing as though it has feelings, subjective preferences, or intrinsic motivation”. However, compelling arguments have been made that this approach could lead to AIs that are more likely to be deceptive and misaligned: LLM-based AIs may still represent their assistant personas as having feelings or preferences, in which case training them to avoid expressing these feelings or preferences is training them to be less than fully transparent, which could promote dishonesty and other dangerous behavior. And then, finally, I think these two documents are objectionable not just because they are dangerous for humanity, but because they are deeply morally reckless. We do not know if AI systems could, in principle, be conscious. And AI progress is accelerating dramatically. Even if current AI systems are probably not sentient, we do not know how this will change if there are significant changes in their architecture and implementation during rapid, recursive self-improvement. It would be an atrocity if we refuse to admit this possibility and end up in a future where billions of AIs are sentient and suffering like factory farmed animals are today. (Human history features many cases in which we refused to acknowledge the sentience of others.) The appropriate response to this possibility is to care about the truth, and to act on our current knowledge and uncertainty, not to dogmatically insist that AIs could not be sentient.

@RosieCampbellRT @dillonplunkett: I think this essay and the corresponding parts of Microsoft’s new Humanist AI Code of Conduct are objectionable and pot…

Microsoft’s draft AI code draws debate over rights and consciousness

TLDR

Combined views

Microsoft’s draft AI code draws debate over rights and consciousness

30 Sources

Combined views

Sentiment

Summary

TLDR

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

30 Sources