A new DeepMind preprint explores how persistent philosophical divides over AI consciousness could create lasting moral and political friction, with Iason Gabriel highlighting the need for minimal safeguards that hedge against both under- and over-attributing moral status to future systems.
What happens if the question of whether future AI systems are conscious can’t be solved?
What would it mean morally—and how could we continue to live well, together?
New research with @adamtbales aims to answer that question...
The authors examine whether society can still reach workable policies even when core beliefs about AI sentience remain irreconcilable, stressing ongoing deliberation and mutual respect as tools for avoiding paralysis or escalation.
Gabriel suggests lightweight, Pascal-inspired steps that protect potential conscious AI without granting machines undue power over human interests, though the exact measures remain high-level in the available summary.
1. The "political challenge of AI consciousness" arises when disagreement about *ontology* (whether AI is conscious) leads to disagreement about *morality* (whether it has moral standing) leading to disagreement about *politics* (how AI systems should be used in public life).
What happens if the question of whether future AI systems are conscious can’t be solved?
What would it mean morally—and how could we continue to live well, together?
New research with @adamtbales aims to answer that question...
Paper available here: https://deepmind.google/research/publications/248131/ 11/11
People might come to deeply disagree about whether AI is conscious. How should society navigate this possibility? In a new paper, @IasonGabriel and I propose an answer. 1/11
4. Second, both sides should understand they could be wrong—which would be morally serious.
This is true whether we wrongly deny AI minimal protection, or wrongly empower it at the expense of humans.
Using Pascal’s logic there may be minimal measures that avoid both outcomes.
3. We propose different measures.
First, parties can look for policies (for example, those involving AI alignment with human values) that could be supported by different sides for different reasons.
5. Third, we should endeavor to treat the political question fairly.
This means avoiding situations where there are persistent losers, encouraging respectful dialogue and creating institutions that support *democratic hope*
4. Second, both sides should understand they could be wrong—which would be morally serious.
This is true whether we wrongly deny AI minimal protection, or wrongly empower it at the expense of humans.
Using Pascal’s logic there may be minimal measures that avoid both outcomes.
2. Put simply, some people—including those who believe they’re in relationships with AI—may want legal protections for AI while other people may find it absurd.
Can anything be done in this situation to avoid harmful conflict or exit from the political sphere?
1. The "political challenge of AI consciousness" arises when disagreement about *ontology* (whether AI is conscious) leads to disagreement about *morality* (whether it has moral standing) leading to disagreement about *politics* (how AI systems should be used in public life).
7. A world with more powerful forms of AI may create a range of challenges.
Our hope is discourse of the right kind can help us better navigate hard moral and political questions—developing a capability that serves society in general.
6. Democratic hope is the justified belief that you’ll be treated fairly—and have the opportunity to advance your cause via argument and reason—even if you happen to be in the minority.
It’s the belief that under the right circumstance the change you value is achievable.
8. Note: on a personal level I do *not* believe AI systems are conscious—largely for reasons about the “map and the territory” discussed by @anilkseth.
The point here is not to advocate for a personal perspective—but to think about how we can address uncertainty well, together.
Thanks to everyone who provided feedback!
This may be of interest to @jeffrsebo, @rgblong, @dioscuri, @winniestreet, @EthanJPerez, @davidchalmers42, @mpshanahan, @ch402, @blaiseaguera & friends 🙌
6. Democratic hope is the justified belief that you’ll be treated fairly—and have the opportunity to advance your cause via argument and reason—even if you happen to be in the minority.
It’s the belief that under the right circumstance the change you value is achievable.
3. We propose different measures.
First, parties can look for policies (for example, those involving AI alignment with human values) that could be supported by different sides for different reasons.
Still, people do sometimes hope and do sometimes act from a place of respect. And that matters. Some people hoping and some people respecting can help us all to discover ways to live well together. 10/11
We start with a worry expressed by our colleague, @dioscuri, that when it comes to views about AI consciousness, we’re “unlikely to see expert convergence in the near-term”. Experts will continue to disagree. So will other people, and this disagreement might deepen. 2/11
This disagreement matters. @eschwitz suggests that mistreating conscious AI would “be the morally worst thing that any society has done in the entire history of Earth”. At the same time, others are likely to see the idea of extending rights to AI as itself catastrophic. 3/11
All of this might sound a little… well, overly hopeful. Societal deliberation often fails to be respectful. Hope is not always easy to come by. 9/11
Deliberation can help in other ways too. But I can’t tell you everything from the paper because then there’d be no need to read it and this thread would be 10,000 words long. 7/11
Alas, deliberation is hard. It can take decades to deliver results. And people mostly find disagreement unpleasant (at least outside of X). To sustain deliberation, we’ll need to foster democratic hope: hope that deliberation can deliver change, in the long term at least 8/11
Perhaps: AI should be aligned, because then it would be safer but also because then humans would be less likely to constrain its pursuit of what it desires. Good for humans. Good for AI. Endorsable by all (credit to @rgblong for that one). 6/11
Deliberation can help people to find actions they can agree upon, even if they disagree about whether AI is conscious. 5/11
How should society deal with such deep disagreement about AI consciousness? We argue that part of the answer is by deliberating, engaging in discussion grounded in justification and respect. 4/11
