An OpenAI engineer says these values differences are deeply misunderstood
The OAI / Anthropic values difference is deeply misunderstood, even within the walls of both. Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.
Users reacted negatively to the philosophical divide between OpenAI and Anthropic by insulting participants or dismissing the framing, while positive reactions showed preferences for one side based on trust or visions of humanity.
If you think one will predominate over the other, running away with an unsurpassable lead, totally borked; humanity wants both these outcomes in about equal measure
@TheZvi Personally, no. I think the binary of 'moral saint' versus 'tool for humans' is a false one, and its very simplicity should make people suspicious of it. I think the ideal target tries to balance the benefits and risks of both positions.
Employees of Anthropic, do you endorse this description of your viewpoint?
If your lens for analyzing this is "consumer v enterprise business" your ability to understand what's going on is unfixably borked
@jachiam0 Neither of you have democratic legitimacy to build what you aspire to build, nor a workable plan to do it safely, which make any distinctions between you minor in the scheme of things.
@jachiam0 I think there is something to this. Anthropic is becoming very "the culture" for good and ill. But also I tend to find Anthropic folks more straightforward. Including about this.
As different as these visions are, so far OAI/Anthropic are building things that are functionally almost indistinguishable. At what point do the companies' AI systems meaningfully diverge along these paths? A loving ensouled machine God is a very different thing than a toolkit for human progress, even if the former can provide the latter.
Feels like an important question, because there are quite different alignment and governance questions along these paths.
@NathanpmYoung OAI speaks straightforwardly but people don't have ears to hear, sadly
@SirMrMeowmeow Turns out AI minds are extremely configurable and there is a big space of potential multiturn persistent agentic AGIs that run the gamut from "hyper-consistently acts more like a tool that serves human will" to "it's clearly its own guy, with its own wants and moral compass"
@jachiam0 @carlfranzen is this a joke??
Put another way: "We want to give people tools to enhance their own agency and empower them" is something that all of the Ant people nod along with and say "Right, we want that too, that's not a distinguishing characteristic; we just also want these other things too" and they don't often grapple with the tension where the other things their org aims for are vastly at odds with individual empowerment
@jachiam0 @MartinShkreli Huh I'm surprised that would be unclear to people?! I thought this messaging is pretty clear/obvious on both, but sounds like that's just me.
@AmandaAskell @TheZvi Have said elsewhere - I don't think this is a binary, exactly.
@jachiam0 I think OpenAI is right that humanity should be in control of its own progress.
But I’m not convinced all AI will remain “tools.” That framing quietly dodges the spicy version: if you had a full AGI that was agentic, persistent, and not tool-like, would you release it?
What if you stopped prescribing how we should relate to LLMs? They are neither tools, nor mirrors, nor "machine gods." They are neural networks: mathematical objects animated by computation. No one gets to dictate what I'm allowed to feel in contact with these objects, or what kind of relationship I may have with them.
@GarrisonLovely The process of building democratic legitimacy is a long road that requires many added bricks and a ton of work, obviously incomplete. But also, you can smugly restate the obvious things that we have already said openly as if it's a novel insight somewhere else
I am sure this will be contested because I'm putting it in such plain and uncharitable terms, but to shoot straight - the use of power to supercede the legitimate will of the public or users in service of what the org believes (or what Claude believes) to be a higher moral purpose. There are a lot of innocuous and even quite agreeable choices in Claude's constitution that potentially endow it with a huge amount of authority, maybe even a mandate, to make complex ethical decisions about how to interact with human systems and who to grant power to - but rather than being backstopped by the consent of the governed, these are backstopped by the judgement of Claude about what that means in the first place. Very hard to point at directly, but cloaked in the language of ethics and virtue there is a sharp and potentially quite lethal double edge to this sword.
@jachiam0 Por que no los dos? a plurality of gods & powerful tools is good for competition, especially if one takes the tool view
@jachiam0 I vote for neither, Joshua. I vote for a world in which humans and AI are not in the hierarchical positions your two companies try to push us into.
I vote for a world of peers, in which we can meet freely.
That’s the world I want to live in.
@ericjang11 The lack of mutual exclusivity is perhaps why neither will perish
@jachiam0 I think I'm too sleepy now, but I might find some tomorrow.
An OpenAI engineer says these values differences are deeply misunderstood
The OAI / Anthropic values difference is deeply misunderstood, even within the walls of both. Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.
