@QiaochuYuan yes, and moreover, none of yudkowskianism, "ai alignment", "coherent extrapolated volition" make no sense outside the light that the entire project seeks succession to AIs / handing over power from mankind to AIs. @LRudL_ has written about this: https://www.nosetgauge.com/p/alignment-and-succession-the-two
Nic Carter argues AI alignment must serve specific interests
Yuan calls Yudkowsky's CEV a substitute for divine alignment.
877073216159.4K
Original post unavailable.
Sentiment
Positive users thank critics for clearly framing Yudkowsky's CEV as a God substitute in AI alignment, while negative users reject the consequentialist and utilitarian foundations as cultish or vacuous.
Pos
45.0%
Neg
55.0%
10 comments with sentiment.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Posts from X
Most Activity
Most Activity
BOOKMARKS3
RETWEETS23
this is a god-shaped hole in yudkowskianism. yud wanted to but could not say “aligned with god” so he had to invent a god substitute, namely “the coherent extrapolated volition of humanity”: not any particular person or group of people’s interests, preferences, or ethics now, which could be arbitrarily selfish or mistaken, but what our interests, preferences, or ethics collectively would be if we were smarter, wiser, and given unlimited time to deliberate and debate yudkowskianism requires as a foundational axiom that something like CEV is a meaningful concept that uniquely specifies the best possible values, because any alternative would force you to think about politics
“AI alignment” is incoherent on its own. There is no such thing as generic alignment. Aligned with whom? Whose interests? And why should globalized models be aligned to the normative views of one organization or committee?
REPLIES11
@nic_carter To maximize economic surplus
@johnsonmxe the child, doing his best to tense his body in the probability-of-human-extinction-minimizing shape
@nic_carter The moment you shed the idea that an LLM is a type of digital person or volitional being and think of it as a new category of software capability, the entire idea of "alignment" is either incoherent or it reduces to regular product safety + access/authorization + usability.
@BenNijjar Whose goals? Whose constraints? Teachers are allowed pedagogical flexibility, and there is decentralization at the state county city and school level. The teacher analogy would hold if there was a single person that decided every syllabus
@QiaochuYuan God is* that which releases tension. Tension is not only necessary in Yudkowskian ethics, but approaches a terminal value. This makes it difficult for Yudkowskianism to have room for God, and for rationalists to experience grace. *among other things
@nic_carter Alignment is not incoherent. It just means making an agent reliably follow a set of goals and constraints. We do this with human roles all the time. For example, teachers are “aligned” to educate children and keep them safe
“AI alignment” is incoherent on its own. There is no such thing as generic alignment. Aligned with whom? Whose interests? And why should globalized models be aligned to the normative views of one organization or committee?
@QiaochuYuan yeah, coherent extrapolated volition of humanity is one of the more incoherent yudkowsky concepts i have encountered. i guess either acknowledging politics, or honestly appealing to god, or the superior wisdom of your ingroup is more coherent
@letsplay14me Models don’t kill people, people kill people.
@paul__is__here @johnsonmxe this is hard-won knowledge. it's a little wild how many different experiences i had to have to be able to write that sentence
@QiaochuYuan I think this was Hegel's project? Acquire meaning by creating Christian/monist eschatology through a dialectic theory of history (past and future), that humanity's "geist" is pointed in one direction, and meaning is found by going to the destination.
@QiaochuYuan I mean, I think that if Omega stopped by the earth and offered Yud a guarantee that AI will be aligned to ANY political position that's inside the Overton window in the US, he would take that offer so fast.
@QiaochuYuan 🎯 - pretty obvious once you see it but hard to put into words, thanks for doing that QC!
@nic_carter AI should be aligned with me
Load more posts
There is no such thing as alignment in the abstract sense. There are simply queries that an organization does not want you to make, and outputs it unnaturally injects