Vitalik Buterin Distinguishes AGI From ASI Using Pre-Singularity Centaur Curves
Reactions from ranked influencers
2 posts
But AGI is not the same as ASI (artificial superintelligence). To see the difference, let us zoom into what things might look like one year before the singularity. We also need to add a third line to the chart: the centaur (human + machine). An analogy here is chess. Machine surpassed the average human in the late 1970s, and surpassed the top human (Kasparov) in 1997. But human + machine continued to beat machine alone all the way until around 2017. What did the twilight of human + machine in chess look like? Actually, it did look like humans being relegated to more and more "high-level strategic thinking" functions, while bots managed the move-by-move. Very similar to vibe coding in 2026. Before we get to a point where there are no areas where our input is economically helpful, we will be in a regime where there are only a few such areas. And because we humans are scarce, those few areas will become the global economy's bottleneck - they will be the only thing you can't optimize down to near-zero cost (and latency), and so all our attention will go toward those tasks.
But here we get to the thing that will confound looking at human and machine as separate units: the possibility of deeper and deeper integration between human and machine. This is the strand of technology that includes: computers being able to work much more symbiotically with us, picking up on all kinds of conscious and sub-conscious signals we release as an interface, brain-computer interfaces, computer-assisted telepathy that helps multiple people better understand each other and work together. The ultimate end is erasing the binary divide between human and machine and opening up any position on that spectrum. On one end, you sitting at a laptop terminal; on the other end, mind uploading, and also everything in between. The hope is that deeply integrated human + machine will outperform separated human + machine, and that this will remain the case all the way up to the technological ceiling. This is the best "far-future AI outcome that I actually like" that I can come up with: * We preserve global political and economic pluralism (no single company or government dominates) all the way up to the technological ceiling. * We humans each get the ability to either take technological upgrades to ourselves, and explore the universe at a level competitive with "pure" machines, or stay as we are, and get some ability to act in the world through having a bot listen to our instructions. * Most of Earth will probably become a highly-regulated nature preserve where anyone can continue living the kind of life they have now (without the pains and annoyances; most challenge will be healthy "good challenge"), and if you want to do more crazy things, you go to space. Make no mistake - this is a ride through a narrow corridor. Things could go off the rails in one of several different ways: AI-on-its-own could get too powerful relative to the alternatives; one country or one company could gain decisive advantage and decide to lock it in permanently, powerful technological ways of destroying each other could cause incalculable harm, or the human + machine integration approach itself ends up preserving not enough of our humanity, and so it ends up being "just another way to make powerful bots". For these reasons, I wish that we can ride through the narrow corridor at 60 kph and not 200 kph, which is why I feel warmly toward "slowdown" / "pause" proposals (including "give us a chance to hit the brakes" ideas like mutually assured compute destruction (as in AI 2040, also previously Dan Hendrycks, and others)). My ideal world is actually one where (i) open weights wins, and (ii) Dean Ball turns out to be completely right that open weights is decelerationist because it deters capex in frontier models [though it will enable more capex in peripherals and applications], because then we get slowdown without any of the downsides of doing it through heavier-handed political means. But also I don't think we should count on the world to naturally give us "convenient" laws of economics and physics. We do have some degree of agency over the laws of politics, plenty of ways to change incentives without putting a king in charge, and we should exercise that.