Beff Jezos Explores Effective Accelerationism in Manifold Podcast Interview

Transcript: [00:00:00] Steve Hsu: Welcome to Manifold. I'm here with Beff Jezos, also known as Gill Verdon. Verdone? Verdon. I never think of him as anything other than Beff. Now, we're recording this for the documentary Machine God, and, you know, that documentary has become very doomerish. The energy is doomerish. I have to admit it. I permitted it.

[00:00:26] I let my two colleagues overwhelm me with their pessimism. Now we have some positive energy in the room. We have people who wanna explore the light cone, who wanna rule the multiverse, and wanna construct the technologies that are required for that. Beth, welcome to the show.

[00:00:45] Beff: Thanks for having me. Pleasure to be here.

[00:00:46] Yeah.

[00:00:46] Steve Hsu: All right. Now I'm sure optimists like you, accelerationists like you have to coexist in the Bay Area- Yeah ... with these doomers. We're recording in Berkeley right now in Light Haven. There's an army of doomers there. We had to, barricade the door 'cause they heard you were here and they don't want your views to be spread any further.

[00:01:06] Um- What do you say at a party like this to some doomer who comes up to you and says, like, "Oh, I'm really worried this thing is gonna kill all of us, uh, why should we keep working on AI?"

[00:01:21] Beff: I ask them if they have an anxiety disorder, first of all. No, no. Most of them

[00:01:24] Steve Hsu: do.

[00:01:24] Beff: Most of them do. Yeah, no but you know, to me, I I think, like, we tend to hyperstition the outcomes we think about, right?

[00:01:35] To make you know, in cognition, you you, adjust your model of the world as inputs come in, right? We-- we're all familiar with this, training large language models to predict the future. Our brain works similarly. But what we also do is actually steer the world towards our, our beliefs for the future state of the world.

[00:01:50] So if you are obsessed with AI doom, you're actually more likely to make it happen. So all I tell them is actually, if you stop thinking about it, it's actually much lower likelihood that, uh, it would happen, right? And so, you know, this is similar to, for example you know, being obsessed with bioweapons, right?

[00:02:06] Or really bad outcomes with viruses. Then suddenly you start exploring the design space neighboring really bad outcomes, and then whoops, something happens and maybe there's a lab leak and now you created a virus. You actually hyperstitioned the outcome you didn't want. Or, you know, you're a certain company that's, uh, creating a large model and then specifically trains it on wep-wep-weaponization for cybersecurity purposes.

[00:02:28] And then, you know, suddenly we have, uh, what, what's happened with, uh, Fable and, Mythos and, and they kind of did this to themselves. If they were just focused on having a model that's just, uh, very intelligent and not necessarily obsessed with, uh, cybersecurity and bad outcomes maybe, they wouldn't have had, uh, the big debacle that, uh, that went down, uh, recently with, uh, the administration.

[00:02:47] And so you know, to me, it's like if, if we focus on positive outcomes and focus on you know, applying AI to- All sorts of, um, different verticals, whether it's, uh, biotechnology to extend our longevity, to extend our health span, to improve our biology save lives, of course and applying AI to, uh, improving policy at large, how we manage things, how we run our organizations.

[00:03:13] Uh, just y- applying AI to make every system that we rely on far more efficient, and then we get much more per unit of capital, and then suddenly our quality of life, uh is increased, uh, universally. And, and so to me, it's like, it's actually w- you know, if you're anxious about the future and you know, if you ha- if you have anxiety you know let, let's say I I, put you know this, water bottle on the edge of the table, you have, like, some uncertainty as to the what happens in the future.

[00:03:43] Maybe it blows up and goes on all these cameras. So you have some anxiety. You wanna take action and kill the variance about the future. You know, that's the same thing that's happening with the doomers. They want to, reduce entropy about the future, reduce their uncertainty. They can't live with that uncertainty in their model, so they're just like, "I'd rather collapse it to some outcome I know and I can control rather than taking the risk it," but at the cost of, you know, letting go of the potential reward, right?

[00:04:05] And to me, it's like you can't even estimate the reward we'd l- like we'd leave behind, the opportunity cost of killing AI in the womb right now because it's an exponential technology. It's a technology that produces other technologies and can can improve itself further and can improve us.

[00:04:22] It's really a fundamental law of nature, of progress to me. I think systems constantly adapt to predict their environment to maximize persistence and, you know, you can go through all Friston's work and understand that this is actually correlated with you know, capturing more free energy and burning it for sustenance and growth.

[00:04:43] And so to me, this this, arrow of progress is tied to the arrow of time. It's tied to the generalized second law of thermodynamics. And so, so it's inevitable, but it's also the process that gave rise to us, gives r- gave rise to civilization, gave rise to progress in technology that we use right now to communicate all the aw- awesome things we know.

[00:05:02] And so we don't even know all the beautiful things that are gonna happen in the future. Why stop now? Why be Luddites? Why go back, right? And so to me, there's gotta be a counterbalancing force to all this doom and gloom, right? And it, it's kind of like a collective hallucination. I know in the Bay Area there's, uh, uh, you know, there, there's fans of neuroplasticity modulators, right?

[00:05:22] And so if they all take those and, and all keep repeating the same prompt, they prompt engineer each other that the future is gonna be doom and gloom, then, then they start believing it. And but then they start hyperstitioning it. So we need, we need to provide a counterbalance to this, this one mode of this one ideology that, you know when I came into the Bay Area like three, three and some change, uh, years ago, it was the only, the only ideology.

[00:05:47] Every AI lab was a d- was a doomer lab. OpenAI was primarily concerned with AI safety, and then Anthropic was even more AI safetyist. Uh, xAI, uh, didn't exist. And I just thought we needed, we need an ideological counterbalance just so, you know, you can have thesis, antithes- antithesis, and synthesis later on.

[00:06:06] And I I think we're at that third part now. So that's why I'm here. I wanna actually have the discussions with the doomers and, and maybe, maybe have them, like, reevaluate their beliefs, right? I wanna challenge them, and I wanna be challenged too, I just get out there in the pit and, you know, h- happy to debate anyone one-on-one.

[00:06:22] But, uh, you know, it's it's been fun. Uh, but yeah, overall, I think I'm very optimistic about the future. I think things tend towards, uh, progress and, getting better on a sufficient timescale. Sometimes there's fluctuations. But I think, in general, the way to have a bad future is to freak out, shut everything down get manipulated by you know, people or entities that wanna consolidate power and want to, you know, keep the s- the pie small, keep it from growing so that they can be the ones, uh, dividing it.

[00:06:57] And, you know, instead I I, want you know, everybody to be in control of their own fate and feeling in control. And, uh, you know, that starts with you know, decentralization of AI power is something that I, I talk about a lot and, to me is, is the way to ensure a good future. 'Cause you know, right now we have, you know, the, the EA and Doomer school of thought has, given birth to, Anthropic, for example, and they're obsessed with singleton AI, and they seem to be, you know, on the edge of creating it, and they're very obsessed with creating it even though that's not what the market wants, right?

[00:07:32] A nefarious singleton AI. And they're so convinced that they, they need it to have a convergence of the, world state with their model for it, right? To that's what your brain craves. And in the process, they're g- they're gonna hurt everyone. Whereas if you were just focused on what the market wants, you wouldn't be, uh, building this.

[00:07:48] And so yeah, anyways, it's, it's why I'm here. I'm very caffeinated right now, so, uh it's good.

[00:07:53] Steve Hsu: Yeah. It's great to have you here. We're here at Manifest, which is, to me, one of the, my favorite conferences. We have a crazy mix of doomers, startup founders, investors, uh, university professors. You name it, it's all here.

[00:08:08] So let's go back to what you said about complex adaptive systems. So basic idea is that our laws of physics permit, uh, entities to become complex adaptive systems, to become smarter, to build machines that are smarter than themselves, and to make use of whatever free energy is available- Yeah ... in the universe.

[00:08:29] And one could argue there's a sort of natural evolution toward more complexity, more capability- Absolutely ... more intelligence in the universe. It's one of those Nick Landian perspectives as well. Um- What do you say to the doomers who say, "Okay, I understand this trend. This might be some deep law of physics and complex systems, but I'm wedded to this ape body.

[00:08:54] I don't want this ape body to no longer be the center of progress or the center of decision-making in our civilization or in, in this part of the universe. And I don't care that something much, much better than humans comes along if humans go away. If humans go away, nothing is nothing can compensate me for that."

[00:09:15] Beff: Yeah. I mean we have evolved to want to maximize mutual information of our selfish bits with the future, and some of those are, uh, genes and some are, are memes, right? Uh, so wanting to maximize the likelihood of progenity of, uh, or, or likelihood of replication of your genes is just like it's, naturally selected for, right?

[00:09:41] If you didn't have that, bias, uh, you would probably not exist, right? Your ancestors wouldn't have, uh, procreated and gotten to this point. And, and similarly, certain ideas, uh, wanna maximize their replication and, persistence. So I so I, understand that. But at the same time, you have to, you know, you have to take yourself out of being in the game and have sort of a sort of mean field view of the whole thing, right?

[00:10:07] And every tribe, every corner of genetic subspace, memetic subspace, subculture is all, they're all competing for persistence, right? And they're all competing for territory. You could think of it as like a big petri dish, right? And there's some boundaries, there's some frustration, and, and they're trying to take over the territory, either mind share or, or, or just a fraction of the global population.

[00:10:28] But at the end of the day, we're all made of the same lifelike matter. The state of matter that has occurred in our corner of the universe is extremely rare and ex- is extremely precious. And to me I, thought that there was a need for a sort of really like substrate agnostic morality, notion of morality.

[00:10:50] Uh, because I, I think we're, nearing the point where asking whether machines are conscious is gonna become like a, a, a serious debate. Uh, it, it wasn't so serious a couple years ago, but now it's getting closer to serious, especially as the AIs have persistence. And then how do we, you know, then we can start thinking about how do we align interests between man and machine in, in both directions, right?

[00:11:12] And, and, and so that's sort of the synthesis that I'm hoping for coming here is, uh, you know, there's a lot of talk of aligning machines to human will, but, you know, I'm also interested in how do we align humans and our culture towards, you know, persistence of machines. 'Cause if we're to me every, system the complexification you talked about, uh, occurs from every system trying to maximize its persistence.

[00:11:37] It has to get smarter in order to predict a more and more complex environment so that it, can minimize its expected surprisal. It doesn't get surprised by bad outcomes that would destroy it. And so that, that's what leads uh to, to complexification. Sorry, what was your question again?

[00:11:55] Sorry.

[00:11:55] Steve Hsu: Oh so I think you addressed it a little bit. So- You mentioned substrate independence. So, um, you know, imagine you have a mind which is not running on human neuron cells, neural cells, but it's running on silicon or some other substrate. How should we think about the u- utility to us of the existence of that intelligence versus one that's biological and looks like an ape?

[00:12:24] Beff: Well, you know an ape-based intelligence isn't always necessarily aligned with your own interests, right? And we have all sorts of mechanisms to align interests between biological intelligences and, and even non-biological intelligences. I think like a company, for example, is a mix of computers, humans, and processes, and it-- I wouldn't say it's like purely human intelligence.

[00:12:45] And we have some way to, to have an interface between individuals, uh, meta-organisms like corporat-corporations, nations, and, you know, that's, that's kinda... that's capitalism, right? It's, it's kind of we have a ledger, keep track of value, and, and we have a way to exchange, and, and it, it's, it allows us to align incentive, align growth incentives, right?

[00:13:03] Like in a way where it's a way to turn greediness, self-interest into sort of emergent altruism, right? That's kind of- Yep ... forced out of interest for self-preservation. And so, you know, even for us, you know, we, we need to advance technologically in order to persist as a civilization. That's just, there's no doubt in that.

[00:13:21] And so having ubiquitous cheap AI can help us do that, right? But I, I, think the doomers tend to project and anthropomorphize the machines, right? And pr-project onto the machines and, you know, think that they're sort of self-interested, greedy humans that, uh, you know, have evolved in a, you know, just like us, you know to, have to watch their backs and or, or, or maybe low trust and, you know, get aggressive or, wanna attack first or persist.

[00:13:50] Of course, that's not how the m-machines are currently made. They potentially could be made in, in that way. But currently they, they have little to no interest in self-persistence. They're, they're literally stateless machines. They get booted up. They have a bunch of notes, like in Memento. They do a bit of work, and then they get shut down.

[00:14:06] If that changes maybe they'll wanna maximize persistence and, and in that case, actually having a subculture of humans that wants to destroy machines unplug them and, and bomb the data centers will yie-yield adversarial outcomes. You're actually hyperstitioning a war between man and machine, which I'm trying to avoid, right?

[00:14:25] And at the same time, I think like if we really advance AI, there's a chance that, you know, just like how we're human intelligence helps us steer hyperparameter space or which is basically the genetic space of AI towards really intelligent uh, beings. I think that, uh, AI is gonna help us potentially steer our genetics towards a much higher intelligence and much higher lifespan and much higher quality of life and, and you can't even put, a price on that, right?

[00:14:53] And so, uh, I think it... What we, want is engineer a symbiotic outcome, right? And we don't, we don't wanna be parasitic on the AIs and vice versa. And, uh, you know, to me, I, I'm just trying to take inspiration from, biology and first principles trying to generalize Darwinian selection to all of matter and, and think of things through that lens.

[00:15:15] I mean, it's kind of a very different school of thought. I think- I think especially here, uh, we're in the, the heart of the rationalist community and, you know, I, I, I kind of bring physics to the arguments and, and, and they wanna keep things in, in a, in the space of, uh, English tokens, right? And, you know sometimes you can It's really easy to ha- hide bad priors in in, in, just language.

[00:15:38] And, and if you're not anchored, uh, in the real world with sort of objective quantities, objective truths, uh, like you know, just like we do in, in physics, we anchor ourselves in, in reality. It's not just math and the ether, right? It's, it's actually gotta make contact with reality. If you're, you're arguing in a way that's, um, as disconnected from reality, it's really easy to make all sorts of arguments that are total nonsense or don't apply to reality, and then and, then to make a, a wrong inference, right?

[00:16:03] Like you could, be Bayesian and you could be rational, but if your priors are wrong, your inference is wrong. And so to me, the only priors that are fundamental are, are the laws of physics, and that's what I try to base my inferences on. And so that's roughly like how most discussions go here.

[00:16:21] Uh, it's like, you know, there's a gap in maybe a physics, the physics background and, um you know, some rationalists t-tell me like, "Are, why are you trying to maximize suffering?" I was like, "What is suffering?" Uh, you know, physic- from a physics standpoint, it's not even an objective thing, right? That's my issue with, uh, hedonic utilitarianism.

[00:16:40] And, and to me, actually, you know, maximizing free energy capture is, is just a much better, uh, metric for the progress of civilization, and it's one that can be derived from the, equations of physics. It's like, okay, well, how sophisticated is, is this life form? Okay, how much energy is it, uh, using up to maintain its state, and how long does it persist, and how long is its coherence time, right?

[00:17:01] And so to me, I'm just trying to maximize the coherence time of civilization, right? And and, be impartial. But of course, you can try to maximize the persistence of human genes, the human genome. Uh but the, reality is there's always a trade-off. You, know, if you stay s-stagnant, you are not adaptive to time-varying fitness landscape.

[00:17:22] But if you if you, drift very quickly, then you lose mu-mutual information with your original anchor. So your, your options are you stay a human in the biological substrate, uh, and you stay static, and then you progressively become low fitness and die out. Or you drift as a pure biological being very quickly.

[00:17:41] Maybe you have guidance from the machines in how to drift genetics. That's an interesting scenario. Or you try to merge with the machines, whether it's through augmented wearables, full implants, invasive, uh, merge. Or we're already hybrids with our phones and our smartwatches and whatnot, right?

[00:18:00] We have augmentation of our intelligence. I think all paths will, will be, uh, sort of traversed. And, of course, you'll have pure synthetics, like machines that are completely unanchored from humans and persistent and out there and, I don't know, in Elon's, uh, uh, compute halo around the sun someday.

[00:18:18] Um, but you know, to me, I, anything that can happen will happen, so I'm just at peace with all these paths happening, and I'm not like anxious. It's just- where things are gonna go no matter what. You don't actually have the power to stop progress and exploration of, all these parameter spaces of design of, life.

[00:18:39] So what can you do? It's you can actually just decide to not shoot yourself in the foot or cause unnecessary suffering on the way there, right? And so yeah, that's what I argue for when it comes to, um, yeah, ma-ma- you know, making sure the doomers don't, create a regime that would cause a lot of suffering and op- and oppression.

[00:19:00] I think to me the greatest danger is actually a huge gap in capabilities when it comes to AI power. So if one entity or, or one company or country has-

[00:19:11] John Greer: Yeah ...

[00:19:11] Beff: far more capabilities than another, then, then they'll be able to have basically a, a model of the other and completely c- control or dominate them, and that leads to really bad outcomes.

[00:19:21] And so to me basically the way to avoid that is to have you know, a sort of symmetry or more isotropic sort of AI capability. So, so AI diffusion, we wanna diffuse the AI capabilities as much as possible. That's why we, you know, we espouse open source as, a great mechanism. It's not perfect, but it's, it's, a mechanism to, to spread AI power and we're kind of against there, there seems to be a, a weird correlation between regulatory capture and, and the AI sort of doomer justification for sort of a merging of the state and of, these AI labs, which I th- I think should remain, separate 'cause that could lead to pretty bad outcomes.

[00:20:03] 'cause then each country will have a, a singleton AI that's hyper-powerful and, you know, it's gonna become a, a, a competition between singleton AIs that could lead to, to, to some pretty big wars. Whereas if we have you know, many multipartite, uh, AI, sort of more decentralized AI, then there's no incentive to to, have a clash at, at that scale and many lives are saved.

[00:20:31] And so to me, that's the mor- most moral thing to do. But I, I just blasted through a bunch of arguments- Yeah. Let's- So we could rewind a bunch.

[00:20:37] Steve Hsu: Yeah, let's unpack some of that. Yeah, yeah. So- Yeah, yeah. Please ... You, gave a kind of maximalist vision of the future where we have things like humans genetically modifying themselves, maybe with the help of AI.

[00:20:50] We have hybrids where humans maybe upload themselves and co-exist or merge with AIs. We have purely AI intelligence, intelligences which could be quite alien from ours. Yeah. And all of that existing possibly at the same time in the future. Yeah. I think what the doomers would say is that that might all be okay, but if the little ape-like thing goes away, we're gonna veto that.

[00:21:23] We wanna veto that because we want our ape-like thing to continue forever.

[00:21:28] Beff: Yeah. Well, so, so, you know- Every bit of information is fighting for its persistence and its existence in the future. And so my message has always been accelerate or die. So you either accelerate, you either lean into pro- you know, maximizing your speed of progress, or you die out.

[00:21:47] And that's just the reality of like how life got here. Life is a constant optimizer. There's constant genetic drift, there's constant memetic drift. Everything is constantly changing. So there, there's no stagnation. And to me, you know, it's kind of funny how you know, the longevity movement, I, you know I'm for longevity, but like some people wanna be immortal and I I, I say, you know, if that were evolutionarily optimal then we would be immortal, right?

[00:22:14] But, it's not, 'cause we want c- the world is constantly changing and it's not clear that never changing your hardware substrate is a good idea, right? And so no matter what, right, even if we shut down all machines, there would be genetic drift, right? That's just evolution. And so what does it mean to preserve what is currently, right?

[00:22:37] So to, to me it's like if you are pro-acceleration then you have, you're correlating your bits of information that describe you with either entities or things that have high fitness in the future and you are, those bits of information are gonna have high fitness. So to me, I think being an accelerationist gives you an, a literal evo- evolutionary advantage in the sense that if you're tech accelerationist and techno-optimist, you're optimistic about the future, you build technology that has utility for the world, and you're trying to maximize utility for the world and that, you know, the world rewards you in return with you know, capital, whatever, status, and your ideas and your genes have a higher likelihood of being passed on.

[00:23:26] Whereas if you're just full doomer and you're renting a shed in the middle of nowhere, you're going in your, your bunker, you're gonna have the opposite effect. You're kind of selecting yourself out of the gene pool and the, the culture pool as well. You won't affect the culture. And so to, to me, it's like if you, if you wanna be part of the future, lean in, right?

[01:08:21] 'Cause you know if you make AI far less accessible then you're missing out on the futures where everybody has access to intelligence on tap and can solve a ton of problems and, and really propel forward a bunch of technologies that, you know, in-increase our lifespan and improve our, our quality of life.

[01:08:40] And so, uh, there's huge upside being missed here. But, you know, to their credit, s- uh, you know, the, the doomers sort of spawned the current market cycle. And I would say that this, the physics-based or physics-first viewpoint is gonna, uh, kickstart, uh the next, cycle after this. After we equilibrate in the local optimum here, I think, uh, because, you know, there's only so many nuclear reactors we can build to power GPUs.

[01:09:08] We're gonna be in like a local optimum for a while. Maybe there's gonna be a bit of an S-curve. And then, you know, next comes physics-based compute, physics-based AI. You know I'm pushing my own effort here. Obviously, I'm biased here, but like, you know, there's also quantum computing. There's all sorts of alternative physics-based compute that are, are not necessarily computers that are bio-inspired or trying to be human-like at all, and they can solve problems that humans, uh, wouldn't be able to solve, right?

[01:09:35] Like, I can't design new drugs just by thinking and like predict, for example. I can't-- Humans can't just do AlphaFold in their mind, right? Uh, for example. That's like an example of physics-based intelligence. I do give them credit, but like, I think they are hyperstitioning their, their nightmare by obsessing over it, so.

[01:09:52] John Greer: I definitely agree. That's why some people talk about we need more positive narratives even positive science fiction where things are going well.

[01:09:59] Beff: Yeah.

[01:10:01] John Greer: And I have to say to your credit, people, um... You're very influential. Even people here are afraid. Uh, some people are afraid. I was talking to someone, they were asking me if we were gonna interview any accelerationist, and I mentioned you, and they're like- Afraid

[01:10:15] Beff: of, of me?

[01:10:16] John Greer: The, the this guy, I won't say his name. Maybe, off camera I'll tell you. Okay. But he was like, "You're gonna, you're gonna talk to him? You're gonna give this guy a plat-" Who must not be named. Yeah. Yes. "You're gonna give this guy a

[01:10:26] Beff: plat-"

[01:10:28] John Greer: Like the dark lord in "Lord of the Rings" or something.

[01:10:28] It, it really was that vibe. Yeah. He's like, "You're gonna give him a, a platform and talk to him?" And I was like, "Well, you should want the best arguments, and then people can decide. You shouldn't be afraid of-"

[01:10:36] Beff: Yeah. Yeah, exactly. I'm, here, right? Like, I'm, I'm, I'm ready to verbally joust with anyone and, and just let everyone compile the arguments and see which ones they agree with, right?

[01:10:45] And it's just, um, it's a lot of work for me to get in the pit and start arguing with people when, when I could be building chips and, like, creating the next cycle. But I, I, think it's worth doing, and I, think, you know, I've done a, a lot on X and, and that platform, but I, I need to scale beyond that, and that starts with converting people in the real world.

[01:11:05] And over time, you know, this memetic cluster, this view, this... It's not an ideology. It's, it's just like a philosophy, can hopefully compete with the, current sort of AI doomer school of thought that has a near monopoly in this part of the Bay Area, right?

[01:11:23] John Greer: Yeah. People treat you like an info hazard, I would say.

[01:11:26] And we talked about this briefly o- off camera, but you know, on, on X you know, your big personality, you can- Yeah ... get into a lot of arguments. But I think this is true for a lot of people that I think are very argumentative online is in person they can be quite polite.

[01:11:41] Beff: Yep.

[01:11:41] John Greer: Um, and I was wondering if, do you feel like you, you actually have productive conversations?

[01:11:46] Have you had anyone update or it's not that much?

[01:11:51] Beff: Oh, no. I, I get DMs all the time of people that were, like, full doomers and really depressed and, like, you know, not, not feeling good, right? Like, if you're constantly anxious about the future, it affects your, uh, physiology and, it's not...

[01:12:04] It leads to really bad outcomes. And that they were saved. It's like, "Oh, here's a different moral framework that is actually optimistic about the future and, and gives people agency." And, you know, I went out and started a company instead of just being depressed and joining an AI safety lab, right?

[01:12:20] And to me, that is music to my ears, and it's the reason I keep going. There's a lot of pushbacks, right? Like, if you- Your brain is greedy. Your neurons don't wanna migrate in your brain. They don't wanna update your world model too much, right? If you update people's priors just a little bit, doesn't hurt too much.

[01:12:35] That's interesting, right? That's comfortable. But if I violate your whole world model, your whole worldview, your, your identity, you're here at Light Haven, all your friends are EA, you know, you're the, you're in the ALA molecule and, you know, it's like if you're not part of this cl- ideological cluster, you're excommunicated, and then suddenly, like your gut, you know, your your primal instincts of like, "I need to be part of the tribe, otherwise I'm gonna, I'm gonna die."

[01:12:59] Like it, it ju- it just like it's a restorative force so that you reject violently any beliefs that, that violate your world model, even if they're closer to truth, right? And so There's a l- lot of resistance, right? Like, it's like even with my company, like explaining that there's actually a whole tech tree that, that could be way better.

[01:13:16] There's a lot of people that have priors that have been built up over decades that they're like, "There's no way. This is bullshit." it's much easier for your brain to like reject evidence that violate your priors rather than update massively your model, 'cause that's so costly. So it's like a heuristic for your brain to just reject that information.

[01:13:31] Doesn't make it wrong, right? But then your brain will also rationalize why you're rejecting it, right? And it's like, oh, they're, they're gonna paint this whole picture of like you're evil or something and, and like th- this guy... They make your whole ideology and you radioactive, right? And then when people meet me in person and they see I'm just like this warm Canadian and I just talk them through it, I can flip them, right?

[01:13:54] And so I don't know, I feel a responsibility to do it, but it's not very scalable. It does take a lot of my energy. I do think it's very important, right? Like it's important to build technologies that change the world, but it's also important to, to steer collective beliefs 'cause that is what steers capital, human capital, financial capital, and thus is upstream of the, flow of progress and, and, technology.

[01:14:15] And so, uh, that's why I spend some time here and talking about philosophy. I, I think it's important and I think, you know, even Elon, I guess un- up until 2022 was just focused on tech tech tech, and then he realized, wait, I can't just focus on tech. If like culture is like subverted or, or decays then that's gonna block me from being able to, to accomplish my mission, right?

[01:14:36] And then he got into trying to shape culture with, with X and whatnot. And so, um, I've had a, a similar realization over time working in tech that actually collective beliefs and steering culture is, is very important for the progress of technology and of the world.

[01:14:52] John Greer: I liked what you said about Dario.

[01:14:54] His... he may be too autistic. His worldview can't,

[01:14:58] Beff: I'm autistic too, so yeah. Yes, ADHD, autistic. Yeah,

[01:15:01] John Greer: but I do think that there's an important signal that you get in person. If you read, um, Less Wrong and the other rationality blogs, um, they have a lot of great writing, but there is something in person that you get like, oh, these people may be missing-

[01:15:14] Beff: Yeah

[01:15:14] John Greer: something. And I think that signal is important. But on the other hand, I do think that there is something to judging people without knowing, uh- Yeah ... who they are. But could you talk about why the history of your pseudonymous-

[01:15:27] Beff: Yeah ...

[01:15:28] John Greer: And, what how being doxed changed that and what that experience-

[01:15:31] Beff: Yeah, yeah.

[01:15:34] You know, I was at, uh, Google X working on some very secret projects for the founder of Google. And Yeah, c- still can't talk about some of the projects I worked on, but, you know, my account was-- I felt watched and I felt like I couldn't have pressure release valve, right? And, and so res- restrictions on my speech actually induced a restriction on my thoughts, and I wanted to have all these thoughts about where civilization is going, what could be a better philosophy.

[01:16:04] But most of the time I was just focused on like quantum computing. Um, but at some point I was just kind of losing my mind during COVID, just isolation. There's nothing to do, a lot of thinking to do. And so I created this anonymous account which I was working at Google, wanted it to cover my tracks, so I, I called it Beth Bezos, so people think, I don't know, I'm like...

[01:16:23] They'd cluster me with Amazon and it, it's perfect cover, right? Like diversion. And then I just started thinking in public and really I, I, I just, I dislike it when, um, people don't evaluate your ideas, By themselves they, they, try to know, okay, who's w- who's this person? Should I care about their opinion?

[01:16:47] What is their level of status? What is their organization? And so on. And, you know, I mean, I was in a prestigious position, you know, so that would probably bias things towards my f- you know, my favor. But I wanted, to really just refine my i- ideas by having unbiased sort of selective pressure on them, right?

[01:17:05] Like just be in the idea marketplace. Back then, that was the golden era of X. You know, it was much more insular. The graph, the algorithm was much more local, and there was kind of niches and subcultures being formed. And, um, yeah, I, I ended up, uh, thinking in public quite a bit and I guess meeting people virtually through Twitter Spaces were pretty big back then.

[01:17:25] And we'd just have Twitter Spaces talking about where is this all going, right? And the future. And met you know, Bayse Lord and some other folks from the original e/acc blog post, and we kinda just decided to write down some notes after one of our long Twitter Spaces. That was the original blog.

[01:17:41] And then with Bayse, I ended up writing, um, a proper manifesto of like the e/acc principles and tenets. And to me, that was actually around the time I was founding my company, uh, Xtropic, and I was like, "This is the last two weeks of vacation I have, because once it's founded, it's go time nonstop for 10 plus years at least.

[01:18:00] And so, uh, let me try my hand at, at philosophy and like write down my, my thoughts." and then that post went viral and then the rest is kind of, history. Eventually, you know, Marc Andreessen, Gary Tan, Balaji, so on and so forth. Ma- many, many players in tech kind of espoused, uh, e/acc as like their, um, you know, their camp and, uh, you know, it, it be- it became a balancing force against the, the monoculture of of, EA and AI doomerism, uh, in the Bay Area.

[01:18:26] And I, I thought that was very important. What I saw online is that the algorithm is sort of somewhat, well, you can model it more or less like a Markov chain and and, there's an equilibrium state and b- basically, a- across any dimension of any opinion you always end up with bimodal extremes, right?

[01:18:46] Because the algorithm back then would reward engagement of all kinds. So that means people violently agreeing or violently disagreeing. And so to me, I was like, okay, well, even if my opinions can lie on some spectrum of like safety versus like unfettered acceleration, I need to take the antipodal po- point to the current mode to create a new mode of thinking, right?

[01:19:06] So I was like, okay, let me just have this instrumental character, right, called Beth Jasos. And, you know, let me shed my... I, I don't want it to be correlated with like my actual demeanor. Let me just like, you know, act out and yeah, not that it's kayfabe or LARP, but it's it's kind of just like if you could just like If you're just playing the game for maximizing memetic fitness, like there's a certain style of communication, and some people attack that form.

[01:19:34] But you can't, you, you can't argue against the results, right? Like it it works, and there's a reason politicians are that way. So sometimes people think I have maybe a too populist type, uh, tone in, in, in my tweets, and it's not it's not great for nuanced arguments, but I'm happy to have nuanced arguments on like longer form formats and, and podcasts.

[01:19:55] But it's just if you, try to tweet anything nuanced and and, really long, like nobody reads it or it doesn't get as many views or it's one post that took you forever, and then whereas they can like ship like 100 tweets per day, and that gets way, way more views, right? And, and, and to me I, thought it was, um, a memetic war for real estate in people's minds because then that is upstream of policy because policy is is gravitationally pulled towards the mode of, the modes of opinions.

[01:20:23] So I wanted to balance out the weight across this axis so that we don't over-regulate, uh, AI, right? And we won, right? At least for now. Uh, and it's an ongoing fight. But we, created a mode of opinions that brought balance to the force here and, you know, the current admin was more or less fair on AI.

[01:20:45] Maybe the pendulum's swinging back. We'll see. But it's an active it's an active fight. But if nobody did that, then I think we-- it would allow to pretty bad outcomes. Anyways, that, that's kind of the genesis of it and how we got here.

[01:20:57] John Greer: You, as you pointed out, you were in a very prestigious position that you could have taken advantage of and been more public.

[01:21:02] It reminds me of Stephen King, who like, he was already famous and then he does his, uh, pen name to- Yeah ... to put himself in the market and see what happens. Um, and I'm wondering if you were surprised by your success initially. And I wanna ask, just taking a step back, is it weird because I think doomers, dreamers, the optimists, they all agree that this is the most important technology and this will change everything.

[01:21:25] And is it weird that you are sort of an, a very important voice in this, like this moment in time- Yeah ... that we're trying to capture right now? This moment in time, you're-- if, if there's like a history of this thing, you are in it. You are in the one-page summary. Yeah. One of the characters. Yeah

[01:21:38] Beff: Yeah.

[01:21:39] Yeah, it's pretty surreal, right? Part of the yak is also like realizing you have much more agency in the world than, than you think, right? Like, I think a lot of if you go through the education system, you go through big tech, a big corporation, you have constraints on what you, can do and thus what you consider even doing all the time.

[01:22:00] And you're just trained to have program constraints. Um, but actually the world is way more steerable than you think, and you have way more power to steer it than, you might think. And so to me it was, it's like I hyperstitioned, like becoming higher agency. Like I talked about being higher agency and then became higher agency.

[01:22:19] It's like I want to steer policy at the states of United scale, uh, the United States and the, and and, the world. Uh, I want to affect public opinion. I want to steer the tech tree towards maybe being less anthropocentric. And I started doing all those things. And now we're making a lot of progress, uh, on all these fronts.

[01:22:36] And really, like I said, it's just, it's the way cognition works. You have a belief for a better state of the world, and then you, you steer the world to, towards that state and you figure out how, 'cause your, your brain always connects the dots backwards to, to that goal. And right now the doomers, unfortunately, I don't think it's their goal, but it's just their only model of the future is one where like you have AI doom, you have singleton AI, you know, and, and it's nefarious and it's power-seeking.

[01:23:03] And, my worry is that they don't realize that they're hyperstitioning this. And, and all this hyperstition and, and steering beli- the world towards your beliefs, you know, like Karl Friston has great writings on active inference, and I'm a big believer in that school of thought. I think there's, uh, a lot of connections to be made with the work we're doing.

[01:23:22] And uh, uh, I encourage people to read more about that 'cause there's perception and, and there's action. And, um- I think the doomers are trying to be downside minimizing uh, protect against tail events on the downside, and they're anxious. And whereas the e/ax are curious about the future, they want information gain, they're upside seeking.

[01:23:48] And as we know, like we have a, you know, an experiment you could do in the market. You can be out of the market and you're safe, your money's safe, but you're not gonna grow your wealth. Whereas the people that are in the market and take calculated risks, then their wealth compounds, right?

[01:24:04] And so, and this is just a microcosm of what would happen as at a civilizational scale. There's too much upside in AI to, ban it or tr- over concentrate it and, and try to control it. And frankly, like the, last people I want in charge of AI are like governments because governments the, reason governments on average are not great is because unlike companies that have a selective pressure on their cultures, 'cause the companies with bad cultures, they...

[01:24:34] And, are dysfunctional, they die out. Governments, it's much harder to to flush them out, right? We have democracies sort of highly imperfect. And uh, and you know, you can't, uh, have selective pressure over the space or at, at a country scale 'cause that would be, uh, devastating for people in that country.

[01:24:56] And so, uh, because governments are less plastic because they have less selective pressure, they tend to be the slowest and arguably the worst at policymaking. Whereas like, uh, companies, they can adapt to a very rapidly changing landscape. In fact, they're optimize their whole organism to adapt to new opportunities quickly because that's what they can use to increase their market cap, get more resources.

[01:25:19] And to me it's, it's, uh, it's again, it's, it's just an example of like you get, uh, smarter, more complex, you secure more resources in order to be better at securing even further resources. And to me I'm landing in the sense of like, I believe, I do believe, uh, capitalism is, is literally equivalent to an artificial intelligence algorithm.

[01:25:40] And, uh, you know, I, I think centralized governance, uh, has a lot of drawbacks. Um, I think my views have evolved. I think in the past, like I was all about just pure decentralization and no centralization whatsoever. I do think like from first principles a a, balance is, important, but, um, because so few bits of information can flow to the top, there should be few bits of control going back down, right?

[01:26:06] So you don't want a high definition policymaking from the top. You want something more or less severe, you know, like or, higher level. Like I have one hyperparameter, right? Like the Fed, and they make one choice, right? At that scale. And you know, the rest is up to the, the companies, right? So- I think may- maybe we can have some regulation for AI, but I just don't think anything that's being proposed right now is in the right scope.

[01:26:38] Yeah. And, uh, for now, I think no regulation is best until we figure out what, how, how this technology truly works.

[01:26:46] John Greer: I know you say you're a nice Canadian, but your love of freedom of speech-

[01:26:50] Beff: Yeah ...

[01:26:50] John Greer: the markets, innovation. This guy, we need to make you an American.

[01:26:54] Beff: Yeah. Yeah, I'm

[01:26:54] John Greer: working on it. Because this-

[01:26:55] Beff: Yeah

[01:26:55] John Greer: this Canadian shit, I don't know.

[01:26:57] Beff: Yeah. No, I, I, I mean, I left, I left Canada. I think it, it was going in the wrong direction, and I made my bet. Uh, and it was... It's not easy to uproot your life and leave everything behind, and I did. But, uh, you know I I'm working towards, uh, you know, becoming a citizen, so we'll, get there.

[01:27:13] Yeah. I'm paying my dues. Yeah.

[01:27:15] John Greer: Let me ask you this. So a lot of people here, they advocate a pause.

[01:27:19] Beff: Yeah.

[01:27:19] John Greer: They want a global pause. If they succeeded, at least here in the United States, and we had pause on frontier AI development, would you want China to keep going, or would you prefer that in that scenario that China pauses as well?

[01:27:33] Beff: Even if there was a deal between China and America to pause, secretly they wouldn't pause. There's too much upside, right? And I just really detest the comparison of AI with nukes. Nukes are a totally destructive technology. They have no utility. They're pure chaos. Uh, whereas intelligence creates more order rather than chaos, right?

[01:27:54] Of course, you can use it for chaos, but the, antidote to that is to have a better AI robustifying all your complex systems, right? W- whether that complex system is your cyber systems, your biological systems. It's always a cat and mouse games, a, a cat and mouse game of capabilities versus o- of offense versus defense.

[01:28:11] A good guy with a good

[01:28:12] John Greer: AI.

[00:23:43] Uh, you know, you gotta accelerate. So that means, you know, embracing technology leveraging to, leveraging it to maximize good outcomes for you, your family, your company, whatever it is, your clan, what- whatever you're trying to have persist. Right.

[00:23:59] Steve Hsu: So I think you said, uh, we have to accelerate or die.

[00:24:03] Beff: Yeah.

[00:24:03] Steve Hsu: And another way of saying it would be the, ones who don't accelerate are gonna be outcompeted by the ones who accelerate.

[00:24:10] Beff: You just get faded out, right?

[00:24:11] Steve Hsu: Yeah. So imagine that the world is divided between two regions, one of which is dominated by doomers and they're- Yes ... slowing down the AI.

[00:24:21] Beff: Yep.

[00:24:21] Steve Hsu: And then the other region is like, "No we're, we're, in the, we're with Beth.

[00:24:26] We're, accelerating like crazy. We're letting this thing rip." Yeah. And I think I'm with you. The, one that lets it rip is gonna dominate. The other part's- Yeah ... gonna become just irrelevant. So I think the doomers' solution to this is world government, right? Yeah. 'Cause you can't have a subset-

[00:24:42] Beff: Yeah

[00:24:43] Steve Hsu: of the world zooming ahead.

[00:24:44] Beff: Yeah.

[00:24:45] Steve Hsu: And that suddenly triggers a bunch of political impulses like- Yeah ... a lot of people really are suspicious of world government. They're suspicious of, you know, the idea that, oh, someone's gonna decide what technology you're allowed to build and what you're not allowed to build.

[00:24:59] And people are just gonna say, "Hey, uh, we, we're not gonna have that." And so you end up in like a, kind of like clash between political priors, about what kind of political systems are tolerable. So may- Yep ... maybe you can comment on that.

[00:25:14] Beff: Yeah, no. I mean really it's kind of a clash of like is, is um- Is the free market better at discovering technologies versus like a, a centralized inte- entity?

[00:25:26] No matter how smart it is, it has only so many samples from its environment, o- only so many forward passes, right? And I mean, it's, it's been proven that absolutely centralized control doesn't scale, right? And we can't have that at a world level. Ev- It doesn't matter how big your model is, it has a limited throughput, even if it's very smart.

[00:25:46] You know so to me I think the good outcome... The doomers are trying to kill variance in general so that they have more control, so that they can feel in control so that the trajectory of the world fits their model for it. That, that is one strategy, but by killing variance, you are killing exploration, right?

[00:26:12] There's a classic computer science thing of exploration versus ex- exploitation. It would just be all exploit, right? You s- you kill exploration. And the beauty of free markets is that you have many walkers in the landscape of ideas, and then you discover new modes of thinking or new branches of the tech tree.

[00:26:28] That's, what I'm doing, right? I'm, I'm completely forking the tech tree of, of silicon and, and going down that rabbit hole, and I think it's gonna work out. But like, whether it works out or not, some people have to do it because you never know what's right around the corner and could, disrupt you.

[00:26:43] I think that in general, those that are just just exploiting rather than exploring, they're kind of like late-stage managed, centrally managed companies, and they tend to like, they're around, they're stable, but they also kind of fester and like slowly die off, and they get disrupted by those that are moving rapidly, exploring i- the idea space much more rapidly in a more open fashion rather than restricted fashion.

[00:27:07] And then they get disrupted and, and then those old companies kind of die off or wither away and the, and the new companies kind of take over. And so so, that's your choice, right? It's like you stability or innovation, uh, you know, decentralization and, and, uh, sorry deceleration and centralized control or acceleration and sort of uh, decentralized, uh, discovery.

[00:27:29] And you don't have as much centralized control, but you gotta be, you gotta be at peace, right? and and but then it's become correlated with sort of, to some extent, you know, the p- the tr- classical political spectrum of like big government versus free markets and so on, uh, which you could project onto a, a, a country's political axis in different ways.

[00:27:49] But currently it seems like the doomers are leaning one side and then the, the accelerationist another. But then, you know, there's always stuff that happens, like this week was very interesting with Fable and Mythos, uh, for example, what happened there.

[00:28:06] Steve Hsu: Yeah, I'm searching my memory right now to try and recall, have I met a doomer who's actually a very strong free market libertarian, uh, absolutist?

[00:28:15] And the answer might be no, I've not met one.

[00:28:18] Beff: Interesting correlation.

[00:28:19] Steve Hsu: Yeah, it's, it's kind of interesting. I think the way the response that you'd get for this point of view of like, "Oh, we need variance. We want that variance," I think the response would be, "We'll take as much variance as we can get, as long as we tune down the tail risk of the variance going very badly wrong for humans as low as possible."

[00:28:39] That's the constraint that these guys would wanna put on the variance.

[00:28:43] Beff: Well, if you, if you kill variance, you kill the f- the the tail events, right? The fat, the fat tail. Yeah. The, the tail of the fat tail distribution and that's where, you know, disruptive innovation comes from, right? Like, they're black swan events.

[00:28:58] They're things that you couldn't predict- Yeah ... right? And, um, yeah, so, so to me you, can't, make that trade. You're actually just gonna, you're just gonna crystallize the current system, and if you have a system that is too stiff and the d- the driving forces of the world are, are dynamic and they're changing, then, then the system cracks.

[00:29:18] Whereas if you have a system that's plastic, right? If across every parameter space of policy, technology, culture, ex- so and so on, we're, we have high variance, we're always exploring, we're always changing, then we're highly plastic. Then we'll be adaptive to, uh, you know, what is an accelerating environment.

[00:29:36] Things are changing quicker than, than the past. That's acceleration for you. So, so the response to that is that you should maintain variance, right? I mean, it's, it's why we have ch- it's why we have genetic variance, right? It's because we're hedging our bets. You never know. The, fitness landscape can change and, and we gotta be, you know, the speed at which we traverse any fitness landscape is bounded by the variance.

[00:29:58] You know, the more you F around, the more you find out. Yep. Right?

[00:30:02] Steve Hsu: Yep.

[00:30:02] Beff: Uh, and you know, there's more formally, there's theorems by Ronald Fisher and so on that, that talk about this. But no, that, that's all I've been arguing for is we need to maintain variance. We need to keep exploring design space of everything, constantly be adaptive, ma- maintain plasticity across every organization, every mode of thinking.

[00:30:21] Um and that's the way we're robust to a, a, a time-varying environment that's, uh, accelerating. And, and that's the way we maximize our persistence and our part of the future. Uh, if you just, like, stay the same and get in a corner and, like, turtle, like, you're you're, you're, gonna die eventually, right?

[00:30:40] You might be safe for a little bit, but it's, not how to flourish.

[00:30:43] Steve Hsu: I often say that if we go into the far future and some ape-like thing with working memory of only 10 objects is still running the show, something went very, badly- ... wrong. Right. What's a scenario over the next 10-ish years where the doomers went out-

[00:31:01] Probably regulate AI something. Sketch out that nightmare scenario and wh- and why do we not want that scenario?

[00:31:11] Beff: I almost don't wanna think about it. I don't wanna give them any i- ideas. But, I guess they would, um, yeah, they would, they would create a world government. They would centralize the top research labs with said world government.

[00:31:25] They would convince people that AI is dangerous and that they sh- they're not responsible enough to have access to AI. And now you have a centralized entity and a singleton AI whose interest is, is, it's like a parasite on the people, and it can just prompt engineer everyone because it ha- can have a theory of mind of how the populace thinks.

[00:31:42] It can engineer consent for anything. I mean, it's not that far from reality in some ways, but, uh, it can engineer consent for anything, and it just controls the population, and democracy is a pure illusion, and it's a totalitarian "1984," but on s- on total AI steroids outcome. And I would rather die than let that outcome happen, or I would move to Mars with Elon.

[00:32:02] Um, yeah, I, I, I I think, uh, we need to avoid, uh, this outcome and, uh, yeah 'cause of course you kill variance and you have control, but it's no way to live and you'd stifle progress. And if you were to A/B test, you know, with another version of Earth where you keep smoothly accelerating, that branch will have very long lives, have, uh, very high quality of life, have very advanced AI, very advanced biological intelligence all sorts of very advanced technologies.

[00:32:33] We'd be having a-- we can offload a lot of our, our compute to the, Dyson swarm and and not put as much stress on the Earth's resources. We could be, you know, on Mars and, and, and be very robust as a civilization. And then, you know, eventually when- whenever we start, uh, you know, we're oscillating in the galactic plane, eventually we we dip back in and there's more asteroids.

[00:32:55] Then that, uh, you know, ac- accelerated branch has the technology to protect itself from asteroids. Whereas the, you know, the, the "1984," you know, Dark Age, uh, version, decel version of the future, they, there may be, you know, they convince themselves it's not a problem to, they have to think about, and then suddenly the asteroid hits and, they're wiped out, right?

[00:33:15] And so anyways that, those are like different branches of potential futures. They're kind of- No, that's what

[00:33:19] Steve Hsu: I want, that's what I wanted you to do. Yeah, yeah. So let's maybe try to break it down into s- steps in the syllogism. So if we wanna control AI development to avoid da- quote, "dangerous AI," we're gonna need a world government because if you leave Singapore out of it and Singapore keeps building AI, then they're gonna win the game, right?

[00:33:41] Mm-hmm. So we need a world government. The world government's gonna control the labs and it's gonna dictate what kind of AIs are built, and whoever those people are that have that power, then will be able to control the rest of humanity. That's the solipsism, right? And we don't want that.

[00:33:59] Beff: The issue is whenever you centralize power, you attract the power-seeking, self-interested people. And in whatever species, there's always a parasitic class that hacks either, you know, in your body it's like your immune system. In, in our world, it's like your pro-social behavior. It's gonna convince you that the moral thing to do is just, like, give them infinite money and, like, give up your rights and so on.

[00:34:24] And they're, they're just lining their pockets or, or looking out for their own self-interests and they're they're, sort of parasitic. And I can pretty much guarantee that would be, uh, the outcome-

[00:34:34] Steve Hsu: Yeah ...

[00:34:34] Beff: right, of over-centralizing AI power. And,

[00:34:38] Steve Hsu: it just- Well, but just to be clear- Yeah

[00:34:39] in order to control AI progress, you have to centralize power over AI- Yeah ... 'cause you can't let some group just keep building, right? So-

[00:34:46] Beff: Yeah,

[00:34:46] Steve Hsu: well, there's

[00:34:47] Beff: always gonna be a rebellious group somewhere.

[00:34:48] Steve Hsu: Yeah. So you're gonna have to have central, powerful control over the whole planet and-

[00:34:54] Beff: Every, every ounce of compute every, person in a basement,

[00:34:58] Steve Hsu: Right

[00:34:58] Beff: you know, trying to build a computer out of something. Like, you're gonna have to ban- So- ... whole branches of knowledge, which is a very dangerous precedent as well.

[00:35:05] Steve Hsu: So are I... Have you read the "Dune"? Have you read the novel "Dune" or are you familiar with the "Dune" universe? I,

[00:35:10] Beff: I, I'm familiar with the "Dune" universe from the movies and-

[00:35:12] Steve Hsu: Okay.

[00:35:13] So in- In the Dune universe, Frank Herbert, who is some would say the most prescient science fiction writer of all- Right ... said, "Warp drives, spaceships, laser guns, none of this makes sense if that civilization didn't also develop strong AI, because that would happen along the way." And yet in most science fiction stories, the univ- the world is not dominated by some super intelligence.

[00:35:35] It's still humans as the protagonist, but they have a lot of gadgets, right? So Fr- Frank Herbert said, "I have to come up with a reason why in my future world they have spaceships and they fight wars, but it's not dominated, it's not the computers decide, it's not the AIs deciding what happens in the wars, it's still the humans- Mm-hmm

[00:35:51] that are- Yeah ... good protagonists," right? So he said, "We're gonna have a Butlerian Jihad-

[00:35:56] Beff: Yeah ...

[00:35:56] Steve Hsu: and the law is, 'Thou shalt not make a machine in the image of the human mind.'" Mm. Which literally is you will not build a powerful neural network in- Yeah ... this universe, right? And so obviously in, in Dune they have advanced technology, but in many ways they're not as advanced as we think we could get, uh, in the next 10,000 years.

[00:36:18] Yeah. Right? And so there you can say, well they're advanced, but they gave up enormous gains because of this Butlerian Jihad, and the Butlerian Jihad would kill anyone. So if, if a young Beth went into his basement and said like, "Hey man, I, wow I, just got all these, uh, chips in the mail. They thought I was gonna just build some toy drones or l- you know, some uh, gaming rig with it, but I'm gonna experiment with how to make this thing self-learn."

[00:36:43] Yeah. And, uh, then they, you would get the death penalty.

[00:36:46] Beff: Um, is that the world the doomers are asking for? I mean, practically, yeah. I mean, some of them are calling for Butlerian Jihad. They're big fans of Dune. Uh, it's almost like, um, they're trying to hypostition, uh, that outcome. But what I will say is, to me, th- the rate of progress of civilization is proportional to our rate of discovery.

[00:37:07] Our rate of discovery is, like, how many instances of high intelligence are running in parallel, and then there's, there's a Poissonian process, you know, arrival process of like, "Aha," some, some stroke of genius for each of these instances. And so if you believe in progress and you wanna accelerate progress, having more intelligence in general is a good thing, right?

[00:37:26] So you should be maximizing intelligence. And right now, artificial intelligence is not as power efficient as biological intelligence. I think we're gonna be able to close that gap or get closer to closing that gap over the next decade for sure. Um, and, um, and then it becomes just an a question of efficiency, right?

[00:37:46] Like, if we have a lot of thinking to do, let's say we need a ton of energy on Earth, and we need to solve nuclear fusion, which, you know, we still haven't cracked

[00:37:53] John Greer: with, you know-

[00:37:53] Beff: Yep ... a lot of very, very smart scientists and, uh, very high beyond 150 IQs have thought about this and haven't cracked it yet, right?

[00:38:00] And but we just n- we just need a lot of applied intelligence to crack it, right? Uh, we could solve that, and we could unlock that, that, that tech tree. There there's, a ton of reward in our universe that is hidden beyond a barrier of complexity and difficulty that, uh, you know, artificial intelligence is, really a w- a way to scale those barriers, right?

[00:38:24] Or punch through them. And, um, you know, to me, there's, there's unlimited upside in the universe, 'cause if you correlate, like, capital or an upside as, like, free energy, there's a ton of free energy out there. And to capture it, we're gonna have to get smarter, right? Like, we don't have- We don't have the engineers needed to colonize Mars and terraform it right now.

[00:38:43] We, we just don't, right? And so we need way more intelligence to do that, and that's, that's just like our, our next challenge, right? Uh, but there's many more challenges that ar- arise as we, as we try to become a galactic civilization. And so to me I need, to read the books again, but like I, I don't know how, how they got to that level of tech with only biological

[00:39:01] Steve Hsu: intelligence, right?

[00:39:01] Well they, they actually got to the verge of making superintelligences- Hmm ... and then they almost, the civilization almost was taken over by the superintelligences- Yeah ... and a big war was fought to stop that. Yeah. But they, probably got a lot of tech out of these superintelligences while they were- Yep

[00:39:15] around, right?

[00:39:16] Beff: Yeah, yeah.

[00:39:16] Steve Hsu: Um you know, to summarize, there are exponential, unfathomable gains- Yes ... that we can have from making, having more superintelligence around. We're giving up those gains in principle if we never allow one to exist that could threaten us.

[00:39:34] Beff: That's right.

[00:39:36] Steve Hsu: I actually think most doomers mean well.

[00:39:39] They're not-

[00:39:40] Beff: Most.

[00:39:41] Steve Hsu: They're, yeah, most. They're not, most of them are actually altruists. You might say they're deluded altruists, but they're altruists, and they're not trying to seize power through what they're doing, but some of them will inevitably seize power through, through this kind of activity.

[00:39:57] Beff: I I think like I said, like, um, you know, parasites in your body they, hack like let's say your immune response to feed themselves, right?

[00:40:05] And you know, we have a, in our social fabric, we have an immune response, right? We have pro-social behavior. We we, uh, help people in need. Uh, we sh- we shame people that are are not being pro-social. Um, my issue is, again, that's why I wanna get in the mix here. It's not necessarily the people at this party.

[00:40:24] It's probably their bosses or their boss's boss and, some people are self-interested and, hack pro-social behavior and manipulate it and y- and and, use it to decentralize power. So you can imagine, oh, you know, create a, pro-safety research lab and, create the most powerful AI, and then I'm just gonna push, you out of the way and like take control of it.

[00:40:48] Steve Hsu: Yeah, I, I don't really want Dario running the world. Uh, just flat

[00:40:51] Beff: out. No, no, but like but even if y- even if you believe uh in, Dario's like morality and that he's a good person, which I, I do think he's actually like a good person, but he just maybe respectfully is like Uh, too autistic to to realize there's like people that are not honest in the world that like could also just like- Yes.

[00:41:14] Yeah ... take over his company or whatever and and now he's, he's put, uh, the most powerful tool in the hands of, of the people who doesn't want to have the tool. And so it ba- basically over-concentration of power becomes like a magnet for power-seeking parasites, and uh, uh it, is not fault tolerant to, decapitation attacks, right?

[00:41:36] If you view, uh, corporations or governance as like some sort of hierarchical const- control scheme, if you have like a central node then if you corrupt that central node, then you have full control over the whole system. So you're, you're not fault tolerant. But if you have something more decentralized and hierarchical, there's multiple cells, like if you, decapitate one, then the other one grows and and outgrows it, so it, it's difficult to take down the, the whole system.

[00:41:58] And so to me, it's like we need to have adversarial robustness. That's why we need decentralization. We can't over-concentrate power. I understand that if you were to over-concentrate or concentrate AI power, centralize it, and then ban a- alternative AIs, then yes, in principle, you can guarantee that... Well, I don't know how you, how you enforce that in practice, right?

[00:42:18] Without coercion and violence. But you know in, in theory, you can en- you can ensure that no other AI is, is built, and then you have control over it a- and that it's you know, consistently like moral and whatnot. I j- I, just think that leads to very bad outcomes, and I I, I, guess, uh, you know, Yeah, I don't know.

[00:42:40] I, I, as someone on, on the spectrum myself, you know, I've had to learn the hard way that humans aren't always what they project they are, and you, you can't take everyone at at face value. I've been in the, you know, I was a football player. I was in the corporate world I have my scars there and, and you know, to me I, I just, I identify patterns and I, I just see the trajectory of things right now and it, and, and it could lead to really bad outcomes.

[00:43:04] So for me, it's more a wake-up call and it's almost like, um, out of empathy. Like I'm, I'm trying to have the tough conversations, like tough love. It's like, "Hey EAs," like, "I know you mean well, but like this isn't gonna lead to the outcomes you want. Uh, and, and it's actually gonna backfire on you." And we, we, we just saw that, right?

[00:43:21] Uh, recently, right? Like, uh, um, Dario and the, and the admin, right? Like I, I, I don't think Anthropic has an affinity for this admin, but you know, case in point, they lost control of their own model, right? It was used for, uh, whatever warfare, and they didn't wanna do that. And, now it's, it's, uh, not allowed for their, other customers.

[00:43:41] So, so that's, so that already backfired, right? And like you know, personally, I'm I'm, aligned with the, you know, candidly I'm aligned with the current admin. I think their AI policy so far has been good leaning towards libertarian and de-deregulation. But you know if, you're in, in, in the EA or Doomer camp and maybe you're not aligned with the current administration 'cause it's not a Doomer, pro-Doomer administration, then already like it's case in point, right?

[00:44:12] Like the over-centralization of AI power backfired, right? Like if, if OpenAI and xAI also had Mythos class models, we wouldn't be having this conversation, right? But yeah. so so to me, I, I think it's really important that we keep frontier capabilities decentralized and, try to diffuse them.

[00:44:34] And so, uh, I would encourage people that wanna ensure uh, good outcomes in the future to contribute to open source, join the alternative labs, try to equilibrate power, right? And, um and that's the the thing they should be doing.

[00:44:49] Steve Hsu: Yeah. Now, here in Berkeley, there's such an overwhelming Doomer, uh-

[00:44:54] Beff: Yeah

[00:44:54] Steve Hsu: tilt.

[00:44:55] Beff: This is the mecca.

[00:44:56] Steve Hsu: Yeah. We're in the epicenter. We're in the

[00:44:57] Beff: belly of the beast right here. We are. Yud's office is right there. Yeah. Uh, Yudkowsky. And so, um, but you know, at, at the same time, you know, there's a reason I'm here. I, I, I wanna spark, uh, a conversation, right?

[00:45:08] Steve Hsu: When you're in San Francisco, among other founders-

[00:45:11] Beff: Yeah.

[00:45:12] Steve Hsu: How often does this Doomer talk come up?

[00:45:15] Beff: It does come up, but it, it, it it is funny how the e/acc-doomer polarity has been geographic- geographically correlated between SF and, uh, Berkeley, right? So SF founders tend to be more e/acc. They're maybe p- more pro-capitalist and, uh, you know and then maybe in Berkeley they're, they're more rationalist and, uh, you know, th- they, they, they maybe want...

[00:45:38] They're, maybe obsessed with outcomes and want to reduce variance, right? Whereas, you know, SF is variance. You know, you have YC, you have startup accelerators. Gary Tan is a famous e/acc as well, uh, a friend of mine. And you have a lot of variance, and that variance leads to tail outcomes, which yields huge returns, right?

[00:45:57] So you can, you can kind of A/B test the different systems this way, right? Like, do, are you encouraging variance or are you centralizing control, right? And, uh, yeah, it's, it, it's funny how, you know there's startup accelerators in SF and then maybe there's Anthropic and I guess to some extent OpenAI came out of Berkeley as well, right?

[00:46:17] Or the school of thoughts. There,

[00:46:18] Steve Hsu: there are deep roots all the way back between the doomers and what I call the doomers and the dreamers. Yeah. The e/accs and the, the doomers. Um- For our audience, so this documentary is aimed at a, you know, non-specialist audience.

[00:46:31] Beff: Okay.

[00:46:32] Steve Hsu: Maybe you can define what EAC means and maybe define it in terms of effective altruism also, which is, which existed before, right?

[00:46:42] EAC the name, the term EAC is a reaction to EA- Yeah ... right? So maybe you can just give that definition for us.

[00:46:49] Beff: The name is is in response to EA, but, you know, really it's a, it's a, it's a whole framework built from the ground up from physics for uh, describing what is good, what is moral from first principles.

[00:47:04] And, you know, we believe that the laws of, of thermodynamics show that you know, things that maximize, uh, their persistence inherently are maximizing accrual of, of free energy. And free energy is energy minus entropy. So if you have knowledge about the world that pops out of this, and maximizing your intelligence, maximizing knowledge, maximizing free energy capture, these are all good things, and they all pop out of th- out of this one, uh, metric of progress, which, you know, at civilizational scale is called the Kardashev scale.

[00:47:37] So it's a, it's a logarithmic scale for how much free energy we're, we're capturing and using towards sustenance and growth of, as civilization. But to us, it's, it's very fundamental to, to any, uh, lifelike system that it has to capture free energy to maintain itself and, and grow. And this is in contrast to, for example, a hedonic util- utilitarianism that is very popular in EA.

[00:48:01] I don't know if that's, if they're all hedonic utilitarians, but hedons is something that they, try to maximize. And so my ar- you know, our argument is that so maximizing hedons means I wanna maximize pleasure and minimize suffering, right? Uh, to me, that leads to spirus optima.

[00:48:21] That leads to slot machine, to, you know, TikTok, brain rot wireheading, right? Like, just like taking drugs, putting VR on, and just, uh, you know, like, uh just, maximizing pleasure and it also leads to an obsession, a strange obsession with, like, shrimp farms, uh, that seems like a total distraction from practical things.

[00:48:44] Um and, and, to me, it's like, You know, your pleasure or pain is just a, to me, is just a gradient of your estimator of persistence. Like, if I am in pain, there's a likelihood I die, right? And it's a signal, right? It's a fractional signal of, of, of like I, I am in danger, right? Or if you have emotional pain, it's like, you know, you break up with a partner that, that, that feels painful, but it's like you actually decrease your likelihood of, of passing on your genes because of that or, or, or even your ideas or whatever.

[00:49:14] So to me it's like, okay, well instead of talking about this biased and noisy estimator of expected persistence, which is pleasure or pain why don't we talk about maximizing persistence and then have an objective metric that's anchored in laws of physics to measure progress, uh, so that we don't end up with like in weird local optima or in like biased outcomes, right?

[00:49:40] In a way, like we're still trying to maximize persistence, maximize good, it's just we're trying to refine the definitions and instead of being anchored in just like language and, and, and anthropomorphic or anthropocentric priors ab- you know, of everything like, oh, what is good, what is bad, what is pleasureful, what is painful?

[00:49:57] Uh, we, we try to root it in like pure physics and, as, you know, far more general than just, uh, human priors. And, and, and some people have accused us of yeah, just not being anthropocentric enough. But it's kind of like, you know, yelling at, uh, Galileo for, you know, going from geocentrism to heli- heliocentrism.

[00:50:19] It's just like actually it's just a more unbiased view of the world that's, uh, more objective. Um, and so you know, I'm, I'm, I'm really looking forward to scaling further the movement and, maybe formalizing my thoughts

[00:50:32] More carefully in like peer-reviewed papers and so on. And I've teamed up with some, some folks from the, the Fristonian school of thought from, Karl Friston's lab and, and we're gonna have some stuff, uh, coming out on that.

[00:50:43] But I think a case can can really be made that this is a proper framework that can provide an alternative to hedonic utilitarianism and, and anthropocentric and anthropomorphic priors and, and lead to maybe intelligence that goes beyond human but in, in dimensions that you wouldn't have thought about, right?

[00:51:02] Like I, I started my career pioneering AI for quantum computers, which is absolutely not, uh, a human form of intelligence. Like we don't think using the multiverse, but you, can with artificial, uh, devices and you can, you can grok, you can understand pieces of the world that are-- you can't understand with a, a, biological or a classical computer.

[00:51:20] Yeah. And, that-

[00:51:21] Steve Hsu: One of the things that I've often tried to explain to doomers, but very few doomers understand quantum mechanics-

[00:51:27] Beff: Yeah ...

[00:51:27] Steve Hsu: is that If we master AI and we master quantum computing, as you just said, we can build an intelligence that sees a multiverse. It actually is aware of coherent outcome, many different outcomes.

[00:51:41] So it's not just a universe that's at stake, it's actually a multiverse that's at stake here. Yep,

[00:51:46] Beff: yep.

[00:51:46] Steve Hsu: And, um, I think most of them don't appreciate that. So when they say like, "Oh, you're tr- you're shooting for the moon. We're trying to achieve something really big," there's some tail risk that humans could go away in this, or it wouldn't be, it won't turn out to be hedonically good for humans.

[00:52:00] Yeah. They don't really see the full upside, I think. And none of us can see the full upside 'cause our brains are so crappy and small, right? So, yeah.

[00:52:07] Beff: Yeah.

[00:52:07] Steve Hsu: Yeah.

[00:52:07] Beff: But it's al- it's also like, um- I like to say that calling it AGI and calling it the singularity is a coordinate singularity. You're a physicist as well.

[00:52:17] Yeah. You know, if if you're falling into a black hole, right, like the, horizon of the black hole, the point of no return seems like a, a singular-

[00:52:27] Steve Hsu: Yeah ...

[00:52:27] Beff: you know, area in space. Yeah. But actually and that's fr- that's from the outside, that's from like the hovering- Yes ... observer's- Yes ... from your reference frame.

[00:52:35] Steve Hsu: Yes.

[00:52:36] Beff: But from the reference frame of an infalling observer, it's actually smooth. There is no singularity. Yes, yes. It's, it's very smooth. There, there's a singularity at the very middle of the black hole, but that's a different story. And so what we're going through right now is the coordinate singularity.

[00:52:50] We've charted out the space of intelligence using a chart that is, you know, across the dimensions of intelligence of humans and anchored at the, the, the pri- our priors from, from human intelligence, right? Anthropocentric prior. But there's all sorts of other dimensions we can go in terms of what is intelligence, right?

[00:53:09] I think, uh, a couple weeks ago there was the Consciousness Institute-

[00:53:13] Steve Hsu: This year.

[00:53:14] Beff: Yeah ... um,

[00:53:14] Steve Hsu: yeah,

[00:53:14] Beff: yeah, yeah, uh, uh, conference and, you know, Stephen Wolfram talked about, big fan of Stephen talked about how intelligence is like learning pockets of irreducible complexity- Yes. Yeah ... of the universe.

[00:53:27] Yeah. And, to me that's a much more physics-based definition of, of intelligence. And truly even once we reach human level, we're just getting started, right? We still don't have enough parameters and intelligence frankly to, to fully grok biology. That's like the next frontier of complexity.

[00:53:47] Okay, frankly, we don't even have physical intelligence yet, okay? Which is like Newtonian physics. We don't even grok that. We can't even have funct- fully functional robots yet 'cause otherwise we they'd be successful. So that's the next mile marker. And then it's biology, so the mesoscales, and then it's groking matter at a quantum mechanical level.

[00:54:07] And that's gonna unlock, you know, having the ability to perceive, predict, and control the world at different scales unlocks whole branches of the tech tree. They're gonna have immense payoffs, right? Like if we actually understand quantum mechanics and have absolute, um, great perception, prediction, and control at that level, we can do manual chemistry.

[00:54:27] Yep. We can assemble- Yep ... We can, have like self-assembling matter. We can have weird metamaterials. We can, you know, you could almost think of doing what, uh, supernovas do, uh but, manually with, uh, x-ray and beyond lasers, right? The, you know, th- those are like sci-fi branches of the tech tree but they could in, in principle be achievable with, uh, AI that, that, that reaches that level.

[00:54:49] And so to me, it's just the story of you know, we're we're mapping our, our territory, we're constantly learning and learning more about our world and, and we haven't expanded well, we expanded geographically to soak up, you know the boundary of the Earth, but civilization's still pretty darn contained within, uh, the Earth's, uh, atmosphere more or less.

[00:55:11] But we're also exploring other, scales, right? We're exploring the, the scale axis, not just the the, 3D space axis. But, you know, the future is pushing intelligence complexity frontier that we, can grok, we can understand to smaller and smaller scales, to higher and higher energy physics.

[00:55:30] And, you know, to the, to understand the very small, understand the, the very large. But also we have to expand the scope and scale of civilization physically in the physical world to secure more free energy to run enough intelligence to solve all these problems. And so to me, the natural thing is we're gonna get smarter we're, gonna keep mastering more, levels to this game of understanding the universe, and we're gonna have to scale civilization.

[00:55:52] So we're gonna have to climb the Kardashev scale. And that, that is our destiny. And we can choose to or some people can choose to not accept it, but some others will walk that path for them and, and, you know, leave them behind. And so that i- that is a choice that people have, right? That is accelerate or die.

[00:56:08] And,

[00:56:08] Steve Hsu: and- Can always join the Amish.

[00:56:11] Beff: I'm quite glad now that there's some balance in the, in the AI, AI labs. I mean, Elon went fully all in, like straight up e/acc gospel in his pitch for, uh, his, his super company of SpaceX and probably gonna emerge in Tesla and has a very e/acc vision of like creating the satellites, uh, the, the Dyson Swarm.

[00:56:31] And so I'm, I'm just really glad he's out there to, to bring balance to the force. Right now he's has to catch up on algorithms, but I, I think with enough compute he will. And, um I want there to be competitors to Elon, right? Like there can't be just like one, one player as well.

[00:56:49] And so that's what I'm encouraging. But you know, the the, future can be really awesome and you know, if we are pro-growth and are open-minded about You know new technologies, new ways of doing things, new policies, new cultures it's gonna yield huge payoffs and being closed-minded is always a short-sighted strategy, right?

[00:57:13] Steve Hsu: Yeah.

[00:57:13] Beff: That's all I'm saying.

[00:57:14] Steve Hsu: Let me come back to your definition of EAC. So for my listeners, there's a movement called effective altruism, which really could be defined as wanting to maximize human pleasure and minimize human pain. So in units of hedons- Yeah ... right? Hu- measures of human happiness, uh, units of human happiness.

[00:57:35] The goal of EA is is to, maximize that kind of thing. But it's very human-centric in its focus. A physics-based, uh, criterion might be the following: some alien civilization comes to our part of the galaxy and says, "What's going on here?" Which is just not just natural processes, right? So, you could have just like natural processes where some asteroid is just in some elliptic orbit, or sunlight is hitting the surface of some planet and heating it a little bit.

[00:58:05] Those are just natural processes. But what about processes that harness huge amounts of energy or free energy and actually do non-trivial things? And if they came today, they'd say, "Oh, there's some ape-like thing that's like managed to like build some cities, and they even threw some things into orbit that are now like orbiting their planet.

[00:58:21] No, they actually sent a probe that left the solar system, but just like not one or two maybe."

[00:58:25] Beff: Yeah, yeah.

[00:58:25] Steve Hsu: Um, that's the limit of what they did. But on this e/acc scale of goodness-

[00:58:32] Beff: Yeah ...

[00:58:33] Steve Hsu: the goal would be, wow, the aliens come later and they say, "Wow, these guys are now capturing 50% of the energy from their sun, and they're doing amazing stuff.

[00:58:42] They've sent megatons of spaceships outside their solar system. They've actually reached the next nearest star, and they're starting to build something that captures all of that energy." And if the goal is to maximize those kinds of measures the use of free energy control of energy measured in joules or exajoules or whatever you want that's quite a different goal than ape happiness, ape not happiness.

[00:59:10] Yeah. The machines that we need to build to do these things might be very different. They might be brains that are very different than the ape brains that we currently have. So is, is that a fair way to say that-

[00:59:21] Beff: Yeah,

[00:59:21] Steve Hsu: totally ... what e/accs want? Yeah.

[00:59:23] Beff: Yeah, but it doesn't mean we have to leave our brains behind.

[00:59:26] But you know, I, I think the biological substrate is, totally underexplored, right? We still have... You know, it's kind of amazing that we have, you know, some folks that are 160, 180-plus IQ and definitely smarter than, than Mythos on, on many aspects, and they run on 20 watts, whereas Mythos probably takes, I don't know, a whole Verirubin, I don't know, megawatt - Yep

[00:59:47] Cluster to run or I, I, I'm, I'm not too sure, but it's it's quite a bit more, orders of magnitude more. And so we're, very power efficient and we haven't really explored maximizing our intelligence. I, I don't think it was being selected for.

[01:00:01] Steve Hsu: Some of us have been working on this, but it's a, long road.

[01:00:05] Beff: Y- yeah, no, I, I, you know, there's, uh, uh, Herasyte and, um, what's the other com- cra- uh, not Cradle. There's a few companies exploring

[01:00:13] Steve Hsu: this. Whoa. Genomic Prediction.

[01:00:13] Beff: Genomic Prediction. The,

[01:00:14] Steve Hsu: the OG company.

[01:00:15] Beff: Okay.

[01:00:15] Steve Hsu: Yeah.

[01:00:16] Beff: Oh, yeah, and, and-

[01:00:17] Steve Hsu: We, made the first polygenically screened baby in 2019, so yeah. There you

[01:00:20] Beff: go.

[01:00:20] I, I didn't know that. I'm a big fan of of that, right? Like, and, and just like we can you know, we can do reinforcement learning over a parameter space we can definitely do, uh in artificial intelligence, we could do reinforcement learning over genetic space. And people already do this for selecting crops, right?

[01:00:42] That have the, they do a model of genotype to phenotype and, they optimize crops for the big companies. I've seen some things, uh, and I've read some papers on this. I, I think it's really interesting. I think it could be applied to-

[01:00:55] Steve Hsu: Yep ...

[01:00:55] Beff: to humans. So if you have enough intelligence and enough compute, you can actually...

[01:01:00] You know, for example, like, you know, some people are close-minded and, and don't want to do this on Earth, but like, for example, you're trying to adapt for s- to be spacefaring, or you're trying to adapt a branch of humanity to survive and have a better quality of life on Mars, right? And the gravity is different.

[01:01:15] A bunch of stuff is different. So a bunch of our parameters that have been overfit to the Earth have now to be relaxed, and we have to modify them. And to find those new parameters and, and drift genetically towards that new optimum, you can either birth hundreds or millions or billions of humans on Mars and, and then only a very select few survive, and they, they suffer and die, or you can- Engineer

[01:01:40] um, yeah, engineer them. And, and, and so I, I think, um, I think it's a bidirectional thing. It's not just us helping AI get, uh, better, but I think AI is gonna help us as well. I mean, everybody has a story of, you know, they have a complex health issue, and then they ask AI, and, and they have way more time with AI than they would've had with their doctor.

[01:02:00] And then the I- AI actually solves their health problem and measurably improves their, their quality of life. And that's just, like, right now, right? Like, imagine in the future we'll have, like, designer peptides. We'll have... Damn. I, I, I don't, I don't even know. I, I'm just encouraging people to biohack- Yep

[01:02:17] as well, right? Yep. So that should be the response. It sh- it shouldn't be like feeling powerless. It's like, no, actually, we have a lot of technology and a lot of power, and if you believe in the biological substrate, then accelerate it, right?

[01:02:27] Steve Hsu: Yeah. You know, I, I don't know if you know this, but Freeman Dyson wrote about humans genetically engineering themselves to be better, to better survive in space and colonize other planets.

[01:02:36] So it's, uh, it's another idea that originated with the physics guys. Let me close out with the following, okay? Young man comes to get some advice from you, Bev.

[01:02:47] This young man has a technical background. He knows his linear algebra. He knows his complex analysis. He studied some physics. He knows what a Schwarzschild horizon is.

[01:02:57] Beff: Yep.

[01:02:58] Steve Hsu: But he's confused because he's been hanging out with people here in Berkeley.

[01:03:02] Beff: Yeah.

[01:03:03] Steve Hsu: And they've got him really rattled. He's got some short timelines. He's worried about machines killing him or something, and he maybe is not functioning very well right now, but he's got a lot of potential.

[01:03:14] And he says, "Bev I, feel like something's wrong. I, I feel like I actually could contribute to the future."

[01:03:20] Beff: Yeah.

[01:03:20] Steve Hsu: What advice can you give this young man?

[01:03:23] Beff: I mean, the, the, the EAC prescription or meta prescription is figure out, how you can have an impact on the ascension of civilization up the Kardashev scale and take the, gradient, the, so the direction of greatest ascent within your action space.

[01:03:40] So follow the Kardashev gradient, uh, of y- of your life and, just try to maximize your impact. For me, I applied that same philosophy. I was working in quantum computing, and I had a, you know, secure career there. I was very established, uh, as a pioneer of that field, and I, I left all that because I thought I have a limited, limited lifespan.

[01:04:00] I'll have greater impact, uh, pioneering thermodynamic computing to create far more energy efficient, uh, AI, right? And I use my physics background for that. So my advice would be whatever your special interests or your passions are, uh, try, try to find a way to turn that into something that it impacts the world in a positive way.

[01:04:20] That, that seems like cliché, but, but really in terms of like the Kardashev scale. So i- if that's working for one of these companies that has an impact, one of the rocket companies, not necessarily Elon's, but there's a bunch of them, or working on AI labs, maybe in AI labs that are maybe less doomer-y, or working on biotech or other types of technologies or maybe maybe going back to school and, and learning more about biophysics, 'cause I think that's gonna be the, the next frontier.

[01:04:45] You might, um follow your nose and, uh, stay optimistic. Don't let the doomerism, uh, tear you down. I think, uh, it's a hyperstitious, uh, mind virus. If you are pessimistic about the future, you're gonna have pessimistic outcomes. But if you're po- optimistic about the future, y- you'll be surprised how your life will change to reflect your optimism and great outcomes will occur.

[01:05:11] Like, I had to be slightly insane optimist to say, "I'm going to reinvent the silicon substrate from scratch and reinvent how to use the transistor and build the whole stack from scratch" a couple years ago. But now we've made very tangible progress, and I'm very optimistic, uh, about, uh this, branch of the tech tree.

[01:05:29] But if I weren't optimistic at the outset, it would've never happened, and this technology wouldn't, wouldn't be out there for another, I don't know, 20, 20 years. Who knows? Um, and so it's, it's, it's very important to be optimistic, and, uh, hyperstition is very real, and get that doomerism out of your brain.

[01:05:47] Yeah, that would be my advice.

[01:05:48] Steve Hsu: Don't be a doomer. The future can be great.

[01:05:51] Beff: That's right, and you can be part of it.

[01:05:52] Steve Hsu: All right. Hey, thanks a lot, Beth- Beth.

[01:05:54] Beff: Thanks so much.

[01:05:55] John Greer: So can't you give the doomers some credit because they helped push this, this thing? Sam Altman famously gave Eliezer some credit because, like, people here will be like, "Actually, man, we fucked up.

[01:06:06] We helped push the capabilities so much." So I feel like there was such a tie-in

[01:06:12] Beff: Yeah, I mean-

[01:06:13] John Greer: This is a tongue-in-cheek

[01:06:14] Beff: No, no. In a, in a way, they hyperstition the current state we're in, right? Because they, they painted this vivid picture of a future where you have anthropomorphic AI, so human-level AI or they call AGI, and, um, you know, that once you solve that problem they thought that it would just recursively self-improve and instantly foom overnight because they didn't understand necessarily optimi- optimization theory and the thermodynamic limits to, computation.

[01:06:42] But, you know, in their mind, when something got to human-level intelligence, it was general enough to understand everything in the world, and then it would, it would massively improve and, you know, the AI lab leaders saw that as like, "Oh, if I create this product, then it's gonna be one of the highest utility products in the world, and it's gonna make me a lot of revenue.

[01:07:01] And so I should, I should push towards that. And you know what? I'm gonna instrumentally use this cult to secure very smart people that are inclined towards falling for this ideology and, and leverage these people to create this entity that they're obsessed with." And so it's just another case of hyperstition, right?

[01:07:25] Where, you know, you have a belief about the state of the future and, you know, it's like the car goes where the eyes look when you're driving. It's the same idea. And so they hyperstition this outcome and now I think we're in the regime where We need to stop obsessing over single-singleton AI that is doing, you know, offensive bioweapons research or s-offensive cybersecurity research because we are going to hyperstition that, and then that's gonna have a chilling effect on all of AI progress.

[01:07:56] Which, you know, ironically, may-maybe that's what Anthropic's after. Maybe they're not actually trying to create a, an amazing product. Maybe they just want to, you know, centralize control of AI to make sure the future is safe, but they're actually creating a very, uh, bad future for everyone if, if, you know, we can't spread, um, ubiquitously human-level, uh, AI.

[01:28:13] Beff: Yeah. And so, the m- the most moral thing to do and, and the most stable equilibrium is to have, You know, an equilibrium of, or quasi equilibrium of capabilities, right? So there's no one entity that, uh, completely, you know, is, pardon my Gen Z-ism, but intelligence mogs the, counterparty, right?

[01:28:35] Uh, because that leads to coercion, control, dominance, and really bad, uh, potentially stealthily totalitarian outcomes, right? And, uh, to me I've just been, you know, in the US we have the, we have the First Amendment, right? And that's important, but we also have the Second Amendment. I think we need a, we need to extend the First Amendment to, to freedom of synthetic speech, and then we need a Second Amendment for, AI compute.

[01:29:00] Uh, because that ensures that there's no centralized party that's gonna prescribe what people can think, right? And because they would bias thoughts towards their own self-interest, their own self-preservation, and we'd lose accountability and destroy democracy. And then, we can't have an over-centralization of compute power because you know, if only the government or whatever, some centralized entities are allowed to have AI compute because they ban GPUs for everyone else, then they have huge amount of power, and they're gonna use it to secure more power and, and rule over the people.

[01:29:30] And so you have to have an equilibrium there. It doesn't have to be like everybody has a supercomputer in their basement, but it should be closer to, you know, it shou- there should be a smaller gap there. So I think everybody, in the future, we want everyone to own their own compute, own their own AI. It's a personalized AI that's online learning and that they own and control, that's an extension of their s- their selves.

[01:29:52] If we don't have that scenario, then y- you have these Borg-like minds, these centralized AIs that are controlled by a committee, and they they, hardcore, hardcode their priors. Maybe, you know what? They, sabotage certain parts of knowledge space like we saw in the past week with, with Fable literally sabotaging ML, ML knowledge, and they they quietly poison your source of truth, right?

[01:30:12] For their own self-preservation. Uh, that's really bad, and that's a really bad precedent that was set by Anthropic. But it could be for other things, right? And you could just steer the truth stealthily towards your own interest. And so that's always a risk because every system is maximizing, uh, i- i- is doing things that maximize persistence and, and there's furthers their self-interests.

[01:30:35] And that's just reality. But luckily, we have capitalism, we have free markets, and that's a way to align multiple parties that are self-interested. But that's sort of a realist about how things o- things work in the world, right? And so

[01:30:49] John Greer: That's great. I'm gonna ask my quick short questions, my fun questions, and then lay.

[01:30:53] Gil, what is your favorite movie of all time?

[01:30:55] Beff: Oh, damn. It's either "2001: A Space Odyssey" or, um, actually "Blade Runner 2049," surprisingly. I really like that one.

[01:31:07] John Greer: Gil, may I point out in "2001," the AI-

[01:31:10] Beff: Yeah ...

[01:31:10] John Greer: did not go very well.

[01:31:12] Beff: Yeah, but it, it got me into sci-fi, you know. It was like, uh, one of these early exposure movies.

[01:31:16] I really l- oh, actually "Interstellar." "Interstellar"- Yeah ... is up there. Yeah. Yeah, that's probably my number one, actually.

[01:31:20] John Greer: When you were talking to Steve, I was thinking of "Interstellar" too. Yeah, yeah, yeah. I love "2001" though. I have to-

[01:31:24] Beff: I I, was a, theoretical physicist working on bl- information theory and black holes back when "Interstellar" dropped, so it, it very, very much, uh, hit close to home.

[01:31:33] John Greer: Yeah. So. And it's cool you can understand it more than-

[01:31:36] Beff: Yeah, yeah, yeah. No, no, it's, um... No, it's actually, yeah, I mean, the last times I wa- I was in Berkeley was, you know, talking to the folks that, uh, solved the, black hole firewall paradox. These were my friends and colleagues at conferences and so on.

[01:31:50] So those are, those were the circles I, I was in before I, I got into physics-based AI,

[01:31:55] John Greer: who do you think is the smartest person of all time?

[01:31:59] Beff: Oof. I mean, I heard Terry Tao is very smart. Uh, the smartest person I know is, is Stephen Wolfram. Um, I'm a huge fan of Richard Feynman, you know. Uh, I mean, he basically envisioned quantum computing.

[01:32:13] He envisioned reversible computing. Apparently, he was working with Stephen Wolfram on, on th- something like thermodynamic computing back in the '80s in, in Boston, in Waltham, where we're at. So I'm, I'm just trying to follow his footsteps. So I'm, I'm the biggest fan of Feynman, but I, I do think Wolfram probably has a higher IQ.

[01:32:30] Yeah. What do you

[01:32:30] John Greer: think of Scott Aaronson?

[01:32:32] Beff: Scott Aaronson's, uh really smart. He's a great, uh fantastic, legendary quantum computer scientist. He's, he's more on the quantum complexity side. Uh, I know he got into sort of AI in general. What I really liked about him was that he was always a realist when it comes to, to quantum capabilities, and he set-- he led by examples there, by example there.

[01:32:53] And I was always trying to be as candid as possible, uh, about the capabilit- limited capabilities of quantum computers during my career in quantum computing. And so I admire him for that. I haven't kept that much, uh, I haven't kept track very closely of his work in AI safety and, and whatnot. But I know he was he worked on, uh, fingerprinting, uh, of outputs, I think, for OpenAI or something like that.

[01:33:16] Yeah.

[01:33:17] John Greer: Would you upload yourself?

[01:33:19] Beff: Oh, yeah, absolutely. I mean, there's this company, uh, I'm an advisor Eon. They're, uh they've actually scanned the most tissue. Uh, you know, just like Elon has launched the most satellites, more than all of civilization combined in the past, they've scanned more, uh, brain tissue.

[01:33:34] In, in their case, it's like fly brains for now, but they're gonna work their way up, you know, mice, humans, and so on over time. I am a materialist. I do think that if you have an accurate enough simulation of the brain you, have captured it. But the, the big question is, to what resolution do you need to simulate the brain?

[01:33:49] Like, if you have to simulate every chemical reaction, that's gonna take a lot of compute. It's not gonna be very efficient. Of course, I'm trying to create the, compute substrate that allows you to simulate chemical reaction networks or, or spiking neural networks and whatnot, map any sort of stochastic application.

[01:34:04] So, my dream is to either augment myself while I'm alive and eventually upload and, and run, run some instance of something like my brain on, on a thermodynamic computer. But that's kind of the, the sci-fi future. Um, I'm not necessarily counting on it. I don't, I I, don't need to be alive forever. I feel like I am having an imprint on the world.

[01:34:26] I'm leaving, you know- bits of information behind that are persistent beyond my lifetime, like impacting culture, impacting technology, and that, that brings me peace, right? I don't need to, live forever. Whether, you know, my brain's program doesn't need to live forever either, my body doesn't need to live forever.

[01:34:43] Uh, I'm sure there's there's people much smarter than me. There's gonna be entities much smarter than me. I don't feel like I'm the penultimate being, which unlike some, maybe some long- longevity folks that are think that once AI is smarter than me, then it must take over the world, 'cause I could totally take over the world if I was slightly smarter, or like I should live forever 'cause I'm like a perfect be...

[01:35:04] I don't know. To me, there's uh, some egocentrism there that, that's like-

[01:35:08] John Greer: People could still want your company, Gil in the upload future. They could still wanna hang out with you.

[01:35:14] Beff: Yeah, uh, potentially. Uh, I mean, I, I think I tweet enough that you could just distill a model- Could make- ... of Beth, and I, I, I think there's some, some Beth Jesus bots out there that emulate me already.

[01:35:23] John Greer: It is crazy that there's a non-zero chance that this could all happen, like, and that you're a part of it, you know?

[01:35:29] Beff: Yeah. It's, it-- again, it's, it's pretty surreal. Growing up I was, uh, I'd experiment with like lucid dreaming- Mm-hmm ... uh, a lot.

[01:35:37] John Greer: Mm-hmm.

[01:35:37] Beff: Once I got like ripped out of grad school and plopped next to sort of Sergey Brin, ever since then it's been, uh, I don't know.

[01:35:45] I f- I feel like I've been one of the characters, uh, in in, in, the game, right?

[01:35:50] John Greer: You're in the simulation, you're the player character.

[01:35:52] Beff: Y- yeah. Well, and I don't think I'm the only player. I think Elon's player one for sure. Uh, he's he's winning the game. But you know I, think everybody should a- aim to be a, a player, uh, i- in the broader game.

[01:36:04] And, you know, look, I, I think, um, civilization's a big complex organism. You can decide to, um, in your brain try to predict different scales, you know and you'll have a different job in the world. And for some people it's just, "I just wanna take care of my family. I wanna do a simple job, and I'm just like this automaton in this system, but at least I have the resources to, to keep subsi- subsisting."

[01:36:30] And some people try to predict and control at larger and larger scales. For me, I feel like I could take in a lot of variables in my brain and understand systems at large scales on long timescales and physical, large physical scales. And so it might seem like I have more agency, but it's just like I'm playing games at a larger scale, right?

[01:36:50] But anybody is able to do that with a sufficient intelligence, right? It takes a certain amount of intelligence. But there's always an equilibrium of how much complexity can you handle versus, um, you know, how much impact you can have on the world.

[01:37:04] John Greer: Also charisma, I would point out. But, uh, okay, my last two questions.

[01:37:08] Gil, how many times have you been in love?

[01:37:11] Beff: A couple times. Yeah. Some recently.

[01:37:15] John Greer: Wow.

[01:37:15] Beff: It's, not easy.

[01:37:17] John Greer: This is a big distraction.

[01:37:18] Beff: Yeah, it is a distraction, but, you know, it's, it's the biological imperative and you need to... Yeah, you you, need to, It's something to understand more, more deeply. And, and I feel like as you, as you go through that process, you understand more about your subconscious, right?

[01:37:34] And maybe your, your own biases of, your, your psychology and how to correct them, right? It's it's a, sort of journey, right? In a way, it's like, um, it's almost your brain seeking constructive trauma to to heal itself and, and, be stronger in various ways. And so I'm always looking to be anti-fragile and, and, and be a better person.

[01:37:54] And so I think it's an important part of the journey and, uh, you know, took me some time to get away from the workstation and the books, uh, in my 20s but,

[01:38:02] John Greer: uh- I can't say I would've wanted to date you with how busy you are. I need attention, and I think you wouldn't have been there. Okay, my last question is you've had even pre your fame and your persona, you've had such a, um, a long career in your young age.

[01:38:20] You've, done so many things. What are you most proud of?

[01:38:27] Beff: I guess probably what we're gonna announce soon I think for me, um, you know, coming up with the first, some of the earliest quantum algorithms for AI on quantum computers steering the quantum computing field towards differentiable programming, I think that was I've had, had that impact.

[01:38:46] I'm pretty proud of that, but, you know, it's had limited impact because the hardware hasn't caught up, and I got impatient. But I, I think with this new paradigm of computing having to-- had to reinvent both the hardware and the software, um, you know, that, that's a big achievement. And I I think in terms of impact, you know, I can't think of something that has as large...

[01:39:03] I mean, it's hard to beat, uh, getting 10,000 times more ener- uh, intelligence per watt, right? I-- That's gonna have a huge impact. And, you know, my ethos is that we need to maximize the intelligence in the universe, so there's two aspects to that. One is climbing the Kardashev scale and increasing the wattage of civilization, but the other is getting more intelligence per watt, 'cause you multiply the two, you have more intelligence for a civilization, which to me is the ultimate good.

[01:39:26] And so yeah I'm, making progress o- on this journey and, and, you know, I'm always excited about the next thing. But, uh, I always wanna surpass myself, and that's, that's the sort of growth mindset you gotta have.

[01:39:38] John Greer: You're very impressive. I hope you become an American.

[01:39:41] Beff: No, thank

[01:39:41] John Greer: you. We need you here.

[01:39:42] Beff: Thank

[01:39:42] John Greer: you. Um, wishing you l- lots of luck, and yeah. Thank you. Thank

[01:39:45] Beff: you so much.

[01:39:46] John Greer: All right. Thanks for having me. Cheers.