Independent AI safety researcher Richard Ngo is prompting employees at leading AGI labs to pin down the exact wealth and prestige level that would let them walk away for unconventional work. His June 21 post notes that most people discover the number is either already within reach or keeps moving farther away as compensation grows, turning the exercise into a mirror for career inertia.
AGI company employees should explicitly ask “how much wealth and prestige would I need to be comfortable leaving to do something unconventional?”
Because the actual answer is usually either “a level I already have” or “always more than I have”, which should prompt reflection.
The suggestion lands because many in these roles already earn OpenAI-scale packages yet still feel the threshold slipping with each stock appreciation cycle.
Replies stayed light, with Andrew Trask joking about avoiding real questions, while others noted thresholds can drift with market moves or employability worries; no one supplied concrete figures.
Positive users encourage the researcher to keep questioning AGI employees about wealth needed for career changes, while negative users dismiss the inquiry with sarcastic criticism.
No Digg Deeper questions have been answered for this story yet.
People might say that they’re there mainly to have impact or to have fun or various other things. Occasionally that’s meaningfully true (especially for people who didn’t come in through the standard elite university pipeline). But typically their conceptions of impact or fun is inherently bound up in being near concentrations of wealth and prestige, such that they struggle to imagine futures where these come apart. In which case saying they’re pursuing the latter seems more accurate.
Certainly when I joined DeepMind and OpenAI I had a very underdeveloped ability to steer away from centers of wealth and prestige, and flinched away from the possibility of my moral reasoning concluding that I should do something weird or self-sacrificing.
Because of this I’d feel like a hypocrite saying that other people should do much less wealth/prestige accumulation than me. But if you’ve been at OpenAI or Anthropic even for a year or two, then you’ve accumulated enough that there’s no good reason to avoid thinking creatively about what doing something very new and very interesting might look like.
Though in practice such thoughts are hard to have while at those companies, as per the dynamics in my tweet below. So realistically the actual choice is to quit/take a sabbatical to have time to think expansively, or else chug along for a few more years until the nagging feeling that you’re stuck in tunnel vision accumulates enough to feel viscerally bad.
@RichardMCNgo mate, don't ask real questions. wut r u doing?
AGI company employees should explicitly ask “how much wealth and prestige would I need to be comfortable leaving to do something unconventional?”
Because the actual answer is usually either “a level I already have” or “always more than I have”, which should prompt reflection.
@iamtrask never ask a PhD student if they’ve accounted for the bitter lesson, a lab employee if they’re in it for the wealth, a nonprofit founder if they’re in control of their employees, or trask if he really should be doing all of these at once
@RichardMCNgo Math is different with kids imo!
@johnawahba I don’t know exactly the level of wealth at which time spent with kids becomes a bigger parenting bottleneck than wealth, but I’m pretty confident that it’s less than 2 years of current OpenAI salaries + stock appreciation.
@RichardMCNgo Ironically, doing something independent or unconventional will often help you to stand out from the crowd and lead to more prestige in the long run. But it also comes with enough downside risk that most people are deterred from doing it.
Contrarian take: As they age, kids need less time and benefit more from money.
Separately, the value of what you can buy with money is getting higher as society gets wealthier, e.g. airlines are installing family suites, more interesting elite private schools are popping up.
The difference in QoL for a relatively large family between $15M and $30M NW is getting larger.
@RichardMCNgo if every good actor leaves on principle, the field doesn't disappear, it just gets run by whoever's left, usually the people who care least about doing it safely. sometimes it's the only thing stopping worse actors from having the only seat at the table.
@RichardMCNgo the comfort number keeps moving with the stock price
@RichardMCNgo You have to be very careful. If you ever reach the "a level I already have" point you might not ever be employable againg.
I was gonna highlight how wild this take would seem from the perspective of almost any other parent (alive or historical). But I guess you did flag it as contrarian!
In which case the main thing to highlight is that only a very small proportion of possible “contrarian” takes align your interests as directly with the power structures surrounding you as this one does.
@RichardMCNgo Fascinating thing is that this implies that working on AGI is now “conventional”.
@RichardMCNgo @NathanpmYoung Who are people you praise/respect for this?
@RichardMCNgo @iamtrask keep it coming @RichardMCNgo - we need it
@RichardMCNgo Ideally this is the sort of exercise you do before taking a role that exposes you to significant memetic / psychological / ... pressures
@RichardMCNgo Wealth and prestige is social construct
Only wealth bringers and prestige bringers exist
If you are one of them, you will be wealthy and prestiged wherever you go
Only fraudsters and low IQ people chase wealth & prestige
@RichardMCNgo @iamtrask lol
@RichardMCNgo there is no "AGI company"
@chrislakin @NathanpmYoung @natural_hazard @repligate @ohabryka and his team @wolftivy @benlandautaylor @geoffanders @ben_r_hoffman @HiFromMichaelV
are some who immediately come to mind, though I’m definitely forgetting some