Richard Ngo says technical AI safety research would remain ineffective under slowed progress because methods fail to generalize and researchers stay politically engaged, in exchange with Max Nadeau
Nadeau cites article arguing superintelligence needs real-world deployment beyond datacenters.
@MaxNadeau_ If progress slowed to a crawl, then the field would still be fake insofar as it’s still aiming to align AGIs, and also insofar as researchers are still trying to do politics in the way discussed in my previous tweet. But it’d be a less clear-cut case than it is today.
Max Nadeau@MaxNadeau_FWIW I also don't think my QT'd objection applies to your tweet today. A different question, though: suppose you knew AI progress was about to slow to a crawl (to, say, the speed of real-world data collection: https://meagreprotestanthistory.substack.com/p/the-goodhart-singularity). Would you still think technical AI safety research was a fake field? I'm assuming "no", because your prior tweet implies your big issue with the field is that the techniques "will. not. generalize" to future, more capable models. I don't think "fake" is an accurate way of summarizing that critique, and I think the "fakeness" of psych is pretty disanalogous. I think a better analogy for the "fakeness" is if you call 1980s psychology "fake" because its findings don't generalize to monkeys or intelligence-augmented humans. And tbc I *agree* that (on current margin) fewer capable, impact-chasing people should work on safety techniques that won't scale, and more of them should work on techniques that might.
To be clear, fakeness as I mean it is affected by how the field presents itself, in particular the extent to which it claims precedence over other approaches.
Academic psychology (and academic disciplines in general) hold themselves as far superior to non-academic work—to the point where they often define science in terms of engagement with academic institutions (like peer reviewed journals).
Meanwhile mainstream AI safety holds itself as far more valuable than non-AI-safety work, and indeed often defines what work is good for the world in terms of how entangled that work is with the AI safety community.
These factors make me judge both as more fake than I would otherwise.
Richard Ngo@RichardMCNgo@MaxNadeau_ If progress slowed to a crawl, then the field would still be fake insofar as it’s still aiming to align AGIs, and also insofar as researchers are still trying to do politics in the way discussed in my previous tweet. But it’d be a less clear-cut case than it is today.