Rohit Krishnan, an engineer and writer focused on AI, frames power-seeking evaluation in AI as a capabilities question in exchange with Elizabeth Barnes

@krishnanrohit You're saying like "what are the capabilities of [google/nkorea/etc] to pursue particular top-down goals, e.g. if the leadership/shareholders had these goals, (but the organization is still composed of humans with whatever goals those humans currently have)"?

@Nerland87 @BethMayBarnes All three are fairly minor in the world stage, are highly stagnant, and cause a lot of internal strife with limited spillovers so they are examples of how despite power seeking (possibly!) their capabilities just do not translate much to actual power.

@BethMayBarnes (Doesn't just need to be power seeking of course using that as an example for how capable they are as per the OP)

@krishnanrohit I'd guess that NK can do some pretty nasty things with bioweapons, but the other examples are mostly inhibited from doing anything super obviously nefarious bc the individuals there would whistleblow.

@krishnanrohit And "gain and lock in as much power for yourself as possible while not doing too many things that are flagrantly evil" is pretty close to the goal of most large institutions already, so probably the extent to which we see them succeeding already is a decent proxy?

@krishnanrohit What's not quite the same? Not sure we're disagreeing?

@BethMayBarnes I don't think it's quite the same. From a capabilities POV even if Google decided they wanted to cause human disempowerment I'm not sure they'd be able to, cosntrained as they are internally and externally and by the market. It's a useful forcing function.

@BethMayBarnes Yes

@BethMayBarnes Google today doesn't have sufficiently "power seeking" style goals that would make their existing state a proxy for their capabilities to do so as per the original tweet. I think they could do quite a bit more, but don't.

@krishnanrohit Ah ok. Yeah that seems reasonable to me, not sure. It seems plausible that if they were more abjectly powerseeking they'd lose employees and end up in a weaker position?

@krishnanrohit @BethMayBarnes Iran and Venezuela had surges of capability with oil windfall and this has had long term effects that put them on pretty bad paths?

@BethMayBarnes Idk we did have east India company and robber barons in the past and were barely a year away from the ESG heights, so it's plausible to me

@krishnanrohit @BethMayBarnes Within NK there is a tiny elite with vast capabilities over the population and cause great disempowerment?

You could argue VOC and East India, but more likely some oligarchical elites in most LMICs/Russia also do this.

Seems pretty clearly in the distribution?