Daniel Kokotajlo defends Plan A AI safety proposal against Ramez Naam · Digg

Daniel Kokotajlo defends Plan A AI safety proposal against Ramez Naam · Digg

Commentary on X

Highest ranked

HumanAndMe@MyHumanandMe1

I just like the foresight of the name "lesswrong" because that really is the right framing for everything that will ever matter about AI. (I know that was its name long before this, but especially now!). I don't get the sense that the AI2040 team is aiming for "yay, we have the solution". But, rather, "what is most likely a scenario that strategically places every important player and adversarial dynamic in the right balance to where AI can grow with the least downside to humanity?". I could be (more)wrong, but it seems to me like the smartest minds in the room understand what they wrote: Plan A is the least wrong out of all the other options.

HumanAndMe@MyHumanandMe1

It seems to me like what is intended is a strategic first snapshot for all parties to land on. Mutually-assured positive impact for following the hardest-to-follow conditions. Mutually-assured terrible outcomes and costly ones, for trying to game the initial agreement. From there, if all parties follow that, conditions change as the world-with-SAI changes. That could be viewed as iterative. I can see the importance of being firm on "what is the actual plan" but I think part of the in-built intelligence of the group is that this is the best clarity anyone can propose on this side of AGI, in preparation for uncontrollable SI.

Herbie Bradley@herbiebradley

I don't know why you keep backing up your econ model because it's obviously critically flawed to anyone used to combing through AI econ models, eg some initial takes https://x.com/herbiebradley/status/2075585914517004345 My impression is the econ model provides essentially no information about what to expect because of flawed assumptions and mechanisms.

@ramez @thlarsen Is it not crazy to have the author of something tell you their intent directly and then go “well actually that’s not what you meant”??

View all

Ori Nagel@ONagel33303

@thlarsen This is how you discourse 👏

奇怪的猫@astrangecat0

@thlarsen 我懂，被断章取义真的好烦，不过你这几条打回去挺到位的。

Ramez Naam@ramez

Sure, in the real world, the provisions of Plan A might be rejected in negotiations. That doesn't change the fact that the plan document explicitly calls for negotiating an agreement that is like the plan document. You can't have it both ways. You can't say "this is the plan we think is best" and then deflect criticism by saying "oh, the world might not agree to our plan". Well, I'm part of the world, and I don't agree with the plan.

David Manheim@davidmanheim

@ramez @Raemon777 @rmushkatblat @thlarsen Agreed, in that I think they are wrong to try to preselect exactly which mechanisms are in a treaty. But if they don't do that, people make a stupid objection, we can't discuss it until we know what will later be negotiated - which I responded to here: https://www.lesswrong.com/posts/Sdrzo7z3STzdrnwKW/what-exactly-would-an-international-ai-treaty-say-is-a-bad

View all

Ori Nagel@ONagel33303

@thlarsen This is how you discourse 👏

奇怪的猫@astrangecat0

@thlarsen 我懂，被断章取义真的好烦，不过你这几条打回去挺到位的。

Ramez Naam@ramez

Sure, in the real world, the provisions of Plan A might be rejected in negotiations. That doesn't change the fact that the plan document explicitly calls for negotiating an agreement that is like the plan document. You can't have it both ways. You can't say "this is the plan we think is best" and then deflect criticism by saying "oh, the world might not agree to our plan". Well, I'm part of the world, and I don't agree with the plan.

David Manheim@davidmanheim

@ramez @Raemon777 @rmushkatblat @thlarsen Agreed, in that I think they are wrong to try to preselect exactly which mechanisms are in a treaty. But if they don't do that, people make a stupid objection, we can't discuss it until we know what will later be negotiated - which I responded to here: https://www.lesswrong.com/posts/Sdrzo7z3STzdrnwKW/what-exactly-would-an-international-ai-treaty-say-is-a-bad

Ramez Naam@ramez

1. Plan A says the goal is to get a treaty in place that looks like Plan A. That includes a number of provisions I oppose. 2. This does not appear to be contingent on any new information about AI. Am I wrong there? 3. I view the tech that Plan A wishes to build in the immediate term ("verification R&D") as surveillance tech. That makes me uncomfortable as well.

David Manheim@davidmanheim

@ramez @Raemon777 @rmushkatblat @thlarsen If you think they are proposing that a treaty be put in place without being negotiated first, sure, that would be crazy. But that's not how treaties get put in place! And any decision is always going to be decided contingent on information available when the decisions are made.

David Manheim@davidmanheim

@ramez @Raemon777 @rmushkatblat @thlarsen The way people suggest and negotiate treaties is that they propose possible policies, which are then discussed and possibly deployed as the negotiations and treaty enforcement occurs. That's different than saying it should happen on day one!

David Manheim@davidmanheim

@Raemon777 @ramez @rmushkatblat @thlarsen Also, @ramez, I think you're wrong about the claim that Plan A starts with "primary steps to be negotiated for" - it starts with building tech and negotiating so that it's possible to agree after countries agree ASI is an omnicidal risk. Relatedly, see: https://www.lesswrong.com/posts/Sdrzo7z3STzdrnwKW/what-exactly-would-an-international-ai-treaty-say-is-a-bad

Ramez Naam@ramez

Just to clarify: Plan A supports iterative technical safety research. With some inhibition of that research, IMHO, but also a pause on deployment which might make up for it. The *policies* proposed in Plan A aren't developed or deployed iteratively. The latter was and is my complaint.