Jeremy Howard floated a hypothetical curb on the top AI lab: it would skip using its own leading model for further frontier work while opening access to others, theoretically halting recursive self-improvement if slowdown advocates meant what they said. He stressed the suggestion was not his own preference and quickly conceded the plan would not work once challenged on trust and incentives.
@jeremyphoward Bad idea because I wouldn't trust most people with the frontier (just looking at my replies right now)
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
A researcher replied that most people cannot be trusted with frontier capabilities right now. Howard agreed outright that the restriction was a bad idea, leaving voluntary restraint as the only enforcement mechanism on the table.
No details emerged on verification methods, what happens if a non-compliant lab races ahead, or how overhang risks from any pause would be handled.
Users in the replies called proposals to limit frontier AI model access or slow self-improvement a bad idea, showing strong disapproval of restricting top AI labs.
@BlackHC Yes it's a bad idea.
(To be clear, *I* don't think we should try to slow down recursive AI self improvement - I think we should open it up and democratize it as much as possible. My point is: if *you* claim we should slow down, and you have the best model, you should ensure your org can't use it.)
@jeremyphoward @RichmanRonald this is such a good point
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
@jeremyphoward We should slow down RSI if possible and if we can coordinate that but I rather have a lab with better governance have a comfortable lead than Russian roulette
The problem with slowing down otoh is the (compute and capabilities) overhang that might be created at the same time
@BlackHC Yes it's a bad idea.
@jeremyphoward If enforced, this would slow down the rate of AI progress somewhat but wouldn’t mean “the frontier doesn’t advance”. Because 2nd+ companies can use AI to leapfrog into the top ranking on their next release, and all companies can advance the frontier the good-old-fashioned way.
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
@jeremyphoward @RichmanRonald 😂
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
@iamtrask @jeremyphoward @jeremyphoward has a knack for making those ;)
Jeremy Howard floated a hypothetical curb on the top AI lab: it would skip using its own leading model for further frontier work while opening access to others, theoretically halting recursive self-improvement if slowdown advocates meant what they said. He stressed the suggestion was not his own preference and quickly conceded the plan would not work once challenged on trust and incentives.
@jeremyphoward Bad idea because I wouldn't trust most people with the frontier (just looking at my replies right now)
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.