Other labs could still build on the leading model.
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
Users backed restricting top AI labs from using their models to slow self-improvement to avoid power imbalances and support open-source efforts, while a critic accused Anthropic of unsafe conduct.
It also has the critical benefit of avoiding a dangerous power imbalance.
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
@finbarrtimbers If they believed that, they'd be doing the opposite of what they chose.
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
@jeremyphoward they probably also shouldn’t release a big blog saying “btw we’re doing recursive self improvement now it’s sick as hell haha”
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.
Anthropic has chosen the *opposite* of the safe path: they are allowing themselves, the current top lab, to use their top model for frontier AI research. They've said they'll sabotage others who try.
This means the AI frontier advances, & power imbalance increases.
(To be clear, *I* don't think we should try to slow down recursive AI self improvement - I think we should open it up and democratize it as much as possible. My point is: if *you* claim we should slow down, and you have the best model, you should ensure your org can't use it.)
@jeremyphoward What is the incentive to train the next frontier model then
@jeremyphoward And if they don't, then it's just another case for why:
https://opensourceaimustwin.com/?share=v2
@jtlicardo If your goal is to slow down frontier AI development, then you don't want create that incentive.
@jeremyphoward That’s why AI should be open source, so that there is a check and balance in power / knowledge
@jeremyphoward I'm not sure how this would be enforceable. What would the mechanism be?
@jeremyphoward Why the second point?
Why does that slow down rsi?
Other labs could still build on the leading model.
Easy solution to slow down recursive AI self improvement:
- The lab with the top-ranked model must agree THEY must not use it for working on frontier AI - But everyone else should have access to it.
By definition, this means the frontier doesn't advance.