Reactions from ranked influencers
4 posts
1) It was and remains an incredible unforced error that the AI Safety community leaned so heavily into attempts to ban open-source models (often in thinly veiled trojan horses). It was a bad move strategically and relied too heavily on speculative/Trojanunjustified reasoning.
2) It's important to remember that the OS community didn't respond by saying "yes, there are risks and we should accept them and move ASAP to mitigate them." The standard argument was that superhuman cyber capabilities were speculative/unrealistic.
@RyanGreenblatt From my recollection, a lot of people believed banning OS models would be necessary, proposed regulations that likely would ban open source models, and dismissed concerns that proposals would harm the open source ecosystem.
@dhadfieldmenell Did they lean heavily into attempts to ban open-weight models? I think some people did stuff in this direction or thought this was a good idea, but this seems like a big overstatement. I agree this doesn't make sense.