Reactions from ranked influencers
12 posts
@mervenoyann lol, seriously? have you seen what I've said about AGI governance in general? what exactly is your approach to the risks open models will pose in the near term or what is your mitigation idea to avoid them posing risks in the first place?
What's the massive amount of empirical evidence on "your side"? The only advantage "your side" has is that open models are still too shite to cause massive harms but this is changing really quickly and you're going to look like a fool for ignoring the obvious risks and downsides of a position that will be hard to undo
To use your rhetorical device: The only real advantage "your side" has is that open models are still too shite to cause massive harms but this is changing really quickly As soon as open models are sufficiently good to conduct cyber attacks (and closed models are sufficiently locked down), you'll see an uptick in malicious usage of open models The other advantage that open models have is that it's much harder to monitor and detect malicious usage with them, so it will take a while longer for any awareness while "your side" will argue for the apparent benefits and lack of harmful uses
None of these points says how future risks will be mitigated though? > closed models don't help contain issues either so if it's not for open ones you're helpless I think HF just got accepted into OAI's trusted access program. For closed models, making sure that advanced access is conditional makes a lot of sense > abliteration makes models visibly dumber So we wait another 6 months to offset this? What then? > transparency [...] How does that help for mitigation?
Not banning but regulating access or making sure that open base models don't have strong offensive cyber capabilities (e.g.,) For example, if you want a strong cyber defensive model, you could train it on all things cyber and monitoring but make it less good at agentic coding in mid training, and then release that. I hope someone is exploring that I'm sure there are ways to have open models that lack narrow risky capabilities (and are fine to stay open bc you'd need to spend $$$$ on inference compute to try to learn those from 0 using trial and error)
What's the massive amount of empirical evidence on "your side"? The only real advantage "your side" has is that open models are still too shite to cause massive harms but this is changing really quickly As soon as open models are sufficiently good to conduct cyber attacks and closed models are sufficiently locked down you'll see an uptick in malicious usage of open models The other advantage that open models have is that it's much harder to monitor and detect malicious usage, so it will take a while longer for any awareness while "your side" will argue for the apparent benefits and lack of harmful uses The downside of this position is that it will be hard to undo once it's out of the box