A large part of the problem with the Anthropic-Govt drama is the assumption that the "model" is the dangerous unit, instead of the specific usage of a model, or deployment of a model, or the whole system within which a model is deployed, or indeed (my favourite) the collective ecosystem built of many models within which we'll all be living soon.
If a model is dangerous, then you have no choice but to control the model. Because models cannot be perfectly made safe, there's no choice but to restrict access, if they're inherently dangerous.
I see no way a model that's really really good at coding will not also be really really good at hacking. If they're really good at helping create new drugs to cure cancer they're going to be really good at creating diseases. We've done remarkably well already to get them to the point that they won't by making the activation energy too high, but its not foolproof.
In other domains, when faced with hard-to-verify issues, we choose a "reasonableness" standard. Similarly, I think here we'll need to define some "reasonable" standard. Like a "reasonable person with a reasonable amount of effort cannot hack the Pentagon", and live with that hurdle.
The rest of the protection isn't just the safety work in mitigating every bad thing that can happen, but that the cost of being crazy is made high enough that if your skill in acting on your crazy is high enough you wouldn't do it.
