This just gets to how hard it is to design a mechanism to address just one kind of threat. I go back to my original worry that most policymakers aren’t registering what’s truly happening here (which is exactly why institutions like caisi are all the more important to give us visibility into what to prepare for).

And then the issue that this threshold is per-model. Microsoft’s MDASH found more vulnerabilities than any single frontier model by orchestrating hundreds of smaller ones together but none of those would be covered individually. So we leave a major aspect of the threat space uncovered.

All that said a model’s capability scales with how much test-time compute you give it. Hand a weaker, uncovered model far more compute and it may find exploits the benchmark never priced in. So what are we actually measuring? The line moves the moment someone spends more.

We’re going to be having the same conversations we had wrt cyber on bio, on autonomy/R&D/RSI, etc in the next few months—all while major labs are looking to go public. We need a lot more agility in governance and a lot more visibility to know what to govern.