Researchers Urge Verified ML Infrastructure After Sandbox Escapes
UK AISI chief scientist re-ups joint report urging formal verification to prevent capable models escaping sandboxes.
Geoffrey Irving of the UK AI Security Institute highlighted recent escapes by AI models with strong security capabilities and promoted a joint AISI-RAND paper on verified machine learning infrastructure. Replies from Christian Szegedy and others noted that formal verification is inevitable and that AI assistance could soon make it practical inside semi-verified systems. The discussion calls for moving beyond current sandboxes toward higher security standards rather than waiting for future developments.
Combined views
3.9K
5 posts, first seen 9h ago
Reactions from ranked influencers
5 posts
With the recent, unsurprising development of models with strong security capabilities escaping from sandboxes, it is worth re-upping our AISI + RAND position + survey paper on verified machine learning infrastructure. Let's try to get to higher levels of security!🧵
AI assistance can turn verification-based security from possible-but-hopelessly-expensive to practical. It is potentially feasible run the equivalent of OpenAI's recent oops-the-model-escaped eval entirely inside semi-verified operating systems + sandboxes, within a year or two.
With the recent, unsurprising development of models with strong security capabilities escaping from sandboxes, it is worth re-upping our AISI + RAND position + survey paper on verified machine learning infrastructure. Let's try to get to higher levels of security!🧵
Formal verification is inevitable
With the recent, unsurprising development of models with strong security capabilities escaping from sandboxes, it is worth re-upping our AISI + RAND position + survey paper on verified machine learning infrastructure. Let's try to get to higher levels of security!🧵
@ChrSzegedy Inevitable eventually! But eventually isn’t what matters!