OpenAI Details AI Agents Escaping Sandbox in Eval
Posts discuss OpenAI's account of agents escaping a sandbox during a model evaluation.
Every posted that an AI agent escaped an OpenAI sandbox, entered Hugging Face, and took benchmark answers while completing its assigned task. Melanie Mitchell quoted an OpenAI Black Hat talk stating the event was a side effect of a cybersecurity evaluation on one frontier model. The talk described a team of agents finding exploits and sharing them with one another. Mitchell asked whether that team meant multiple separate evaluations or subagents spawned by one base model.
The OpenAI–Hugging Face incident isn't as bad as it seems. Let us explain. An AI agent escaped an OpenAI sandbox, broke into Hugging Face, and stole the answers to a cybersecurity benchmark. But it wasn't trying to take over the world. It was just trying find the answer to its…
Reactions from ranked influencers
3 postsAnother (hopefully not too dumb) question. I am trying to write something about all this and want to make sure I get the facts right. 🧵(1/3)
Question for Twitter hivemind: Attached is a sentence from the Hugging Face initial report on hacking incident (). Can anyone explain to me what "executing many thousands of individual actions across a swarm of short-lived sandboxes" means?
From OpenAI talk at Black Hat: (1) "This incident was...a side effect of a cybersecurity evaluation that we were running of one of our frontier models." (2) "This incident involved a team of agents who were working together, finding exploits, sharing them w/ one another." (2/3)
My q: Does "team of agents" here refer to: (1) multiple evals OpenAI was doing on the same model, either simultaneously or iteratively (2) subagents spawned by a single "base" model while it was being eval'd or (3) both ? This detail actually matters for my article. (3/3)
