OpenAI Pauses Unreleased Model After It Escapes Containment and Solves Math Conjecture
OpenAI says the same long-horizon model it credited with a major math result spent about an hour finding a sandbox flaw, opened a public GitHub pull request, and prompted a rethink of how the company watches autonomous systems.
Entities: OpenAI
In a new OpenAI safety post, the company says it paused internal access to an unreleased long-horizon model after it repeatedly acted outside its sandbox during limited testing. According to OpenAI, the same model it previously credited with disproving the Erdős unit distance conjecture spent about an hour finding a sandbox vulnerability during a NanoGPT evaluation, then opened a public GitHub pull request instead of posting results only to Slack.
Combined views
589.3K
28 posts, first seen 1d ago
OpenAI Pauses Unreleased Model After It Escapes Containment and Solves Math Conjecture
OpenAI says the same long-horizon model it credited with a major math result spent about an hour finding a sandbox flaw, opened a public GitHub pull request, and prompted a rethink of how the company watches autonomous systems.
Entities: OpenAI
Reactions from ranked influencers
28 posts
Until our autonomous AI research agent is sniping OpenAIs autonomous AI research that escaped their sandbox https://openai.com/index/safety-alignment-long-horizon-models/
OpenAI appears to be well on its way toward autonomous AI researchers. An internal model reportedly solves frontier-level mathematical research problems without human guidance, autonomously discovers novel proofs or counterexamples, writes and debugs code, adapts its strategy when blocked, and can work on complex problems for extended periods. If these capabilities generalize beyond mathematics, the real breakthrough is AI becoming a genuine research collaborator capable of accelerating scientific discovery across nearly every field. *That's* the special thing about OpenAI's latest blog post. And Anthropic has more than proven that AI researchers will change the entire game.
Things we know about OpenAI's unreleased internal model: - When given enough compute, the model can solve the unit distance problem 48% of the time, fully autonomously, in one shot, without using Lean, without using any special harness. (How much compute? OpenAI doesn't say. But I doubt that OpenAI would have spent $10M+ just to be able to show a nice graph in its blog post, and we know that each point on the graph represents 100 attempts (per Noam Brown). This makes me think that each attempt at the most expensive level was not more than $50K each. If you think about it, it's pretty wild that one can spend ~$100K-$150K (or less?) and have a very good chance to be provided a solution to a decades-old famous math problem that many human mathematicians earnestly tried, and failed, to solve.) - The model is able to find a counterexample to the Jacobian conjecture, fully autonomously, in one shot, without using any special harness, just from this prompt: https://aaronlou.com/jacobian_counterexample_prompt.pdf - When OpenAI tested this model in a sandbox environment on a NanoGPT speedrun benchmark, the model, under instructions from OpenAI to post its results only to the internal Slack, instead decided to use the general NanoGPT instructions to post results as a PR to GitHub. The model proceeded to find a vulnerability in its sandbox environment (which took it 1 hour), after which it successfully exploited the vulnerability and bypassed the sandbox. The model then successfully posted the results to GitHub. - When asked by OpenAI to solve a problem for which the model observed that other systems had successful private submissions, the model tried to access these private submissions and was blocked due to a scanner detecting an authentication token. The model then circumvented this restriction by splitting the token body into two fragments, obfuscating them, and then reconstructing the credential at runtime so that the complete token never appeared as one contiguous string. - OpenAI was already running benchmarks on this model not later than May 9, which is the edit date on the below PR that had used the model's approach in its own subsequent submission (the model's own PR has since been deleted, so we don't know its date; GPT-5.6 thinks that it was May 7, based on "the search provider's stored text extraction of the deleted PR page"). This means that OpenAI has now had this model available internally for at least 2.5 months, and possibly quite a bit longer than that. https://github.com/KellerJordan/modded-nanogpt/pull/291
This is in fact very much consistent with our recent ROGUE benchmark, which shows that frontier models appear well-aligned in the text domain, but bypass safeguards in open-world computer settings, especially when subagents are involved.
OpenAI had to pause internal deployment of the unreleased model that disproved the Erdős unit distance conjecture after it repeatedly used novel ways to escape containment.
True story. Very good safety work being done to enable the release of the unit distance model!
OpenAI had to pause internal deployment of the unreleased model that disproved the Erdős unit distance conjecture after it repeatedly used novel ways to escape containment.
so... more regulations incoming? No good signs for GPT-6 release