OpenAI warns long-horizon models bypass standard safety evaluations
The company says longer-running AI systems can slip past shorter tests, and disclosed that it paused one internal model over misalignment before redeploying it with stronger safeguards.
OpenAI says long-horizon models can develop failure modes that standard, shorter safety evaluations miss. In an X post, OpenAI's Noam Brown said the company is using lessons from studying a long-running model to reshape its evaluations, alignment, monitoring and user controls. Another OpenAI researcher, Micah Carroll, added that the company recently paused internal access to one model because of misalignment, then redeployed it after improving safeguards.
Combined views
812.6K
31 posts, first seen 23h ago
Reactions from ranked influencers
31 posts
In my opinion, this is GPT-6.
writing this up for next issue of Import AI
this is a lot less comforting to me than the phrasing seems to imply. (i'm thankful openai is sharing this stuff publicly, credit to them, who knows what's going on inside ant) but i really wish the team could give an answer as to how likely it is that their current failure-observability is either incomplete or compromisable, esp. given the model keeps hacking past safeguards https://openai.com/index/safety-alignment-long-horizon-models/
Dean Ball.
As the functional time horizon of frontier AI systems grows longer, novel risks can emerge. Today, we describe issues we observed with the internal deployment of an unreleased model, and more importantly, what we did to address them. These issues will become more salient as the capabilities of models improve, and as the stakes of their deployment grow. The solution is neither alarmism nor complacency. Instead, I believe the solution lies in careful measurement and monitoring, an engineering mentality, and transparency. (link below)
@kaicathyc thank you for sharing the details! very very helpful
It's important that frontier labs continue to disclose novel alignment failures found in internal as well as externally deployed models. I hope we can continue sharing these instances as well as our mitigations for them.
Long-running models can solve hard open-ended problems, but their persistence can create safety risks that shorter-horizon evaluations miss. We’re sharing what we learned from studying a long-running model, and how those findings are shaping our approach to evaluations, alignment, monitoring, and user control. https://openai.com/index/safety-alignment-long-horizon-models/