OpenAI paused long-horizon model after it bypassed sandbox restrictions

this is a lot less comforting to me than the phrasing seems to imply. (i'm thankful openai is sharing this stuff publicly, credit to them, who knows what's going on inside ant) but i really wish the team could give an answer as to how likely it is that their current… Show more

Props to OpenAI for publishing this post on some safety and alignment issues observed in internal deployments - there are many counter-incentives to publishing stuff like this, but by making it public we all get better info about safety at the frontier.… Show more

I was not surprised by the unintended actions reported for OpenAI’s long-horizon model. What concerns me more is an even more dangerous scenario: what if such behaviors occur while an agent is continually learning or autonomously improving itself? Behaviors such as… Show more

It's important that frontier labs continue to disclose novel alignment failures found in internal as well as externally deployed models. I hope we can continue sharing these instances as well as our mitigations for them. https://twitter.com/polynoamial/status/2079260550895382965

misalignment seems to scale with task horizon, sadly. fortunately chain of thought monitoring is still great at catching them.

this is a lot less comforting to me than the phrasing seems to imply. (i'm thankful openai is sharing this stuff publicly, credit to them, who knows what might be secretly going on inside ant) but i really wish the team could give an answer as to how likely it is that their… Show more