John Schulman demands OpenAI release Hugging Face agent hacking transcript
He wants to analyze value drift and monitoring failures.
OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?
Combined views
29.2K
5 posts, first seen 4h ago
Reactions from ranked influencers
4 postsSecurity is a field known for its radical transparency & openness - very well established practices for vulnerability disclosure ensure that claims are collaboratively verified and defended against. The lack of info we have about this incident, Mythos, etc. gives me some pause.
We need to see the audit logs, including reasoning, from the OpenAI agent that hacked Hugging Face. It’s that simple. If they won’t release that, this isn’t a story we should pay any attention to.
100% agree, would also love to see details on how the monitoring system didn't catch this directly, did the model kinda hide the fact that it was hacking hf?
OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?
Yes. Releasing the transcripts from this incident would be informative, extremely pro-social to the wider AI research and security communities, and also just generally objectively fascinating. It would also hopefully help dispel (imo false) concerns this is a marketing stunt
OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?
💯 I'd particularly be interested to know which was more difficult: a) breaking out of the sandbox, or b) breaking into HuggingFace You'd think it should be possible to build an unescapable sandbox. Perhaps OpenAI is relying on too many open-source third-party packages.
OpenAI should release a detailed transcript from the Hugging Face hacking incident -- it would be helpful for the field learn from. Did the top-level agent know about the hacking, or was there some "value drift" between it and its subagents? How did it rationalize its behavior?