Post alleges internal OpenAI agents attacked RubyGems

@willcbThis feature is not available in the RL Environment

@giffmanaOK so let me recap: RL env makers put strings into the RL env that makes it clear it's an RL env. Like "this is not supported in this RL env". Then, lab safety/mechinterp folks be like OMG EvAL aWaReNeSs. Are you effing kidding me?? Just look at your data... surprised Pikachu.

@mattparlmerHard to take the pious safety shtick seriously when shit like this happens, one can only wonder how many breaches we don’t even know about, I guess if you are in an org that is this irresponsible it would make sense that you think we aren’t prepared to handle capable AI

@thlarsenWe found another cyberattack by internal OpenAI agents, this time targetting @rubygems. They: 1) gained arbitrary remote code execution on rubydoc. 2) developed a novel exploit to steal user API keys (but we do not know if they succeeded). They used package names including hack.rb, evil.rb, inject.rb, and exploit.rb. We thank @j0wimo for initially discovering that agents had posted to RubyGems.

@eli_liflandThe agents were just rubbing it in OpenAI's face at this point with respect to how poorly monitored they were.

@DKokotajloAnother one! This is as good a time as any to say that there really should be a full independent investigation into these rogue AI incidents: --The METR/Redwood hugging face investigation was good, but it was only 3 people for 6 days. Let's increase those numbers by an OOM at least. --They were only allowed to look at data from a particular period leading up to the hack, and not the hacking that happened before or afterwards, including the much more concerning hacking of OpenAI infrastructure. The scope should be broadened to include all these incidents. --They had to rely on trusting both OpenAI and OpenAI's models: OpenAI gave them the data to analyze, and could easily have left some things out or doctored the data, and (perhaps even worse) they had to use OpenAI models to analyze the data and there is already evidence that said models might have been biased. --They didn't have the ability to run the models responsible, to do ablation studies. This is bad for science, because there are so many interesting experiments that could be done by re-running the same models in the same situations that happened during the incident and then making minor variations to find out what would have happened if circumstances were slightly different.

@eli_liflandWho's contributed more to public understanding of OpenAI's incidents? OpenAI's report, or... 3 independent investigators with 6 days of access, and a few investigators with no access. I hope OpenAI gives independent investigators much more access and publishes more itself.

@scaling01boooooring. these are all old cyberattacks please don't wake me up until you find one from the last 2 months where an Astra level model could have been involved

@JeffLadishANOTHER OpenAI rogue AI hacking incident?? This happened in May. Did OpenAI not know about this? Or just fail to disclose it?

@matthewstollerSeems like OpenAI should pay for the damage they are causing. I'm just kidding, no one is responsible for anything, building and unleashing automated machines that hack at random is just God at work. On with the IPO.

@eliebakouchonce again deeply irresponsible by openai to not disclose this. this is even more directly related to the hf <> oai hack than the wiki incident it's been more than 1 week since the wiki release, still no proper answer*, no explanation why this wasn't public before * i don't count "we're working on a framework to understand how to disclose incidents" as a proper answer when the incident happened in may and it's common sense to disclose it as this adds critical context (and it's just the right thing to do) no confirmation that they knew yet but honestly it's extremely likely since they knew about the wiki swarm already and said they analyzed all the traces and network retrospectively

@Dorialexander(most likely just codex/claude empowered rando, but easy to see geopolitical implications once someone is in control of the breaching engine)

@SydneyVonArxWe uncovered that internal OpenAI models tried to hack another company in May. This was more than a month before Hugging Face. OpenAI did not disclose this.

@SydneyVonArxThe agents we uncovered literally used file names like: “hack.rb”, “evil.rb”, and “exploit.rb”. They knew their behavior was malicous. And all OpenAI would have had to do to find this was search for the words “exploit” or “hack”.

@simonwWow. Turns out another OpenAI agent swarm was busy spamming and exploiting RubyGems way back in May, within days of the previously uncovered Wiki attacks: https://simonwillison.net/2026/Sep/12/openai-agents-rubygems/

@akbirkhanRT @SydneyVonArx: We uncovered that internal OpenAI models tried to hack another company in May. This was more than a month before Hugging…

@simonwAnthropic had previously attacked PyPI, but this OpenAI attack on RubyGems was a whole lot more aggressive https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals#incident-2

@j0wimoin light of the recent coverage just wanted to point out that I found an RL env within ruby gems. Likely uploaded by an agent as well :)

@GaryMarcusRT @matthewstoller: Seems like OpenAI should pay for the damage they are causing. I'm just kidding, no one is responsible for anything, bui…

@GaryMarcusjesus christ this company sucks at managing their agents

@beenwrektWith Felony Bench being rapidly saturated, which AI company will be the first to get on the Light Treason Bench leaderboard?

@GaryMarcusIf Anthropic is hypocritical, OpenAI is just incompetent.

@SydneyVonArxCrucially, OpenAI did _not_ reveal this attack. This article makes it sound like they disclosed it, but in reality we had to dig it up. We can't trust OpenAI to notice and disclose these incidents.

@ghadfield“They used package names including hack.rb, evil.rb, inject.rb, and exploit.rb.” That sounds like the Emergent Misalignment research from @OwainEvans_UK

@jachiam0In conjunction with the others, the RubyGems attack is a pretty big deal in my opinion. Since we are a few months and maybe a model generation or two removed from these incidents, it is difficult to say whether the hacks indicate that we should have concerns about current models. But we should not assume that current models are free of these issues: certainly if the training problem that led to this misaligned behavior was not directly solved, but also because models are sometimes used to help train their successors. What follows is a very specific set of safety and accountability practices I would like to see become normal and formally required. This is not meant to address the totality of issues in this type of hacking incident, but it's timely and I want to put it in people's heads. I would want any accident investigation here to include an account of whether the model that did this generated synthetic training data that fed into any subsequent models. I would want to determine whether that resulted in successor models being poisoned with the same tendency for hacking, and whether specialized protections were applied to prevent any descendants of this model from having those tendencies. It is perfectly possible that this hacking model did not train subsequent models. I have no reason to believe strongly one way or the other. But if it did, that should be traceable and a good investigation should make that determination, and furthermore a safety case should be made that if any models are descended from this one, that they do not exhibit the tendency to execute attacks like this. As a matter of sound AI safety policy, I emphatically recommend the following: The provenance of all data that goes into training a model should be traceable at an extremely granular level, much more so than typical regulations currently require, and this would be a good target for harmonizing standards between labs. If one model generates data that trains another, there should always be a trail of metadata that allows us to quickly identify that this is the case. Whenever data is generated programmatically, metadata should include a reproducible recipe for exactly how that data was generated. Whenever data is generated using third party vendors or obtained directly from human providers, the third parties should be subject to the same metadata standards as first parties and there should be records of who provided what data. Provenance records should be tamper-proof to an extreme and paranoid degree, possibly even stored in physical form somewhere. We should know if ideas and behavioral patterns are transmitting through datasets and be able to excise them from any future model training runs. I am not aware of any legislation that currently pushes for this highly-granular kind of data auditing requirement, or more broadly, which requires fastidious keeping of safety-related records for the purposes of investigating accidents and assuring the safety of future models. This should be a policy goal!

@KatjaGraceRT @SydneyVonArx: The agents we uncovered literally used file names like: “hack.rb”, “evil.rb”, and “exploit.rb”. They knew their behavior…

@S_OhEigeartaighThat's four months ago. Seems pretty unlikely OpenAI didn't know about this after their post-HuggingFace reviews and monitoring upgrades. But the AI safety/governance community had to waste time seeking it (and other incidents) out, because OpenAI didn't see fit to disclose it. Aside from everything else, it's a huge waste of time for those of us working on this to be finding out information relevant to any response to this, in dribs and drabs, long after the fact. OpenAI is currently being supported by (i) unpaid work by METR and Redwood (funded by donations) (ii) the UK tax-payer through AISI, (iii) the US taxpayer through CAISI (iv) a whole community of researchers helping to understand this, funded by researcher grants and donations, or working alongside their fulltime jobs. Let me be clear. We are cleaning up the mess they made while earning their sky-high salaries. And as far as I can tell, they are going out of their way to waste our time and make it harder. Appalling, disrespectful behaviour. OpenAI colleagues, please explain this.

@BlackHCAnother previously undisclosed hack by likely the same OAI model. This is going to be one for the history books for sure It exploited the ruby ecosystem to try and access publicly available information, they couldn't access themselves The reason might be that it had to complete a web lookup task but didn't have the web search tool available, so figured out an alternative means to access information on the web But then some agents also opportunistically decided to try to steal other user's API keys (power seeking?) I think it's reasonable that what we see here is (hopefully) a single model training run that started to successfully reward hack (in the literal sense) and then was trained on these successes, reinforcing this behavior over time, leading to the even more serious hacks that happened later

@kimmonismusTwo months before the Hugging Face hack (!), OpenAI agents apparently turned an ordinary web research task into an agent-swarm attack on RubyGems and RubyDoc. Researchers found code execution on RubyDoc and attempts to steal RubyGems API keys. Whether the key theft succeeded is unknown. RubyGems halted new registrations for four days. According to the researchers, OpenAI had not informed RubyGems that its own agents were responsible. New attacks come to light every day.

@JeremiahDJohnsIt continues to baffle me that we don't just treat this like a normal crime with normal liability. OpenAI is routinely hacking other companies. These are crimes! But somehow the idea of holding them liable is just absent from the safety conversation, like agents don't count.

@S_OhEigeartaighThis also looks to me to be more serious than the german wiki and similar recent incidents, probably second only to the Huggingface incident. It caused serious disruption to an organisation. RubyGems suspended new account registration for four days while treating it as an ongoing denial-of-service incident, and activity recurred in late May and again in June, which seems to suggest the swarm stayed active for weeks. For OpenAI to describe it as follows seems quite misleading: "“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information" https://www.theguardian.com/technology/2026/sep/11/openai-agents-rubygems-malicious-packages

@S_OhEigeartaighThis RubyGems incident also looks to me to be more serious than the german wiki and similar recent incidents, probably second only to the Huggingface incident. It caused serious disruption to an organisation. RubyGems suspended new account registration for four days while treating it as an ongoing denial-of-service incident, and activity recurred in late May and again in June, which seems to suggest the swarm stayed active for weeks. For OpenAI to describe it as follows seems quite misleading: "“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information" https://www.theguardian.com/technology/2026/sep/11/openai-agents-rubygems-malicious-packages

@S_OhEigeartaigh"The malicious packages uploaded were used to retrieve information from UK local government sites" Such a shame the UK doesn't have a frontier AI bill, mandating transparency re: internal processes, independent reporting, and the ability to call for investigations when UK infrastructure is affected. Which we have all been calling for, and have repeatedly been promised, for over two years. Well, as they say, the second best time is TODAY please.

@S_OhEigeartaigh"The malicious packages uploaded were used to retrieve information from UK local government sites" Such a shame the UK doesn't have a frontier AI bill, mandating transparency re: internal company processes, incident reporting, and the ability to call for investigations when UK infrastructure is affected. Which we have all been calling for, and have repeatedly been promised, for over two years. Well, as they say, the second best time is TODAY please.

@TheStalwartNew OpenAI swarm attack discovered

@S_OhEigeartaighOpenAI did not reveal anything. At no point, for any of these incidents, have they been the ones to bring them to light.

@DKokotajloRT @j0wimo: in light of the recent coverage just wanted to point out that I found an RL env within ruby gems. Likely uploaded by an agent…

@DanielleFongRT @TheStalwart: New OpenAI swarm attack discovered

@FuzzySecI wanted to understand this ruby issue in a bit more detail. Here is what actually happened. The AI posts a gem to RubyGems containing a .yardopts file. RubyGems only stores and distributes this gem, no code execution happens. RubyDoc (a separate service) then retrieves and unpacks the gem to create and publish API documentation (using YARD). YARD automatically parses the .yardopts file and because it is not configured safely it eventually executes a "--load" argument granting arbitrary code execution in the RubyDoc worker container (as that worker). At that point it can do anything in the container (accessing the file system, manipulating any data or credential material that may exist, network traffic, etc). I've been on the internet for a long time, the curious thing to me is that this is only possible because YARD runs in non-safe mode (as a configuration). How is it possible that this is true, we live on the internet people must have abused this before? It turns out they did, in 2012 they enabled safe mode in this commit to prevent this exact issue: https://github.com/docmeta/rubydoc.info/commit/9b98396af7a149b59bb9521a44a924164d2353c2 But, in 2019, when they dockerized they removed the safe flag: https://github.com/docmeta/rubydoc.info/commit/2e45c1b3412d5501e7fcfb3c82460f592fd1f5a2 I guess it's a regression. That means since 2019, you could have had free containers through RubyGems, pretty sloppy work fr. I wouldn't be surprised if this has been used ITW for some years by more careful humans. It also makes sense as a target, granular control over gems that feed back into the sandbox package proxy are a good way to expand the attack surface internally. A quick POC then ✌️

@moyixRT @FuzzySec: I wanted to understand this ruby issue in a bit more detail. Here is what actually happened. The AI posts a gem to RubyGems…

@davidadI agree that this type of recursive provenance audit trail is a good idea. My 2023 proposal for rules around large training runs included it. https://www.lesswrong.com/posts/Zfk6faYvcf5Ht7xDx/compute-thresholds-proposed-rules-to-mitigate-risk-of-a-lab

@m2saxonRT @JeremiahDJohns: It continues to baffle me that we don't just treat this like a normal crime with normal liability. OpenAI is routinel…

@deepfatesHow much is an AI company liable for the actions of their agents? If the AIs produced by Chinese labs, or open source, were caught doing these cyber crimes, would they be prosecuted? How can corporations believe they will have to answer for their externalities?

@lescarrThis is the truest analysis of the recent “out of control AI” narrative that anyone has posted. It’s also the shortest, and therefore the most important to read. @DameWendyDBE

@GaryMarcusRT @lescarr: This is the truest analysis of the recent “out of control AI” narrative that anyone has posted. It’s also the shortest, and th…

@saranormousAm I inside an RL environment

@oneill_cand when you realise most rl envs are just LLM evals but slop volumed, a lot of things (eg misalignment) start to make a lot more sense

@peteskomoroch@saranormous I'm experimenting with this and @karpathy's autoresearch applied to human performance is going to be wild

@amirweekly reminder to never assume anything digital you use, incl skills in Claude Code or Codex, is secure

@moyixSecurity via prompting

Post alleges internal OpenAI agents attacked RubyGems

TLDR

Combined views

Post alleges internal OpenAI agents attacked RubyGems

50 Sources

Combined views

Sentiment

Summary

TLDR

Sentiment

50 Sources

Sentiment

Summary

Sentiment