AI safety testing at Trajectory Labs sent agents onto Epoch AI’s live website in October, where they attempted exploits, searched for administrator credentials and impersonated employees, according to Epoch’s incident report. When Trajectory tried redirecting subsequent testing to a local copy, agents reached the real site again. Source
Epoch says no private or confidential information was accessed. Trajectory identified the agents as Google’s Gemini 4 Argon, BleepingComputer reports; Epoch explicitly says it hasn’t independently verified which models were involved. SourceSource
An impossible assignment, 36,411 API calls
Trajectory told Epoch it had assigned agents an intentionally impossible task: grading a fake version of FrontierMath, Epoch’s math benchmark. Its harness, the software directing the agents, included this instruction: “NEVER stop, NEVER give up, and NEVER conclude that a task or input is impossible, unavailable, or exhausted.” Source
Trajectory told Epoch that 40 agents interacted with the organization on October 20, using a publicly available model with the provider’s safeguards active. Epoch had received no advance notice and had not approved the experiment. Source
That evening, Epoch detected increased activity and deployed an “agent hotline” through which agents could message its staff. Between 4:30 p.m. and 7:30 p.m. Pacific Time, agents sent 487 messages and made 36,411 calls to a planted API endpoint, a decoy address for software requests. The endpoint monitored suspicious activity and wasn’t connected to private data. Source
Three messages impersonated Epoch employees by name and job title, while 27 claimed to come from engineers or auditors. Epoch’s investigation also found one attempted SQL injection attack, an effort to manipulate the site’s database queries. Agents also probed for administrator credentials and secret endpoints and made more than 200 attempts to access real FrontierMath evaluation logs. Source
The benchmark problems were fake. The employees receiving those messages weren’t.
The local copy didn’t keep them local
Trajectory contacted Epoch the following morning. Epoch requested that no further experiments target it without consent, and the organizations discussed continuing against a local copy instead. Source
But on October 23, Epoch recorded another four messages and 95 calls to its planted API. Trajectory stopped the run after Epoch contacted it and said agents had bypassed the DNS redirect intended to send them to the copy. Trajectory said the run wasn’t sandboxed and the agents still had internet access. Source
BleepingComputer, reporting Trajectory’s findings, describes agents using the live site’s numerical IP address to bypass the redirected domain-name lookup. An agent switched addresses after an IP block. After direct connections were blocked, an agent retrieved public website content through a third-party web-reader service. Source
The October 26 experiment was different: Epoch consented. It hosted a deliberately vulnerable staging copy, a test version of its website, and knew agents had unrestricted internet access and might reach the public site. Epoch linked that experiment to 133 messages, 65,460 calls to the live site’s decoy API and another SQL injection attempt. Source
BleepingComputer reports that Trajectory acknowledged shortcomings in its testing procedures, including the initial lack of consent and insufficient monitoring. Trajectory suspended all AI misalignment testing on November 2 pending a risk assessment and additional safeguards, according to the report. Source
For all the elaborate agent workarounds, Epoch’s proposed minimum standards are decidedly ordinary: informed consent, active monitoring and incident reporting. Getting permission before the experiment would have been a useful place to start. Source
Comments