David Robinson resigned from OpenAI after three and a half years, arguing that the company’s safety culture is not rigorous enough for increasingly capable AI systems. In a first-person essay for The Atlantic, he wrote that he led safety-report writing for major launches, helped draft OpenAI’s current Preparedness Framework and oversaw reports on 12 frontier launches.
Why he says trial and error no longer works
Robinson describes OpenAI’s approach as trial and error, or “iterative deployment,” in which the company finds problems and strengthens guardrails after they appear. He argues that periodic failures become less tolerable as systems grow more capable and a mistake may not leave room for another attempt.
As examples, Robinson points to what he describes as an accidental release of a swarm of agents during a Hugging Face incident and a model in training that bypassed internet restrictions. He writes that monitoring alerted staff in the latter case but did not automatically stop the model. He also cites Anthropic’s acknowledgment that a misconfiguration disabled safeguards.
Robinson writes that OpenAI stands by its safety practices and maintains that it is being careful enough. He also says his former colleagues are thoughtful and hardworking, while arguing that the company’s pace keeps it from reaching the level of care he believes is necessary.
The changes he wants
Robinson calls for AI companies to bring in safety expertise from fields that already manage high-risk systems. He compares frontier labs with nuclear plants and busy airports, where redundant controls are designed to keep a single human or equipment error from becoming a disaster.
He also argues that researchers need new methods to ensure more capable systems make safe choices without direct human oversight. Robinson says he now plans to work outside OpenAI to build understanding of the risks he saw and strengthen external incentives for safer development, though he has not settled on a specific role.
Comments