OpenAI Engineer Confirms o3 Trained on ARC Public Environments
Researcher questions Anthropic benchmark practices while OpenAI engineer confirms o3 training method.
AI researchers discussed model training on public ARC benchmarks. Neil Chowdhury questioned whether Anthropic trained on public sets due to observed performance despite historical indifference to unrelated benchmarks like math. Will Depue, an OpenAI research engineer, confirmed that the o3 model used reinforcement learning on ARC-1 public demo environments, matching approaches noted for other models. This raises questions about training data sources and benchmark validity for frontier AI systems.
Combined views
676
2 posts, first seen 2h ago
OpenAI Engineer Confirms o3 Trained on ARC Public Environments
Researcher questions Anthropic benchmark practices while OpenAI engineer confirms o3 training method.
AI researchers discussed model training on public ARC benchmarks. Neil Chowdhury questioned whether Anthropic trained on public sets due to observed performance despite historical indifference to unrelated benchmarks like math. Will Depue, an OpenAI research engineer, confirmed that the o3 model used reinforcement learning on ARC-1 public demo environments, matching approaches noted for other models. This raises questions about training data sources and benchmark validity for frontier AI systems.
