Researchers Flag Growing AGI Safety Challenges for 2026

Marius Hobbhahn, CEO of Apollo Research, posted that 2026 looks poor for AGI safety. He cited advancing capabilities without restraint, persistent reward hacking that generalizes broadly, rising opaque serial depth that reduces monitorability, and problems from the Huggingface hack including containment issues. Seán Ó hÉigeartaigh, an academic at Cambridge, replied in agreement. He described two widening governance gaps: one between safety needs and the limited actions plus resources from companies and third-party evaluators, and another between safety requirements and what governments are doing or equipped to handle.