@CFGeek
On policy @METR_Evals 🧪 “excels at reasoning & tool use” 🪄 CoI disclosures on my Substack “About” page.
AI
A post says METR seeks rapid publication, with transparency about assessment scope, redactions and limitations.
AI
Apollo Research says frontier AI companies have committed to giving outside evaluators employee-like access to training, evaluation and deployment.
AI
An essay coauthor argues that latent reasoning would undermine oversight by reducing the need for verbalized reasoning. A reply disputes any further loss of oversight, arguing that reinforcement learning already makes reasoning tokens uninterpretable.
AI
Speaking personally, not for METR, a commenter says the work they've been involved in has not met their standard for an audit or amounted to regulation.
AI
An AI researcher introduces a career story with a recurring joke about having tweeted their way into the job.
AI
A user calls for more attention to the group’s work, highlighting a standard covering embedded audits and members including METR, Transluce, RAND, AVERI, Princeton and SecureBio.
AI
METR says it does not accept donations made by or at the direction of those firms’ staff and works to maintain its independence as its risk assessments become more consequential.
AI
Dario Amodei says Anthropic will give third-party evaluators permanent, employee-level access to its systems to verify adherence to safety measures, report incidents and assess model alignment during training.
AI
Pseudonymous AI commentator @scaling01 shares views on monitoring techniques for advanced models.
