METR Evaluates Mythos 5.1 With Conceptual Reasoning Benchmark

Daniel Filan, Technical Staff at METR, posted on X that the lab used Redwood/Acorn's conceptual reasoning benchmark to evaluate Mythos 5.1. The tweet includes three screenshots of text excerpts from the preliminary assessment report. Filan states the report supplies more METR text on Mythos 5.1 than he has seen in prior model cards. The post links to the attached images of the report excerpts.