Report
METR’s updated AI tests find coding chops outpace research judgment
METR says none of the AI-designed evaluations in its expanded tests were useful for its research.
TLDR
METR’s updated AI progress chart still shows rapid gains on tasks with easily checked answers. The research-judgment results are less flattering: none of the AI-designed evaluations met METR’s standard for research use. Some newer models improved, but the report includes few judgment-heavy tasks, which are difficult to score. That leaves the prospects for automated AI research unresolved.
Combined views
—
First seen ago

Comments