• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI
Report

TasteVal benchmark measures AI 'research taste' on eight development tasks

A post says TasteVal scores have doubled every 3.0 months since December 2025, following slower progress from 2023 through 2025.

1 Source, 1h ago, first seen 1h ago

TLDR

A post describes P-Zero Research's TasteVal as testing how well models design experiments and read results on eight real AI-development tasks, scored by the compute needed relative to human experts. It says scores have doubled every 3.0 months since December 2025 and puts Claude Opus 5.5 at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of experts' cost per attempt.

Combined views

—

1 Source, first seen 1h ago

— likes— comments— saves— reposts

Combined views

—

1 Source, first seen 1h ago

— likes— comments— saves— reposts
Image: @kimmonismus on X

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

Chubby♨️@kimmonismusP-Zero Research's new TasteVal benchmark measures research taste: how well a model designs experiments and reads their results on eight real AI-development tasks, scored by how much compute it needs compared with human experts (the dashed 1x line). Progress was slow from GPT-4 in 2023 through 2025, then the slope steepened: since December 2025, scores have doubled every 3.0 months. Claude Opus 5.5 now sits at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of the experts' cost per attempt. Research taste is the skill that decides how quickly AI could start improving AI, and this curve is one reason Jack Clark now expects that in 2027 (wrote about that yesterday as well).1h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    Chubby♨️@kimmonismusP-Zero Research's new TasteVal benchmark measures research taste: how well a model designs experiments and reads their results on eight real AI-development tasks, scored by how much compute it needs compared with human experts (the dashed 1x line). Progress was slow from GPT-4 in 2023 through 2025, then the slope steepened: since December 2025, scores have doubled every 3.0 months. Claude Opus 5.5 now sits at 2.30x the expert baseline, ahead of Fable 5.1 and GPT-6 Astra, at roughly 1/30 of the experts' cost per attempt. Research taste is the skill that decides how quickly AI could start improving AI, and this curve is one reason Jack Clark now expects that in 2027 (wrote about that yesterday as well).1h
    Today's Rank

    #12

    Today's Rank

    #12