Microsoft releases BenchPress, predicting full LLM evaluation scores within 3.93 points using five probe benchmarks

Original post

Dimitris Papailiopoulos@DimitrisPapail#217inTech

BenchPress is here! A way to predict benchmarks without running them.

Basically can run 5 "principal" benchmarks and estimate the rest within <3.9%. Kinda nuts it works so well.

Evals are rank 2 lol

Yuchen Zeng@yzeng58

💻Tired of running so many slow, expensive benchmark evals across every checkpoint?

Try ✨BenchPress✨ at https://microsoft.github.io/benchpress/: provide a few benchmark scores, then get predictions for the remaining ~100 benchmarks, with trust probabilities and calibrated 90% prediction intervals.

How does this work? In his original post (https://x.com/DimitrisPapail/status/2026531440414925307), @DimitrisPapail first tried the idea as a fun question: collect model-by-benchmark scores into a matrix, find its low-rank structure, and use matrix completion to predict missing benchmark scores from a few observed ones.

We expanded this into a full system: a fully audited 84-model x 133-benchmark score matrix, an optimized matrix-completion predictor, and a reliability layer for trust probabilities and 90% prediction intervals.

Beyond predicting missing scores, we also suggest practical seed benchmark sets. The five-probe set {GPQA-D, HLE, Codeforces, MMLU-Pro, ARC-AGI-1} recovers the rest of a model's public score profile with a MedAE of 3.93 points. A lower-cost set {GPQA-D, MMLU-Pro, Aider Polyglot, MATH-500, AIME 2026} reaches 4.55 points.

See more details below 🧵1/7

This work is with @DimitrisPapail at AI Frontiers, a boutique research lab inside @MSFTResearch.

2:11 PM · Jun 24, 2026 · 26.1K Views

BenchPress · Predict LLM Scores

MICROSOFT.GITHUB.IOVia

VIEWS34.1KBOOKMARKS193LIKES279RETWEETS24REPLIES6

Dimitris Papailiopoulos@DimitrisPapail

The path from VibeBlogging to Science is a couple of months long :D https://microsoft.github.io/benchpress/add-model/

Dimitris Papailiopoulos@DimitrisPapail

http://x.com/i/article/2026523085545857024

1d34.1K279193

Dimitris Papailiopoulos@DimitrisPapail

BenchPress: PREDICT MODEL BENCHMARKS for free :) https://microsoft.github.io/benchpress/add-model/

Dimitris Papailiopoulos@DimitrisPapail

BenchPress is here! A way to predict benchmarks without running them.

Basically can run 5 "principal" benchmarks and estimate the rest within <3.9%. Kinda nuts it works so well.

Evals are rank 2 lol

1d5.4K6223

Dimitris Papailiopoulos@DimitrisPapail

Who si this for? Anyone who doesn't have infinite $ to spend on running 40 evals. Running 5 is enough.

We open source BenchPress as a tool anyone can use for free to predict your own model's scores https://microsoft.github.io/benchpress/

Dimitris Papailiopoulos@DimitrisPapail

BenchPress is here! A way to predict benchmarks without running them.

Basically can run 5 "principal" benchmarks and estimate the rest within <3.9%. Kinda nuts it works so well.

Evals are rank 2 lol

1d5.1K4314

Lisan al Gaib@scaling01

5 "principal" benchmarks to predict most other benchmarks you say?

you will never guess how many different subsections IQ test have

(it's also 5 for WAIS-V and SB-5)

fluid reasoning, knowledge, quantitative reasoning, visual-spatial processing, and working memory

Dimitris Papailiopoulos@DimitrisPapail

BenchPress is here! A way to predict benchmarks without running them.

Basically can run 5 "principal" benchmarks and estimate the rest within <3.9%. Kinda nuts it works so well.

Evals are rank 2 lol

1d7.8K5311

Yuchen Zeng@yzeng58

💻Tired of running so many slow, expensive benchmark evals across every checkpoint?

See more details below 🧵1/7

This work is with @DimitrisPapail at AI Frontiers, a boutique research lab inside @MSFTResearch.

Dimitris Papailiopoulos@DimitrisPapail

http://x.com/i/article/2026523085545857024

1d22.9K11280

Yuchen Zeng@yzeng58

🧵7/7

Paper (https://arxiv.org/pdf/2606.24020), code (https://github.com/microsoft/benchpress), and dataset (https://huggingface.co/datasets/microsoft/benchpress-score-matrix) are fully open. BenchPress is meant to grow with the model ecosystem. Pull requests that add new citation-backed scores or correct existing entries are very welcome. Have more scores? Please contribute them here: https://microsoft.github.io/benchpress/report-scores

1d19432

Yuchen Zeng@yzeng58

🧵2/7

The key reason we can predict benchmark scores is a striking empirical fact: the model-benchmark score matrix has strong rank-2 structure.

We first collect a public score matrix with 84 models and 133 benchmarks, with 2,604 observed cells and a 23.3% fill rate (screenshot 1). We then show its rank-2 structure in two ways: (i) rank-2 matrix completion minimizes held-out prediction error (screenshot 2), and (ii) across different complete submatrices, the top two factors explain almost all score variation (screenshot 3).

Roughly, the first component mostly tracks whether a model scores high or low overall; the second captures a smaller residual pattern across benchmarks.

1d31261

Yuchen Zeng@yzeng58

🧵3/7

Based on this rank-2 structure, we can use matrix completion method to predict the missing benchmark scores of a model. We compared 7 feature transforms and 12 prediction methods. BenchPress adopted the best combination among them: logit transform + bias-decomposed ALS. - Logit: maps percentage scores to an unbounded space, making values near 0 and 100 easier to model. - Bias ALS: separates global, model, and benchmark offsets, then fits a rank-2 interaction for the remaining structure.

1d20821

Yuchen Zeng@yzeng58

🧵4/7

Here are a few examples of BenchPress predictions on both open- and closed-source models, across frontier and smaller releases. As more benchmark scores are revealed for each model, the prediction error drops quickly, often to below 2 points.

1d16021

Yuchen Zeng@yzeng58

🧵5/7

And we can also use BenchPress to choose compact probe sets: small groups of benchmarks that are especially informative for predicting the rest of a model’s score profile.

The figure shows two versions: the most predictive probes overall, and a low-cost probe set. With only 5 low-cost benchmarks, BenchPress reaches 4.55 points median absolute error when predicting the remaining untested benchmarks.

1d14621

Yuchen Zeng@yzeng58

🧵6/7

In addition, we also show that:

- Ranking: A small number of probe benchmarks is enough to preserve model rankings. - Forecasting: Newly released models can be predicted using only earlier releases as training data. - Reliability: Prediction accuracy is driven by identifiable factors, which we use to provide calibrated confidence intervals.

1d20411