Anirudh Goyal Introduces AI Research Preference Models
The models rank candidate experiments using prior results to decide which ones to run next.
Anirudh Goyal, Senior Staff Research Scientist at Google DeepMind, posted about AI Research Preference Models. The work targets the gap between rapid idea generation by research agents and the high cost of running experiments on GPUs. RPMs reframe selection as a ranking task over candidate solutions and research history rather than direct score prediction. An inference-only version uses a frozen LLM to rank plans and code. Tests across 20 ML research tasks with a Qwen3.6-27B backbone produced higher average normalized scores than a no-RPM baseline. An agentic RPM variant performed best among the tested approaches.
Combined views
10.5K
8 posts, first seen 1d ago
Reactions
8 postsAG 
Anirudh Goyal@anirudhg9119AI research agents have a new bottleneck: they can invent experiments much faster than they can afford to run them. Our new work asks: can an agent decide which ideas are worth the GPU time before executing them? Introducing AI Research Preference Models (RPMs) 🧵
- likes: 100
- replies: 7
- bookmarks: 67
- reposts: 6
AG Anirudh Goyal@anirudhg9119Instead of predicting the exact score an experiment will achieve (a hard forecasting problem), we turn this into a preference problem: Given several candidate solutions + the research history so far, rank which candidate is most worth executing next.
- likes: 3
- replies: 1
- bookmarks: 1
- reposts: 0
AG Anirudh Goyal@anirudhg9119We study two kinds of RPMs. Inference-only RPM: A frozen LLM inspects candidate plans, code, and previously executed solutions, then ranks the candidates. No candidate needs to be executed first, so selection is cheap.
- likes: 1
- replies: 1
- bookmarks: 1
- reposts: 0
AG Anirudh Goyal@anirudhg9119We evaluate on 20 ML research tasks, with 24 hours on a single H200 per task and 10 seeds. Importantly, candidate generation and selection use the same Qwen3.6-27B backbone so the gains come from how compute is allocated, not a stronger model.
- likes: 1
- replies: 1
- bookmarks: 1
- reposts: 0
AG Anirudh Goyal@anirudhg9119The result: No RPM: 0.684 average normalized score Inference-only RPM: 0.711 Agentic RPM: 0.729 The validation-oracle ceiling is 0.748. So better research preferences recover a substantial fraction of the gap to an expensive oracle.
- likes: 1
- replies: 1
- bookmarks: 1
- reposts: 0
AG Anirudh Goyal@anirudhg9119The broader lesson: For AI research agents, test-time compute shouldn’t only generate more ideas. It can also decide which ideas deserve real experiments. In expensive-search domains, intelligence is partly knowing what not to run. Paper:
- likes: 4
- replies: 0
- bookmarks: 3
- reposts: 0
