OpenAI introduced MedChemBench on November 20, a 99-problem benchmark testing AI reasoning in medicinal chemistry. The company reports that GPT-6.2 Astra earned 84.5% of the available credit at a reasoning-effort setting labeled xhigh, with tasks covering molecular structures, experimental evidence and compound choices. Source
Thatβs a test score, not a drug-discovery success rate. OpenAI explicitly cautions: βThese results do not establish whether models can discover new drugs or improve laboratory decisions.β Source
Whatβs on the chemistry exam?
According to OpenAI, MedChemBench draws on 117 papers and patents and includes 761 distinct molecular structures. Most problems give models a snapshot of a research program: its biological target, researchersβ goals, compounds studied so far and selected experimental findings. The benchmark grades molecular reconstruction, scientific reasoning, compound ranking and selection, and property predictions. Source
At the same xhigh setting, OpenAI reports that GPT-6 Astra earned 83.3% of available credit, GPT-6.1 Sol earned 82.0%, and GPT-5.6 Sol earned 58.2%. Those are credit totals, not percentages of questions answered entirely correctly. Predictions can receive partial credit for getting close to measured values, and measurements already included in a prompt must also be reproduced correctly. Source
Reading the molecule is only the start
OpenAI says GPT-6 Astraβnot the newer GPT-6.2 Astraβearned 95.4% of available molecular-structure credit at xhigh, but only about half the credit for selecting compounds and predicting numerical properties. The company cautions that those categories use different tasks and scoring rules, so they arenβt directly interchangeable measures. Source
A compound can act strongly against a disease-related protein and still come with inconvenient small print: poor absorption or rapid breakdown in the body. Getting the molecular structure right matters, but it doesnβt settle which compound is worth pursuing. Source
One benchmark problem draws on research into inhibitors of PRMT5, a protein studied in cancer drug discovery. It asks models to predict cellular activity, cell-barrier crossing and breakdown in a liver-enzyme assay. OpenAI says later experimental results, withheld from the modelβs prompt, showed that compound 23 blocked PRMT5 in cells at a lower concentration, while compound 24 crossed the barrier more readily and broke down more slowly in the assay. A stronger result in one column didnβt deliver a clean sweep. Source
OpenAI describes MedChemBench as a test of reasoning from historical evidence. Each response was a single attempt without tools, feedback or new experiments. The company says prior model exposure to the published sources cannot be ruled out.
Comments