OpenAI chief scientist Jakub Pachocki is calling for AI developers to agree on measurable development limits they would not cross until they can better understand and monitor their models. In a November 22, 2026 essay, “Perplexity and Intelligence,” he published internal pretraining measurements as a starting point for that discussion. Source
His proposed industry standard would specify an initial limit and “what concretely we’d need to achieve before going beyond it.” The limits would draw on perplexity—a measure of how well models predict data—and other measurements. As one possible milestone for proceeding, he suggests developing a successor to chain-of-thought monitors, tools for monitoring models’ reasoning. Source
The essay does not announce a pause, name participating labs or set out an enforcement mechanism or implementation timetable. Asking everyone to stop at the same line is not the same as getting everyone to agree where it goes. Source
Why use a prediction score?
Pachocki’s argument centers on pretraining, the broad initial training stage before techniques such as targeted finetuning and reinforcement learning. Perplexity measures how well a model predicts the next token on average; lower values mean better predictions. OpenAI generally tracks its logarithm, called log-perplexity or cross-entropy loss. Source
Pachocki argues the score is worth watching because improvements in pretraining prediction have coincided with unexpected new capabilities. The essay reports approximately 0.99 log-perplexity for GPT-3.5 and 0.91 for GPT-4; at losses below 0.8, he says OpenAI saw transformative capabilities in cybersecurity and mathematics. He cautions that it was not clear in advance that those abilities would appear at those particular scores. Source
These measurements use a fixed snapshot of evaluation data sourced from OpenAI’s internal codebase. They are not scores from an established industry-wide comparison. Source
Those decimals look tidy. Comparing them across companies is much messier. Pachocki acknowledges that developers use different evaluation datasets and train on different data mixtures; finding transferable comparisons remains part of the work he is proposing. Source
Perplexity also misses capability gains outside pretraining. The essay says chain-of-thought reasoning can substantially improve capabilities without reducing a model’s perplexity. It also acknowledges that the metric does not directly correspond to a task people care about. A lower loss number is not a safety certificate. Source
Why he wants limits now
Pachocki warns that AI participation in pretraining research could produce a sharp acceleration as early as 2027. He considers an “avalanche of fundamental pretraining advancements” conceivable, while expecting improvements in alignment and monitoring to fall short of keeping pace with the full potential acceleration. Source
OpenAI attributes most of the pretraining acceleration it reports for 2026 to long-term research and computing investments, with automation contributing a smaller share. Source
The essay does not specify a first numerical development limit or settle the monitoring milestone labs would need to reach before crossing it. Source
Comments