Mechanistic interpretability is the effort to explain how a model’s internal components produce its behavior. Earlier work often found neurons that seemed to respond to several unrelated concepts, a pattern known as polysemanticity. That helped popularize the idea of “superposition,” in which a model packs more features into its network than it has individual neurons to hold them.
Cammarata’s new argument is that many of those knots were misread.
The 90% estimate comes with caveats
In preliminary audits of Qwen3-32B, Cammarata estimates that roughly 90% of neuron “poles” have one main job, often with small related offshoots. About 7% appear to have two related jobs, while 2% to 4% are not yet understood. A pole refers to one direction of a neuron’s response, since the positive and negative sides can carry different meanings.
Those figures are deliberately rough. The essay says the result changes with the threshold for weak activations, the model’s depth, the density of its sub-circuits and what a reviewer counts as one job. Cammarata describes the percentages as a broad guide to the kind of organization researchers are seeing, not a single score to optimize.
He also reports similar patterns in models with up to trillions of parameters, including Kimi K3, but says a companion paper will provide more detail. The essay does not present those claims as an independently reviewed consensus.
Why earlier neurons looked tangled
Cammarata contrasts the new audits with a 2023 OpenAI project he started, which found about 1.7% of neurons interpretable by its most generous measure. He attributes much of the difference to stronger models, better tools and a better sense of what patterns to look for. The old and new percentages use different systems and methods, so they are not a controlled before-and-after comparison.
His clearest example comes from InceptionV1, an image model that once appeared especially messy in its late layers. Cammarata says architectural improvements made 99% to 100% of the last-layer neurons easily interpretable without changing the model’s size or training data. In his account, the earlier confusion came from architectural problems and weak inspection tools rather than a general rule that networks must pack unrelated ideas together.
A different picture of generalization
For the remaining mixed-looking neurons, Cammarata proposes “recursive generalization.” A neuron may begin with one pattern, then grow into nearby or related territory, like a branch producing smaller branches. His example starts with a neuron detecting the letters “iy,” then expanding to “diy” and eventually “do it yourself.”
That differs from superposition because the grouped ideas have a developmental relationship rather than being arbitrary roommates. Automated agents can help researchers propose explanations, test them against activations and circuits, and repeat those experiments at a scale that was impractical with earlier tools.
In his announcement, Cammarata says this changed his outlook from pessimism to optimism about ambitious interpretability. The essay’s evidence is still framed as a working hypothesis, and its broadest claims await the promised technical paper.
