Goodfire researcher Nick Cammarata argues that AI models may be easier to understand by studying their own neurons and circuits than earlier interpretability research suggested. His essay, Neural Networks Are Already Clean, presents this as a working hypothesis, not a completed map of how large models work.
The argument challenges an explanation for neurons that appear to respond to unrelated things. Cammarata contends that some famous examples reflect shared structure and related concepts rather than arbitrary combinations. In the essay, he reinterprets a neuron associated with both cat faces and car fronts as detecting V-shaped features. He calls the broader process of related concepts growing out of one another recursive generalization.
Cammarata contrasts his current outlook with an earlier OpenAI project, which he says found about 2% of neurons legible using primitive tools. The essay describes preliminary audits of Qwen3-32B in which the team estimates that roughly 90% of neuron poles have one main job. He cautions that the percentages depend on thresholds for what counts as one thing, and says language models with trillions of parameters are not close to fully understood because of their size.
His proposed route is to scale up experiments on the models themselves. Automated agents can investigate neurons, examine their connections and test explanations, he writes. In his launch post, Cammarata expresses hope that improving automated scientists could make ambitious interpretability tractable in time to help with AI alignment.
The interpretation is contested. In a public response, Liv, posting as @livgorton, welcomes the prospect of easier-to-understand networks but argues that the essay mischaracterizes what superposition predicts. Cammarata's own essay labels its broad near-term outlook highly speculative.
Comments