Can mechanistic interpretability show how neural networks could work better?

A researcher argues that mechanistic interpretability should study how neural networks could improve, not only how to read or steer them. They cite their 2025 paper’s reported differences between representations in small networks producing Picbreeder images and ask whether similar problems occur in large networks and language models. They suggest a circuit linking van and coyote detectors might illustrate entanglement, while cautioning that anecdotal examples do not confirm the hypothesis.