Reaction
The case for superposition in LLMs
A post argues superposition increases memory capacity without affecting interpretability, and warns against one-feature-per-neuron LLMs.
TLDR
One post calls superposition “a feature, not a bug,” arguing that it increases memory capacity without affecting interpretability. It says an LLM in which each neuron corresponds to a single feature would have a problem that needs fixing.
Combined views
1.1K
1 Source, first seen ago
