The framework explains the growth of focused, monosemantic representations.
@aryaman2020 This seems really interesting in providing more of a framework for what many have observed about the growth of focused monosemantic (“localist”) representations in larger models, while there is still also widespread use of superimposed concepts!
neurons are all you need!
Users find the study on scaling laws for neurons in large AI models interesting because it offers a framework for observed growth of focused monosemantic features.
The framework explains the growth of focused, monosemantic representations.
@aryaman2020 This seems really interesting in providing more of a framework for what many have observed about the growth of focused monosemantic (“localist”) representations in larger models, while there is still also widespread use of superimposed concepts!
neurons are all you need!