Amil Dravid finds that neuron universality and specialization scale predictably in models up to 30 billion parameters

There is still a lot to learn about how the microscopic organization of large AI models - at the level of individual neurons - changes quantitatively with scale and across diverse training settings. @_AmilDravid uncovers new features of universal neurons connecting directly to semantic structure and scaling. I'm excited for the directions emerging from Amil's work!