Goodfire AI researchers identify shape-rotating calculator in language models
Goodfire AI researchers identified an internal mechanism in large language models that performs addition and related operations by rotating shapes in high-dimensional neural space. The reusable structure interfaces with Fourier features and reads from geometric representations learned during training. It extends to other computational tasks and builds on a 2023 arXiv paper on modular addition circuits. Released visualizations display shifting activation clusters in scatter plots.
#314Yuntian Deng@YUNTIANDENGVery interesting finding! Reminds me of the structures Xiaoyan and Itamar found in multiplication reasoning:
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)
#671Joscha Bach@PLINZLLMs are wordcels AND shape rotators
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)
#980Lisan al Gaib@SCALING01BREAKING: LLMs are just stochastic parrots
They have no idea of the real world. How could they? They are just storing text in their database and regurgitating it !!1
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)
(if you fell for this you are regarded)
Lisan al Gaib@scaling01BREAKING: LLMs are just stochastic parrots They have no idea of the real world. How could they? They are just storing text in their database and regurgitating it !!1
#980Lisan al Gaib@SCALING01Lisan al Gaib@scaling01the stochastic parrot LLMs have obviously stolen this concept from mechanical calculators https://en.wikipedia.org/wiki/Mechanical_calculator
#980Lisan al Gaib@SCALING01the stochastic parrot LLMs have obviously stolen this concept from mechanical calculators
Danielle Fong 🔆@DanielleFong@scaling01 that's not true. as you can see. they rotate it
#1306Sonya Huang 🐥@SONYATWEETYBIRDso cool. win for the shape rotators! @ericho_goodfire
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)
#1332Prakash@8TEAPII figured we’d get here eventually.
It’s beautiful.
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)
#1626Tom McGrath@BANBURISMUS_the world inside neural networks is so beautiful. gradient descent learned to make a general-purpose addition module!
Goodfire@GoodfireAINeural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)