The resulting hybrid models maintain performance with minimal degradation
What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices.
See our experiments here: Explaining Attention with Program Synthesis [https://arxiv.org/abs/2606.19317]
No Digg Deeper questions have been answered for this story yet.
👉 New preprint! Automated interpretability by approximating / replacing NN components (here attention heads) with programs.
What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices.
See our experiments here: Explaining Attention with Program Synthesis [https://arxiv.org/abs/2606.19317]
Very cool work ⬇️
What if attention were code? We show that many attention heads in transformer LMs can be replaced by human-readable Python programs. Swap them in and the model barely notices.
See our experiments here: Explaining Attention with Program Synthesis [https://arxiv.org/abs/2606.19317]