Reaction
A circuit view of transformers and deep learning's supposed mystery
A user says “recursive generalization” is essentially what grokking produces and argues backdoors don’t emerge from training without adversarial data.
TLDR
A user argues that viewing transformers as circuits makes much of modern deep learning’s supposed mystery go away. They describe well-regularized machine learning as program search shaped by data and objectives, say “recursive generalization” is essentially what grokking produces, and contend that backdoors are plausible but don’t emerge from training without adversarial data.
Combined views
2.2K
6 Sources, first seen ago
