Joke
A biblical joke about neural network features and residual dimensions
A user imagines the Lord confusing neural network activations after they embedded more features than residual dimensions.
TLDR
A user calls neural networks “unclean, for they have sinned,” then extends the biblical joke to Adam’s neural networks. In the tale, embedding more features than residual dimensions prompts the Lord to confuse their activations and scatter the features across the residual stream.
Combined views
2.1K
1 Source, first seen ago