How training data from many sources may shape LLM internals
A post sharing new work from Simplex says the belief geometry over data from many sources forms “telescoping cones,” and that transformers represent those structures.
TLDR
LLM training data comes from many sources. An author sharing new work from Simplex says the belief geometry over that data forms “telescoping cones,” which transformers represent in their internal activations. The author also announced that the write-up had been published on LessWrong for discussion, calling it especially important for people interested in understanding LLM internals.
Combined views
—
1 Source, first seen ago
