Designing language models around their harnesses
A blog author explores designing models to fit their harnesses—the software that runs them—rather than designing harnesses around the models.
TLDR
The author describes the autoregressive, decoder-only design of language models as an “unnatural shape.” They argue that powerful models can be distilled into new model shapes and cite Jev as an example of a useful tradeoff. They see model shape as a research direction worth exploring, especially for harness design.
Designing language models around their harnesses
A blog author explores designing models to fit their harnesses—the software that runs them—rather than designing harnesses around the models.
