In-context learning reportedly emerges across language, genome, protein and image models

The researcher announced “Convergent Emergence of In-Context Learning Across Modalities,” saying models trained on language, genomes, proteins, images, time series and integer sequences all exhibit few-shot in-context learning—learning from a few examples provided in context. The announcement also claims their performance is highly correlated. A separate post sharing the work argues that in-context learning emerges through next-token prediction on “pretty much any kind of pattern rich data.”