The dispute over credit for neural-network distillation
Schmidhuber challenges The New York Times’ account of the technique’s history, alleging that Geoffrey Hinton’s team republished his 1991 work without attribution.
TLDR
In a September 26, 2026 post, Schmidhuber alleges that Hinton’s team republished his technique in the 2015 paper “Distilling the Knowledge in a Neural Network” without citing the original work. He criticizes New York Times reporter Cade Metz for crediting Hinton with helping develop distillation in the early 2010s. Schmidhuber points to his 1991 technical report and a related 1992 paper, describing the technique as compressing one neural network’s behavior into another. He also reiterates his July support for open-source models distilling commercial models, arguing that commercial companies had themselves distilled material from the internet for free.
Against his better judgment, @CadeMetz continues to claim in the NYT [1,2] that his friend @GeoffreyHinton helped develop distillation in the early 2010s [3], even though Hinton's team merely republished — without attribution — my 1991 distillation technique [4][5]. What a…
I support open-source models distilling what commercial companies distilled for free from the entire internet. I published distillation for free in 1991 in Europe - this was copied in the US and in China (https://x.com/SchmidhuberAI/status/1608870559609421831)…
