The dispute over credit for neural-network distillation

In a September 26, 2026 post, Schmidhuber alleges that Hinton’s team republished his technique in the 2015 paper “Distilling the Knowledge in a Neural Network” without citing the original work. He criticizes New York Times reporter Cade Metz for crediting Hinton with helping develop distillation in the early 2010s. Schmidhuber points to his 1991 technical report and a related 1992 paper, describing the technique as compressing one neural network’s behavior into another. He also reiterates his July support for open-source models distilling commercial models, arguing that commercial companies had themselves distilled material from the internet for free.