Similar Architectures Boost Distillation Over Distant Ones

@aliteracy posted that distilling a model into an architecture very close to the teacher performs much better than one very different from it. The tweet mentions learning how to let machines learn and tags @RhodaAI and @Stanford. It presents the observation as a finding from their work. The post supplies no additional data, experiments, or results. Among visible replies on X, no independent confirmation or details appear in the packet.