Could Chinese researchers distill OpenAI models more sample-efficiently than OpenAI?
A reply disputes the hypothesis, arguing OpenAI and Anthropic have more information through access to logits.
TLDR
One poster hypothesizes that Chinese researchers may have covertly developed methods for distilling OpenAI models more sample-efficiently than OpenAI itself. A reply calls that impossible, arguing that OpenAI and Anthropic have more information through logits, the raw scores behind model outputs. The first poster says they mean end-to-end efficiency rather than gradient per token, and speculates that better ways to mix in trajectories from a stronger teacher model could help.
Combined views
836
6 Sources, first seen ago