Source Posts on X
Every single credible researcher I've talked to these past few weeks has said the "distillation" from Chinese labs is way overexaggerated. Have you tried distilling without logits yourself? Show me the results. Narrative violation: maybe the Chinese are actually good.
@Suhail distilling without logits works perfectly fine. it’s marginally less efficient than with. lots of papers on this!
Posts from X
2 postsEvery single credible researcher I've talked to these past few weeks has said the "distillation" from Chinese labs is way overexaggerated. Have you tried distilling without logits yourself? Show me the results. Narrative violation: maybe the Chinese are actually good.
@Suhail distilling without logits works perfectly fine. it’s marginally less efficient than with. lots of papers on this!