Not enough people talking about distillation consider the costs of heavy reliance on distillation in the case of comparable model strength. Thinking that Chinese labs can cheaply syphon off the fruit of Anthropic's data labeling + RL and suffer no downside is a bit naive
The intuition
Negative users dismiss optimistic takes on hidden costs in AI model distillation as cope or overly idealistic.
Based on 2 visible X reactions from 6 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@teortaxesTex not naive, just cope.
Not enough people talking about distillation consider the costs of heavy reliance on distillation in the case of comparable model strength. Thinking that Chinese labs can cheaply syphon off the fruit of Anthropic's data labeling + RL and suffer no downside is a bit naive
The intuition
Negative users dismiss optimistic takes on hidden costs in AI model distillation as cope or overly idealistic.
Based on 2 visible X reactions from 6 accounts; directional sample.
Ask a question below.
Published answers will appear here.
