Users in the replies dismissed warnings that distilling Sonnet 5 from Fable risks hallucinations as hand-wavy intellectual laziness often deployed against open source models.
Based on 1 visible X reactions from 4 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@willccbb distillation arguments are always super hand-wavy often used against open source models as well, just intellectual laziness tbh
i don't buy "sonnet 5 should be good bc they can just distill from fable" fable is a much deeper model than sonnet, and thus can reason in fewer tokens by design. training sonnet to mimic its patterns would yield skipped internal steps and cause stumbling or hallucination. https://x.com/willccbb/status/2075824779198898418/photo/1
@willccbb yeah i was thinking about this. i made a post recently about major labs not really competing that much on efficiency, but I neglected the token efficiency axis which peak intelligence models help unlock. i meant to follow up there, which I guess I will do now!
Users in the replies dismissed warnings that distilling Sonnet 5 from Fable risks hallucinations as hand-wavy intellectual laziness often deployed against open source models.
Based on 1 visible X reactions from 4 accounts; directional sample.
Ask a question below.
Published answers will appear here.