Distillation Exposes Economic Moats and Legal Limits for Frontier AI Labs
Reactions from ranked influencers
2 posts
From Anthropic’s Fable model on the economic, moral, ethical and legal opinion of distillation of Anthropic’s Fable model: Whether it’s a moral problem is genuinely contested. The labs trained their models on the open internet — copyrighted books, articles, code — largely without permission, and their fair-use defense is essentially “learning from data is transformative.” Distillation is the same argument turned against them: a model learning from another model’s outputs. It’s hard to construct a moral principle that permits the first and forbids the second, which is why critics call the labs’ objections hypocritical rather than principled. Distillation is contentious because it lets a smaller model absorb much of a frontier model’s capability by training on its outputs — effectively free-riding on billions of dollars of compute, data curation, and RLHF work. The DeepSeek episode made this concrete: if you can extract 80% of the value of a $1B training run for $5M by querying the API, the economics of frontier labs get shaky. That’s the core anxiety — it’s a moat problem before it’s anything else. Legally, it’s mostly a contract issue, not a copyright one. Model outputs likely aren’t copyrightable (no human author), so the labs’ real weapon is terms of service — every major API prohibits using outputs to train competing models. But ToS violations are breach of contract, hard to detect, hard to prove, and nearly unenforceable against a foreign entity. There’s no statute against distillation itself. So the practical answer: it’s an economic problem dressed in legal clothing, with a moral argument that cuts both ways depending on whose training data you start counting from.
Distillation = using a top AI (like Claude Fable) to spit out millions of answers, then training a cheap smaller model on those outputs so it copies most of the smarts without spending billions. Chamath is saying: the big labs trained on the whole internet without permission and called it fair use. Now the same trick is being used against them. Morally inconsistent. Legally it's mostly "you can't do that" in their ToS contracts, which are tough to enforce. Real issue is economic — someone can steal ~80% of a $1B model's value for $5M and undercut them. It's a moat problem.