Anthropic reports Alibaba used 25,000 fraudulent accounts in the largest known model distillation attack against Claude

Situation detected: AI labs launch a 1 quadrillion token distillation attack on everything humanity has ever recorded

otherwise it’s just sparkling data theft

It seems to me that if one supports training AI models on copyrighted books and so on, that it's consistent to also support training AI models off of each other.

Is there some way to logically support the former and not the latter?

Anthorpic is built on stealing internet data, open source projects, using their customer data and prompts for training

They stole every major author's work and later when a massive lawsuit was filed against them, they agreed to pay $1.5b in AI copyright settlement.

..and they never compensated any of the open source project who spent decades building the tech to train their model.

Now, Dario is crying.

That's what he does all the time, just act like a cry baby.

@ramez Yes. Anthropic thinks it's fair use to train off copyrighted material, but they write contract terms in the TOS that forbid what alibaba is (allegedly) doing. So copyright vs contract law.

I don't really have a strong take here, but it seems that people generally accept that certain types of interactions with IP fall below a threshold of infringement, and others above it. Seems not unreasonable for Anthropic to consider pretraining on text to be below that threshold, and post-training against their model to steal its specific reasoning innovations to be above it. Seems also not unreasonable for others to disagree.

None of that has to do with whether Congress is interested in letting China train on our models, though. Lots of stuff could be fine for American companies but not acceptable for Chinese companies, when viewed through the lens of geopolitical competition.

@ramez (tbc I'm not saying I agree with that view, but I think it's logically coherent.)

@EntropyChase @ramez I mean @ramez is correct: there is zero legal grounding for synth data protection even more so internationally, only contractual. Labs will have to lobby for this.

@elder_plinius 🥸😂💪❤️💯🫡

@EntropyChase @Dorialexander That legal assessment is about violations to copyright. But copyright isn't being violated here at all. AI generated content, by law, cannot by copyrighted. The outputs of AI models are in the public domain.

@elder_plinius probably didn’t see, but i asked if you were aware of this particular thing doing your own tests https://x.com/nudevise/status/2051717805339218408?s=46 i assumed yea cuz its not hard but interesting nonetheless

@ramez I might disagree here. The argument re training on books etc is that the output, the model, is supposedly quite different from the inputs ("transformative use."). Ofc, it's debatable (&, imo, often wrong). But harder to argue transformative use when th input is another AI model

@EntropyChase @ramez I don’t see the difference: both are transformative and both are a transfer of value/reasoning ability.

@Dorialexander @ramez looks like you're right on that! appreciate the correction. I guess I disagree w that precedent. I think it could get overturned bc it's inconsistent w the spirit of commercial use and the idea of business inputs.

@EntropyChase @Dorialexander Fwiw, I had to go and double check to be sure!

Even if outputs were covered by copyright, though, I think it wouldn't be a copyright violation to train on them, as long as the resulting model produces non-verbatim outputs.

@elder_plinius

@elder_plinius Distilled Claude

@ramez @Dorialexander If the content came from other companies commercial models, those commercial models are a business input, I would think! as the content couldnt have been obtained without them.

@Dorialexander @ramez see the line about use that "usurps the market or potential market of the original work"

@Dorialexander @ramez Using an AI model to make a better AI model is not obviously transformative though. So that kind of use more likely would require a commercial license to use that legally. [https://www.law.cornell.edu/wex/fair_use]