Reactions from ranked influencers
8 posts
Distillation for me but not for thee seems like a shaky and inherently contradictory position?
if you’re a major ai lab publicly complaining that you’ve been distilled, that is not a good look. at all. normal ppl do not care that you feel wronged. they see a company claiming to build the most powerful systems on earth while asking everyone else to protect its homework. the security implications are different & deserve serious consideration, especially cuz model weights are opaque black boxes. but commercial grievance dressed up as principle is not going to persuade anyone.
At night I dream of a distillation debate grounded in public, technical info, not reading tea leaves of backroom deals and political intrigue. Then I wake up and I'm crushed by reality of chaos, potentially classified information, and a spiraling global AI ecosystem.
I've seen no greater irony in my entire life than Anthropic complaining about distillation. They trained on the entire corpus of human knowledge and are now selling it back to us, compressed. All while desperately trying to game regulators into providing them with a business model and claiming the moral high ground, haughtily and with great condescension
Where I think things are at: the public information MASSIVELY favors the state of affairs that distillation is fine, no action needed. Public research shows minimal effect. There have been some classified reports (e.g. data from NSA, other govt agencies), private briefings from labs, which show more influential usage & impact, but not a 100% crucial role in training. I.e. the Chinese are incredibly good at using the APIs to train models, better than anyone in the U.S. or public research. A lot of the "we must act now" rhetoric comes from an additional embellishment of the second point.
This makes me sad. It always makes me sad to see tech companies in litigation in general and in IP litigation specifically. This compounds when any company seeks to do so under the cover of using the whole of government, trade law, and claims of spying. Treating/calling model distillation as some form of industrial espionage is a wild expansion of IP law. If there is actual theft of trade secret then the company/victim should prosecute in the US. If companies build, test, expand their models using what is clearly and provably fair use web scraping, then we're saying competitors can't learn from model behavior. That's such an obvious ruse to gain regulatory capture. US winning in AI depends on innovation, nimbleness, and execution not on whining to the government to use the wrong tool in the wrong context. Now putting all this in context of tech companies, IP law, and why we are here. No AI was used to create this. Opinions and mistakes all personal and mine, as are — —. My first all-company meeting at Microsoft in 1989 including the umpteenth update on an original and endless "look and feel" lawsuit with Apple. No amount of tap dancing made it seem less than the petty litigation it was as both companies saw Xerox PARC's work. In the end this was a routine contract dispute and nothin more. This after cases of Lotus and Borland which Microsoft took the "let us innovate approach." Sadly after that case we saw a turning point where Microsoft decided to build an IP patent portfolio. This rubbed all of us—those that would be inventing stuff—but it was sold to us as "we will only use these to defend out IP never to litigate rather than compete." Call us cynical but few of us believed that to be the case. Linux sadly proved us right which was no consolation for all those patent cubes we earned to garner performance review comments. So did Android. So did a dozen "cross-licenses" which is what big companies do to avoid suing each other. The best products for the market still won, but IP generally just gummed up the system and consumed massive time and money (3000 patents a year and perhaps $200K per...) The web has been pretty consistent in courts that scraping information that is already available to the public is not a violation of any IP laws, extra especially if that information is itself not copyright. Whether LinkedIn profiles, catalogs, or Google search results the courts have shown deference to scraping as legal, transformative, and importantly not copyrightable by ToU/T&C or simply being packaged on a web site. Public is public. Facts are facts. Free is free. Automation and scale do not change that. But here we are the natural endpoint of a mature industry (tech is a mature industry now) where big companies are choosing first to go to the government to get help in competing. In this case competing with China. Make no mistake, my view is the AI companies should compete with each other and compete around the world on products, technology, GTM, and customers —not on suing each other over IP or getting the government to use the tools of trade and tariffs to compete. It turns out at this point in time there's alignment in Washington DC power centers that competing with China is the most critical of national issues. One side sees China as primarily as a military competitor (true) and the other sees AI in general as taking jobs that China already took with tech being a significant enemy. This alignment creates a unique moment where the powers of government can be used to compete. There are three primary tools governments use to compete economically: • Rely on the free market or offer financial support to the domestic market • Introduce trade barriers that disadvantage international competitors • Use the global banking and financial infrastructure to hinder international businesses Tech has gone through waves of governments attempting to juice the local market and essentially all of those have failed. Japan in the 80s was going to take over the US tech market with support from an internationally recognized model of industrial policy (MITI). China has tried to build a national OS, a national chip, and so on but the entrepreneurial market (within China's legal structure) created the big tech companies in that country. Trade barriers which often get positioned as sovereignty, security, or job security have been used for decades across industries. Because every country has something to counter anything one country erects as a barrier, these turn into endless rounds of ping-ponging reactions. We see this happening right now with Canada as if rising prices for homes weren't enough we now have basic lumber with a huge tariff over accusations of dumping. This label has been used on Japanese cars, steel, and DRAM in a past trade war with an Asia economic super power. It is far to look at how the domestic industries faired with this sort of government trade barrier support. Our domestic industries hid behind these barriers rather than innovate and compete in a global market. Detroit literally doesn't even make cars anymore, just trucks, so we do not even have fuel efficient domestic cars. IP is a part of trade barriers and sanctions. In the middle of my career I took a break and went to live in China during the height of "embrace the China market" and worked very hard to understand how to make IP work in China. Some of what I did was tour the giant pharmaceuticals that simply reverse engineered and copied the latest US drugs. I visited endless PC computer malls with their industrialized software piracy. The reality is the culture is different and they have a different view of how piracy and IP works. I was told more times than I could count that "knowledge should be shared and not protected in a home" which was supposedly something an ancient philosopher said. I was frustrated. Then we released a product with copy protection and the government promptly sanctioned us for violating their IP framework. Part of being a global company is respecting laws of sovereigns when doing business in the country even if it seems unfair, or not doing business there. The fact that AI companies are claiming distillation is somehow an IP violation is somewhat rich when you consider how they acquired their training data—not what was licensed (or settled) but what was scraped. The court cases making that legal apply to China too. I a not a lawyer and sure let's litigate distillation but just on the face of it, I cannot imagine the copyright claim. Are these companies claiming the output is copyright? Do they claim all the code generated to be copyright and held by the company or the prompter? Are they asserting they own my chat with Dr. GPT about my labs? If this is true then they should litigate a domestic case of code generation first since that has been at scale as well at least judging by revenue. In the current context banking/economic sanctions are clearly a mutual destruction scenario as our trade balance with China shows. These have at very best a mixed record but no doubt are a favorite policy tool of DC. The Ukraine war, North Korea, and Iran show the limits of attempting to isolate a sovereign from banking. Tech has historically worked very hard to avoid the fate of industries like autos and steel. Tech has preferred competition and importantly coopetition (Andy Grove's famous expression) over litigation and especially over asking the government to get involved. Because of coopetition, tech has always struggled to speak with one voice. Most industries compete on zero sum which is the root of many challenges. Tech generally competes on a growing pie. That's the root of coopetition—building reinforcing platforms where growth is good for everyone. This has reached a moment where part of tech is not actively engaged with a government aligned on China but for differing reasons. This is just like the 80s and Japan when the isolationists and union leaders aligned on using government tools to compete with Japan. Detroit which spoke with a unified voice welcomed this. Detroit that doesn't compete on cars anymore. Detroit that can't make EVs. The other part of tech is labeled as too lax or just favoring open source. A much better label is "competing in the market." Open source is one way to compete in the market. Distillation is what created google -- it distilled the World Wide Web. Scale automation and distillation is how compilers, WiFi, and every device driver ever got built and tested. Successful emulation of hardware, protocols and APIs all happens because of automated and scaled distillation of executing the target environment. This part of tech has always viewed this as market competition and not IP violations and lawsuits. </RANT>
Thanks @mkratsios47 for speaking out on this important issue. Illicit, adversarial distillation is IP theft and industrial espionage that supports adversary military and intelligence capabilities. It is a national challenge that creates serious national security risks for the United States and democratic allies. We'll continue to crack down on it and work with the Administration and Congress to maintain American AI leadership. 🇺🇸