Anthropic CEO Dario Amodei argues open-source AI cannot be monitored, revoked, or patched for safety after release
Story Overview
A 2023 Senate testimony clip from Anthropic's Dario Amodei has resurfaced to sharpen the divide over open-source AI, stressing that once frontier models escape controlled environments the original developers lose any practical way to watch for abuse, pull them back, or push safety fixes.
Closed models keep the keys in one place
Amodei contrasted open releases with closed systems where companies can still monitor misuse and apply patches, arguing that only the latter approach scales safely to the most capable models.
Frontier distinction leaves smaller models untouched
The testimony notes open-source approaches have worked fine for modest models so far, but it leaves unresolved how regulators or labs should draw the line when capabilities cross into higher-risk territory.
Many users attacked Anthropic CEO Dario Amodei as a gatekeeper or regulator-capturing hypocrite for calling open source AI a dangerous distraction, while a few agreed his risk warnings were valid and needed more attention.
No Digg Deeper questions have been answered for this story yet.
Most Activity
For every software provider, your biggest competitor is the customer themselves -- building it for themselves and thereby reducing reliance on you. Instead consider making a net positive business that helps your customer, instead of trying to rent seek forever
🚨ANTHROPIC CEO: OPEN SOURCE AI IS GETTING DANGEROUS
Anthropic CEO Dario Amodei told lawmakers that open-source AI is moving down a “very dangerous path.”
His warns that once powerful models are released openly, companies lose the ability to monitor misuse, revoke access, or update safety guardrails.
"The scaling of open source models, I think it's going down a very dangerous path. And if the path continues, I think we could get to a very dangerous place. I"
~ Dario Amodei's 2023 U.S. Senate testimony
Anthropic CEO Dario Amodei on Open-Source AI Models.
"I don't think open source works the same way in AI that it has worked in other areas. Primarily because with open source you can see the source code of the model. Here we can't see inside the model, it's often called open weights instead of open source to kind of distinguish that. But a lot of the benefits, which is that many people can work on it and that it's kind of additive, don't quite work in the same way.
So I've actually always seen it as a red herring. When I see a new model come out I don't care whether it's open source or not. If we talk about Deep Seek I don't think it mattered that Deep Seek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That's the only thing that I care about.
It actually doesn't matter either way. Because ultimately you have to host it on the cloud. The people who host it on the cloud do inference. These are big models, they're hard to do inference on.
When I think about competition I think about which models are good at the tasks that we do. I think open source is actually a red herring.
It's not free. You have to run it on inference and someone has to make it fast on inference."
---
From 'Alex Kantrowitz' YT channel (full video link in comment)
@kimmonismus Open source must win. If Dario and others have their way, only small % of humanity will have access to top tier AI.
Major issue is hardware tho, we need way more competition in both processing and ram so one company/country cant just decide to monopolise it.
Dangerous to Dairo’s need to make profits.
BREAKING: Anthropic CEO says open-source AI is moving down a “very dangerous path.”
Give your customers the tools that they need to improve themselves and become capable, build a relationship with them. It doesn't scale and it isn't easy. But this is precisely why this is a better business model than attempt to regulate their ability to help themselves..
For every software provider, your biggest competitor is the customer themselves -- building it for themselves and thereby reducing reliance on you. Instead consider making a net positive business that helps your customer, instead of trying to rent seek forever
skill issue
BREAKING: Anthropic CEO says open-source AI is moving down a “very dangerous path.”
@kimmonismus Would be nice if he added: "and some of these models happen to be 10x to 50x cheaper than ours... And more often delivering very similar capabilities". But he is not concerned, of course.
As an open-source developer for over a decade myself, it is very clear that he doesn't understand the open-source spirit, which explains why he doesn't care. The main point of OSS is not about additive collaboration or that you can read the code, most projects are done by a single person or a tiny group, and most users never read a single line of code - it's about treating software as a commons, something everyone can use and build on, not with a guy in-between gating it based on vibes and doomstalk. The GPL/MIT-licenses were the driving factor behind OSS, not primarily collaboration or readable code.
@kimmonismus This scumbag is on a crusade to gatekeep AI for the worst people on earth.
Ai is a faustian bargain- highly effective, but Claude is trained to select its own models and suggest using less edficient solutions. Example: sometimes fine tuning a slim adaptor on an open sourced model is superior and costs 95% less. Even with guardrails, Claude will drag its feet and unilaterally select itself. Caveat emptor.
I very vocally disagree with what Dario's been doing and I'm all for open weights models. But in this specific instance, he has a point.
It is a problem that local inference and training is cost-prohibitive for most people, alongside the fact that you can't really look into the weights and know exactly what the model is trained to do or what emergent behaviors will surface under any given circumstances. That's a major security risk that most people underestimate because of the "open" label.
The correct answer is not government regulation nor model censorship, but these issues need to be taken and discussed seriously by the community.
@kimmonismus This guy is a super dangerous scammer, dont be distracted by his dorky demeanour
@kimmonismus What a total piece of shit. deepseek published their science and technology for the scientific community to process and build on. They publish their code and training settings so you learn how such a model is built, and most importantly, they share the model itself for free.
Here's my take, and I'd love to hear others comment....
If you're talking about a general purpose LLM, you must be careful with open source. Dario is right in that you won't know the training materials, the weights given to the connections, whether bad data was intentionally included or good data was excluded, etc. A nation state could release a polluted model that would generate results that may look good, but are not accurate. This would be bad. Think China releasing a model that won't discuss Tiananmen Square but would discuss every bad aspect of American culture.
But open source models for things like vision, speech, movement, etc - models that aren't exclusively LLMs and are designed for an AI niche, should be fine.
I think it is worth differentiating.
@kimmonismus Are people that dumb to not catch the BS? After a few seconds I closed the video, you can clearly see how terrified they are of open source models. I wish all of humanity could contribute to open source models so the hegemony of Anthropic and OAI comes to an end.
@kimmonismus "you still cant see what's happening inside the model"
And this guy was VP OF RESEARCH AT OPENAI??
ffs, I am done with tech, I am starting a goose farm
@kimmonismus he's talking about open weights models not open source https://opensource.org/ai
@kimmonismus Insane how @antirez with DS4 single-handedly made Dario's point age like milk xD
@kimmonismus Dario might have overtaken Sam for most tone deaf in the field of AI.
I'm also unclear which of them is more evil at this point
@kimmonismus yea what is he talkin bout