Two UC Berkeley researchers are pushing back on the notion that AI safety priorities and concerns over concentrated power must be opposing camps, instead calling for the field to center discussions on how these systems actually land in society and what governance should look like as a result.
The AI community seems to increasingly be heading towards a polarized world when discussing safety and consolidated power. I see this discourse as a false dichotomy, so @profjoeyg and I wrote an essay on how we need to change the conversation (link below).
Framing the issues as safety versus power consolidation creates an unhelpful split that limits constructive paths forward, according to the essay authors who see open science practices as one way to move beyond the standoff.
This essay is positioned as the first in an ongoing set of pieces that aim to reimagine how AI research, innovation, and regulation could work when the focus shifts to broader societal effects rather than the current polarized terms.
Positive users appreciate the essay's nuanced framing of the AI safety versus power debate as a false dichotomy because it encourages broader discourse, while negative users criticize it as wasteful or leading to undesirable outcomes.
A good essay by @pgasawa and @profjoeyg on a more nuanced view of AI advances.
The AI community seems to increasingly be heading towards a polarized world when discussing safety and consolidated power. I see this discourse as a false dichotomy, so @profjoeyg and I wrote an essay on how we need to change the conversation (link below).
Open-science is the only thing that really needs to prevail. Good post.
If only there was someone I knew building an institution like this.
The AI community seems to increasingly be heading towards a polarized world when discussing safety and consolidated power. I see this discourse as a false dichotomy, so @profjoeyg and I wrote an essay on how we need to change the conversation (link below).
We need more rising star researchers like Parth talking about how the technology we are building will impact society. I also happen to agree with him.
The AI community seems to increasingly be heading towards a polarized world when discussing safety and consolidated power. I see this discourse as a false dichotomy, so @profjoeyg and I wrote an essay on how we need to change the conversation (link below).
What is the future of AI research?
Maybe the highest leverage thing we can do isn't to train the next model or write another paper. It’s time to re-think the fundamental process by which science, innovation, and regulation happens, who participates, and what we want the future to look like. (1/2)
The AI community seems to increasingly be heading towards a polarized world when discussing safety and consolidated power. I see this discourse as a false dichotomy, so @profjoeyg and I wrote an essay on how we need to change the conversation (link below).
@pgasawa @profjoeyg interesting read. as unsolicited feedback i think you'll get more readers if you expanded the doc in a tweet thread itself
Link: https://pgasawa.bearblog.dev/unsafe-ai-or-consolidated-power-ais-false-dichotomy/
More smart people should be engaged in the larger picture discourse about the future of AI — reach out if you have thoughts.
If we move past the debate between safety and consolidation of power, open-weights vs. frontier labs, there’s an important opportunity to re-imagine what the future should look like and how we shape it.
My student and I wrote our first essay in a series examining a new path towards open science and we think many more people need to join the discourse. (2/2)
What is the future of AI research?
Maybe the highest leverage thing we can do isn't to train the next model or write another paper. It’s time to re-think the fundamental process by which science, innovation, and regulation happens, who participates, and what we want the future to look like. (1/2)
@pgasawa @profjoeyg We need all hands on deck to change this narrative.
Identifies the problem perfectly then speedruns to the worst possible outcome with "The key differentiator here, is that if these institutions are backstopped by the government, industry partners, and philanthropists, in perpetuity, we could avoid a similar failure mode to what we see today." All these proposals have the same thing in common; 'you shouldn't control it, but I should'. A far better solution is decentralized, broad based control, as a result of decentralized, broad based creation of the models. Which I agree will result in new institutions and power structures. They won't look like the old ones though. https://arxiv.org/abs/2412.07890
@natolambert 👀
I'm curious if people think open-weight models can really make a difference in the consolidation of power? Do open-weight models need to achieve SoTA to matter?
One of our concerns is that SoTA open-weight models will present a significant safety concern and will be difficult to regulate effectively.
@JeffDean @pgasawa @profjoeyg Then give up on the idea of democracy
@JeffDean @pgasawa @profjoeyg Nuance on AI. Risky business on here.
@JeffDean @pgasawa @profjoeyg the third option is always a paper until someone writes the CLAUDE.md for it.
@JeffDean @pgasawa @profjoeyg the essay probably says 'demos aren't production' somewhere.
that insight is free, though. the expensive part is spending months on edge cases nobody told you existed.
@JeffDean @pgasawa @profjoeyg The piece does a solid job cutting through the binary thinking—most people miss how incremental alignment research actually drives the biggest leaps.
@pgasawa @profjoeyg thank you for writing this
feel this is an imp framing for moving beyond the false choice between unsafe openness + control
@natolambert Yes.
@pgasawa @profjoeyg The essay raises interesting points, and is technically sound. But there's some more non-technical/societal claims/assumptions I don't think are right. Some thoughts
@AlexanderLong @jbrukh @natolambert cc @swiftevo1
