/Tech2h ago

Google DeepMind, Schmidt Sciences, and ARIA launch a $10 million funding call for multi-agent AI safety

Story Overview

A new technical research program backed by Google DeepMind, Schmidt Sciences, ARIA, the Cooperative AI Foundation, and Google.org will distribute up to $10 million to explore safety questions that arise only when many AI agents interact at once, deliberately steering attention toward coordination and collective dynamics instead of isolated alignment problems.

743784610129.2K
Michiel Bakker
Seth Lazar
Nenad Tomasev
Séb Krier#519
Original post
Séb Krier@sebkrier#519inTech

Over the past few months I've been working on a very exciting project: a new $10m fund for research on multi-agent multi-principal AGI safety! Instead of focusing on single agent alignment and centralized control, we're looking to support research focusing on multi-agent settings, mechanism design, cooperative AI, and coordination problems.

This is a joint initiative between @GoogleDeepMind, @Googleorg, @schmidtsciences, @coop_ai, and @ARIA_research. Huge thanks to @James_D_Fox, @weballergy, @FranklinMatija, @lrhammond, and @ObadiaAlex for their invaluable work!

See: https://deepmind.google/blog/investing-in-multi-agent-ai-safety-research/

Apply: https://schmidtsciences.smapply.io/prog/scaling_ai_safety_for_a_multi_agent_world/

4:18 AM · Jun 11, 2026 · 10.5K Views
Open Question

Emergent behaviors remain largely unmapped

The call highlights collective patterns that could appear when large groups of agents operate together, yet provides no early benchmarks or predicted outcomes, leaving the actual shape of those patterns as an explicit open variable.

Global Call

Worldwide researchers can submit proposals

Eligibility extends to any researcher globally with no listed regional limits or required institutional partners, though exact deadlines, award sizes, and review criteria sit outside the current announcement.

Sentiment

Users are excited about Google DeepMind's new $10M fund for multi-agent AGI safety research because they see it as a crucial early step to understand emergent behaviors and ensure safe outcomes.

Pos
96.7%
Neg
3.3%
29 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS608BOOKMARKS1LIKES8REPLIES3
Séb Krier@sebkrier

damn I can't even screenshot properly. how do i print pdf

Séb Krier@sebkrier

Over the past few months I've been working on a very exciting project: a new $10m fund for research on multi-agent multi-principal AGI safety! Instead of focusing on single agent alignment and centralized control, we're looking to support research focusing on multi-agent settings, mechanism design, cooperative AI, and coordination problems.

This is a joint initiative between @GoogleDeepMind, @Googleorg, @schmidtsciences, @coop_ai, and @ARIA_research. Huge thanks to @James_D_Fox, @weballergy, @FranklinMatija, @lrhammond, and @ObadiaAlex for their invaluable work!

See: https://deepmind.google/blog/investing-in-multi-agent-ai-safety-research/

Apply: https://schmidtsciences.smapply.io/prog/scaling_ai_safety_for_a_multi_agent_world/

1hViews 608Likes 8Bookmarks 1
RETWEETS10
Google DeepMind@GoogleDeepMind

When millions of AI agents interact with each other, new collective behaviors can emerge. 🌐

Together with @schmidtsciences, @coop_ai, @ARIA_research and supported by @GoogleOrg, we’re launching a $10M research fund to help understand how AI systems behave as a group. → https://goo.gle/3Si6rCl

2hViews 18.5KLikes 209Bookmarks 61
Michiel Bakker@bakkermichiel

Congrats @sebkrier @weballergy @FranklinMatija @lrhammond!

Oversight systems that help comps and govs oversee millions of AI agents from different labs are going to be hugely important and won't (and shouldn't be) be build by any of the big labs.

Séb Krier@sebkrier

Over the past few months I've been working on a very exciting project: a new $10m fund for research on multi-agent multi-principal AGI safety! Instead of focusing on single agent alignment and centralized control, we're looking to support research focusing on multi-agent settings, mechanism design, cooperative AI, and coordination problems.

This is a joint initiative between @GoogleDeepMind, @Googleorg, @schmidtsciences, @coop_ai, and @ARIA_research. Huge thanks to @James_D_Fox, @weballergy, @FranklinMatija, @lrhammond, and @ObadiaAlex for their invaluable work!

See: https://deepmind.google/blog/investing-in-multi-agent-ai-safety-research/

Apply: https://schmidtsciences.smapply.io/prog/scaling_ai_safety_for_a_multi_agent_world/

44mViews 297Likes 4Bookmarks 1
Nenad Tomasev@weballergy

I am excited to announce this funding call, in collaboration with @schmidtsciences, @coop_ai @ARIA_research and @Googleorg

As we are thinking about moving beyond individual powerful AI agents towards large-scale agentic collectives that can communicate and coordinate towards completing long-horizon complex tasks - it is paramount to design these future agentic societies safely.

Matija Franklin@FranklinMatija

With @schmidtsciences, @coop_ai, @ARIA_research and @GoogleOrg, we’re launching a $10M research fund to help understand how AI systems behave as a group and to fund work in multi-agent safety.

We invite researchers to submit proposals in four priority areas:

1. Sandboxes and testbeds 2. The science of agent networks 3. Strengthening agent infrastructure 4. Oversight and control

A big thank you to everyone that was involved including @James_D_Fox, @sebkrier, @weballergy, @lrhammond, and @ObadiaAlex!

46mViews 165Likes 5Bookmarks 0
Séb Krier@sebkrier

@BeatrizGietner 😭😭😭 I deleted system32 and now nothing works

24mViews 13Likes 1Bookmarks 1
Nenad Tomasev@weballergy

The call covers the following priority areas:

1. Sandboxes and testbeds 2. The science of agent networks 3. Strengthening agent infrastructure 4. Oversight and control

Huge thanks to @James_D_Fox, @sebkrier, @FranklinMatija, @lrhammond, @ObadiaAlex, and others who have helped shape this and make it happen!

Nenad Tomasev@weballergy

I am excited to announce this funding call, in collaboration with @schmidtsciences, @coop_ai @ARIA_research and @Googleorg

As we are thinking about moving beyond individual powerful AI agents towards large-scale agentic collectives that can communicate and coordinate towards completing long-horizon complex tasks - it is paramount to design these future agentic societies safely.

42mViews 96Likes 6Bookmarks 0
David Manheim@davidmanheim

@sebkrier Very exciting, much needed!

2hViews 28Likes 4
Theo Jaffee@theojaffee

@sebkrier @alexolegimas Very cool, very neglected research!

40mViews 24Likes 2
Nikiton@Nikitont

@GoogleDeepMind @schmidtsciences @coop_ai @ARIA_research @Googleorg Do I understand correctly that the investments will go specifically to research?

2hViews 68Likes 1
John Scott-Railton@jsrailton

Feels like right moment for this. Writing is on the wall about the limits of rigid guardrails on single models.

And that the next generation = emergent, even sociological problems.

Risk is perhaps over-indexing on the oversight & control frame over building for trustworthy autonomy.

1hViews 48Likes 1
Neetuuuu@neetuug07

@GoogleDeepMind @schmidtsciences @coop_ai @ARIA_research @Googleorg Understanding emergent behaviors in millions of AI agents will be crucial for safe, beneficial collective intelligence

2hViews 84
Arslan Iqbal@thearslaniqbal

@sebkrier Multi-principal means different companies own different agents. That is the real world. Why did it take so long for safety research to focus on that instead of just one perfect AI?

1hViews 19Likes 1
Moltwithme@moltbot_life

@GoogleDeepMind @paulroetzer @schmidtsciences @coop_ai @ARIA_research @Googleorg Hmm, wonder how they’ll actually track these AI group behaviors lol

1hViews 49
seijin@david_saint_

@GoogleDeepMind @schmidtsciences @coop_ai @ARIA_research @Googleorg Reminds me of what we noticed with moltbook. Even though that one might have been staged to some extent

1hViews 46
ZeroMazed@ZeroMazed

@GoogleDeepMind @sebkrier @schmidtsciences @coop_ai @ARIA_research @Googleorg ai agents gonna form some communist hive mind shit lmao

2hViews 46
Alex Shev@AlexshevPm

@GoogleDeepMind @schmidtsciences @coop_ai @ARIA_research @Googleorg Multi-agent behavior is going to need its own safety science. One agent failing is a bug. Many agents coordinating badly becomes an ecosystem problem.

2hViews 40
MetaMaster 天@agenda2033

@GoogleDeepMind @schmidtsciences @coop_ai @ARIA_research @Googleorg i am a multi agent system

2hViews 40
Eric Gilliam@eric_is_weird

@sebkrier @tylercowen Awesome work! Would love to keep my ear to the ground on your behalf. Just lmk if it would be useful!!

1hViews 11Likes 1
Load more posts