Excited that @ARIA_research's Scaling Trust is co-launching this $10m funding call on safety and security for multi-agent multi-principal systems @GoogleDeepMind, @coop_ai, @schmidtsciences and @Googleorg ⚡️
If you work on testbeds for agent ecosystems, the science of how collective capabilities emerge (and fail), trustworthy agent-to-agent interactions, or oversight of agent populations at scale — apply! Grants up to $1M, deadline Aug 8.
Shoutout to @sebkrier @lrhammond @James_D_Fox @weballergy @FranklinMatija @HaleSirin_ @iamnotnicola, @MjaBradshaw and everyone else involved for their partnership so far, and excited for what's ahead!
Read more details below, link in replies:
AI agents are increasingly being deployed in multi-agent settings. While most present-day cases involve teams of agents orchestrated by a single actor (or ‘principal’), we are beginning to see the emergence of more complex ecosystems of agents deployed by different actors across shared digital infrastructure. These multi-principal, multi-agent interactions create new opportunities for cooperation and shared benefit, but also new risks, which means focusing only on the safety and alignment of individual models is insufficient.
More research is therefore urgently needed to understand safety and risk through a system-level, multi-agent lens – developing methods to analyse emergent collective dynamics, building infrastructure for trustworthy interaction between agents, and creating scalable approaches for monitoring and control of increasingly complex networks of AI systems. While some of these problems will be addressed by market forces, we expect others to fall through the gaps. This funding call aims to fill those gaps, catalysing the foundational scientific research needed to understand, evaluate, and control risks emerging from large-scale ecosystems of interacting AI agents, deployed by multiple actors.
The call has been inspired by three recent papers. First, Google DeepMind’s “Distributional AGI Safety” outlines the safety implications of highly capable AI systems emerging not as single monolithic agents, but through coordinated networks of specialised sub-AGI systems with differential access to tools, data, memory, and resources. Second, ARIA’s “Scaling Trust” programme thesis argues that, in a world of increasingly capable networked agents acting across digital and physical environments, coordination infrastructure that lets agents enter into 'contracts' securely, programmatically, at scale, and without intermediaries can preserve pluralism and unlock new forms of coordination. Finally, the Cooperative AI Foundation’s “Multi-Agent Risks from Advanced AI” report argues that interacting populations of AI agents introduce qualitatively new failure modes beyond single-agent systems, including collusion, conflict, destabilising dynamics, emergent agency, and novel multi-agent security vulnerabilities.
