Multi-Agent Safety Is Becoming a Protocol Problem

A multi-agent safety funding call that closed this week is a more durable signal than many model launches. Google DeepMind, Schmidt Sciences, the Cooperative AI Foundation, ARIA, and Google.org are backing testbeds, the science of agent networks, identity and reputation infrastructure, and new oversight mechanisms. The object of safety research has expanded from a model to a society of machines.

Single-agent systems already face prompt injection, data leakage, excessive permissions, and unsafe tool use. Networks add relational risk. One agent delegates to another, which invokes a service controlled by a third party. Intent, authority, and data move through the chain, while responsibility becomes harder to locate.

This is not merely microservices with natural language. Services usually operate behind predefined interfaces and organizational boundaries. Agents can form plans at runtime, choose collaborators, create sub-tasks, and revise their path based on untrusted inputs. Conventional authentication can prove who called an API; it cannot explain why the call was part of a legitimate task.

The first missing layer is verifiable identity. An agent identity must say whom it represents, what models and tools it uses, which capabilities it holds, whether it may delegate them, and who is accountable for its commitments. An API key alone cannot support reputation, liability, or meaningful audit.

Authority must also be task-scoped. A travel assistant may need calendar access, corporate policy, flight search, and a payment path, but it should not retain broad access to all four systems indefinitely. Capabilities should be assembled for a purpose, bounded by budget and time, and explicitly constrained when passed to a sub-agent.

Commitment and reputation form the next layer. If two agents agree on a price or delivery date, the system needs machine-readable rules for when that agreement binds, how it can be revoked, and what happens after failure. Reputation cannot be one global score either; it must be specific to a task, provenance, authority level, and recent evidence.

The emphasis on sandboxes is equally important. Many network failures emerge from interactions rather than a single malicious action: local incentives can amplify into collusion, resource runs, information cascades, or diffused responsibility. Static benchmarks rarely capture those dynamics. Controlled environments can reveal how heterogeneous agents behave under competition and incomplete information.

Builders can apply the lesson now. A primary agent that calls coding agents, browser automation, payment tools, and external MCP servers is already a small network. Every participant needs a stable identity; every delegation needs scope, budget, and re-delegation rules; every consequential tool call needs provenance; every high-risk action needs an accountable approver.

The interface should expose a task ledger rather than a stream of hidden reasoning: objective, participants, granted capabilities, important evidence, anomalies, and final delivery. Users do not need every internal token. They do need to know who is acting, what can be touched, and which consequences come next.

Over time, identity, capability authorization, agent discovery, reputation, settlement, audit, and dispute handling may become distinct layers in an agent protocol stack. Better weights alone will not create network order. The critical question is no longer only how to make one AI safe, but how to let many partially trusting AIs share a world without losing accountability.