Multi-Agent Systems: When Splitting Work Across Agents Helps, and When It Multiplies Failure

A multi-agent system is an AI application in which two or more model-driven agents, each with its own instructions, tools and context, divide a task and pass work between them. The short answer to when to use one: most tasks are better served by a single, well-tooled agent. Multiple agents pay off in three situations only: when the work genuinely parallelises, when sub-tasks need isolated context or isolated permissions, or when an independent checker adds real verification. Everywhere else, each hand-off between agents is a new place for errors, cost and latency to compound.

The evidence for that answer is now unusually consistent. Anthropic reported in June 2025 that its multi-agent research system beat a single agent by 90.2% on its internal research evaluation, and also that it used about 15 times the tokens of a chat interaction. Google Research reported in January 2026 that multi-agent coordination improved a parallelisable financial-analysis task by 81% and degraded a sequential planning task by 70%. Both findings are true at once, and the difference between them is the subject of this guide.

As of the week of October 19, 2026, every major agent framework makes it easy to add a second, third or tenth agent. That ease is the problem. This guide gives the working model: the four common topologies and what each is good and bad at, the costs with the arithmetic, the published evidence, the design rules that make a multi-agent system survivable, how to evaluate and observe one, and a decision checklist a team can apply this week.

The Default Should Be One Agent

Before choosing a topology, decide whether a task needs an agent at all. Many processes are better as deterministic workflows with a model call at one or two steps; that choice is covered in agent or workflow: how to decide. This guide assumes the task does need an agent, and asks how many.

The major labs agree on the default. OpenAI's A Practical Guide to Building Agents (2025) states: "Our general recommendation is to maximize a single agent's capabilities first. More agents can provide intuitive separation of concepts, but can introduce additional complexity and overhead, so often a single agent with tools is sufficient." It gives two practical triggers for splitting: prompts with so many conditional branches that they become hard to maintain, and tool overload, noting that "some implementations successfully manage more than 15 well-defined, distinct tools while others struggle with fewer than 10 overlapping tools." The fix it recommends first is better tool definitions, not more agents. The craft of that fix is the subject of tool design for agents.

Cognition, the company behind the Devin coding agent, made the argument more bluntly in a June 12, 2025 post titled Don't Build Multi-Agents. Its two principles: "Share context, and share full agent traces, not just individual messages" and "Actions carry implicit decisions, and conflicting decisions carry bad results." Its example is a request to build a Flappy Bird clone split between two sub-agents: one builds a background in the style of a different game, the other builds a bird that does not match it, and the final agent is left to reconcile two incompatible pieces of work. Neither sub-agent made an error in isolation. The failure lives in the split.

The reason a single agent is the default is simple. One agent has one context: everything it has seen and decided is available to its next step. Every split throws part of that context away, and the system then depends on the hand-off to carry what matters.

The Four Topologies

When a team does split, it usually chooses one of four patterns.

Orchestrator and workers. A lead agent plans, delegates sub-tasks to worker agents (often in parallel), and synthesises their results. Workers do not talk to each other. OpenAI calls this the manager pattern, with workers exposed to the manager as tools. Anthropic's research system uses it: a lead agent spawns sub-agents that search in parallel and return condensed findings.

Pipeline. Agents run in a fixed sequence, each transforming the previous agent's output: intake, then classification, then drafting, then review. It resembles a workflow, except that each stage is itself an agent with discretion.

Debate or critic. One agent produces, another critiques or argues the opposite case, and the result is revised or judged. The checker may be a different model, prompted differently, or given different information.

Swarm or peer hand-off. Agents hand control to one another as peers, without a central coordinator, each deciding when to pass the task on. OpenAI calls this the decentralised pattern. Customer-service triage that routes between specialist agents is the typical case.

What Splitting Costs: The Arithmetic

Three costs rise with every agent added. They are easy to underestimate because each looks small in isolation.

Tokens multiply. Each agent must be given its instructions, its tools, and enough context to do its part, and each reads and reasons at full price. Anthropic reported that agents typically use about 4 times more tokens than chat interactions, and multi-agent systems about 15 times more (Anthropic Engineering, June 13, 2025). It also found that token usage alone explained 80% of the variance in performance on the BrowseComp browsing benchmark. That is a double-edged finding: part of what makes multi-agent research better is simply that it spends more. A fair comparison gives the single agent the same budget.

Latency adds up in sequence. A pipeline of five agents that each take eight seconds takes forty seconds before anything returns. An orchestrator with parallel workers waits for the slowest worker, then pays for the synthesis step. For interactive use cases, such as a service desk or a sales assistant, latency is often the binding constraint.

Errors compound across hand-offs. If each agent in a chain completes its step correctly 95% of the time, and steps fail independently, a chain of five succeeds 0.95 to the fifth power, or about 77% of the time. At 90% per step, five steps succeed about 59% of the time. Hand-offs also add their own failure: the summary that leaves out the constraint the next agent needed. Google Research measured the effect directly. In architectures where agents worked independently, errors were amplified 17.2 times; with a central coordinator checking results, amplification fell to 4.4 times (Google Research, January 28, 2026).

The arithmetic leads to a rule: a split must buy more accuracy, speed or safety than the hand-off loses. That is a measurable claim, and it should be measured.

What the Published Evidence Says

The research and practitioner evidence published since 2025 points in one direction.

Multi-agent wins when work parallelises. Anthropic's research system, with Claude Opus 4 as lead agent and Claude Sonnet 4 sub-agents, outperformed a single Claude Opus 4 agent by 90.2% on Anthropic's internal research evaluation. The gains came on breadth-first queries, where several independent lines of inquiry can run at once. Anthropic was explicit about the limits: multi-agent systems do worse on tasks that require "all agents to share the same context or involve many dependencies between agents," and it named most coding tasks as an example.

Multi-agent loses when work is sequential. Google Research evaluated 180 agent configurations across five architectures (the revised paper, arXiv 2512.08296, extends this to 260). Coordination improved performance by about 81% on a decomposable financial-analysis benchmark and degraded it by 70% on a sequential planning benchmark. The paper also reports diminishing returns from coordination once a single agent already performs well, and a cost for tool-heavy tasks, where coordination overhead eats the benefit. Its model predicted the best architecture for 87% of held-out configurations from measurable properties of the task, such as how decomposable it is and how many tools it needs.

Most multi-agent failures are design failures. A study from UC Berkeley researchers and collaborators, Why Do Multi-Agent LLM Systems Fail? (Cemri et al., arXiv 2503.13657, first posted March 2025, revised October 2025), analysed execution traces from seven popular multi-agent frameworks and built a taxonomy of 14 failure modes in three groups: system design issues, misalignment between agents, and weak task verification. Its headline observation is that the performance gains of multi-agent systems on popular benchmarks "are often minimal" compared with single-agent baselines. Among the most common failures were agents ignoring the task specification, repeating steps, not knowing when to stop, and ending before the task was verified.

Debate is not a free accuracy boost. A study presented at ICML 2024, Should we be going MAD? (Smit et al.), found that multi-agent debate often failed to beat simpler single-agent methods such as chain-of-thought and self-consistency, while using more compute, though some debate configurations did better after careful tuning. An independent checker helps when it brings something the producer lacks: different information, a different model, or a deterministic test. A second copy of the same model asked to "check the work" mostly shares the first one's blind spots.

Practitioners report the same early failures. Anthropic described its own early versions "spawning 50 subagents for simple queries, scouring the web endlessly for nonexistent sources, and distracting each other with excessive updates." The fixes were budgets and explicit guidance on how much effort a query deserves. A wider catalogue of what goes wrong in production is in why AI agent frameworks fail in production.

When Multiple Agents Genuinely Help

Taken together, the evidence supports three legitimate reasons to split.

1. The work parallelises. The task breaks into parts that do not depend on each other's intermediate results: researching ten competitors, reviewing two hundred contracts against the same checklist, checking a release against several independent test suites. Parallel workers cut wall-clock time and let each worker focus its context on one part.

2. Sub-tasks need isolated context or isolated permissions. Some work should not share a context window: a sub-task that reads untrusted external content should not sit in the same context as an agent that can send payments or change records. Some work should not share permissions: the agent that reads HR records should not be the agent that drafts the public job post. Isolation is a security property, and it is often the strongest reason to split. The identity side of this is covered in identity and access management for AI agents.

3. An independent checker adds real verification. A second agent that checks against something the first did not have, such as the source document, a policy rule set, a test suite or a different model family, catches errors the producer cannot. A checker without independent information is ritual, not verification.

If none of the three applies, the split is a design preference, and the evidence says it will cost accuracy, money and time.

Design Rules for Systems That Do Split

When a split is justified, five rules keep it survivable.

Narrow contracts between agents

Each agent gets a precise job description: what it receives, what it must return, in what format, and what it must not do. A worker told "research the market" will wander. A worker told "return the five largest competitors in this segment with revenue, source and date, in this schema" will not. Treat each agent's interface as an API.

Structured hand-offs

Pass structured data between agents, not free-text summaries, wherever possible: fields for the facts found, the sources, the decisions made and the open questions. Include the constraints the next agent needs, not only the results. Cognition's first principle, share full traces rather than individual messages, is the extreme form of this rule; a typed hand-off record is the practical one.

A single owner of state

One agent, or better, one deterministic component, owns the task's state: what has been done, what has been decided, and what remains. Workers read from it and propose updates; they do not each keep their own version of the truth. Systems in which every agent keeps its own memory drift apart. The broader treatment of agent state is in agent memory in production.

Isolated permissions per agent

Each agent gets only the tools and data access its job needs. The research worker can read but not write. The drafting agent cannot send. The agent that touches untrusted content cannot call tools with side effects. This limits the blast radius when one agent is confused or manipulated.

Budgets and stop conditions

Every agent and the system as a whole need explicit limits: maximum sub-agents spawned, maximum tool calls, maximum tokens, maximum wall-clock time and maximum hand-offs per request, plus a clear definition of done. The MAST study found that not knowing when to stop and repeating steps are among the most common failures. Budgets turn an open-ended loop into a bounded one. Where an action is consequential, a human approval step belongs in the design too; where to put it is covered in designing human approval for agents.

How to Evaluate and Observe a Multi-Agent System

A multi-agent system is harder to test than a single agent because failures emerge between components. Four practices make it manageable.

Benchmark against a single agent at the same budget. The only honest test of a split is a comparison with one well-tooled agent given the same token and time budget on the same evaluation set. If the multi-agent version does not win by a clear margin, remove it. The case for evaluation sets as a management tool is made in AI evals for executives.

Evaluate each agent and each hand-off. Score each agent on its own contract (did the worker return what was asked, in the schema, with sources?) and score each hand-off (did the receiving agent get the constraints it needed?). End-to-end scores alone cannot tell a team where the system broke.

Trace everything, with one trace per task. Every agent's inputs, tool calls, outputs and hand-offs should land in a single trace that can be read end to end. Without it, a failure in a five-agent system is a mystery. The tooling is covered in agent observability in production.

Watch cost and loop metrics in production. Track tokens per task, sub-agents per task, hand-offs per task and the share of tasks that hit a budget limit. A rising hand-off count or budget-limit rate is the earliest signal that a system is looping or that its decomposition has stopped fitting the work.

A Decision Table a Team Can Apply This Week

The checklist before shipping a multi-agent design

• [ ] One of the three reasons (parallel work, isolation, independent verification) applies, and is written down.
• [ ] The multi-agent version beats a single agent at the same budget on the evaluation set.
• [ ] Every agent has a written contract: inputs, outputs, schema, prohibitions.
• [ ] Hand-offs are structured records that carry constraints, not only results.
• [ ] One component owns task state.
• [ ] Each agent has least-privilege tools and data access.
• [ ] Budgets exist for sub-agents, tool calls, tokens, time and hand-offs, with a definition of done.
• [ ] One trace per task covers every agent.
• [ ] Production dashboards show tokens, hand-offs and budget-limit hits per task.

Key Takeaways

• Default to one well-tooled agent. OpenAI's guidance, Cognition's experience and the research all point the same way: maximise a single agent first.
• Split for three reasons only: work that genuinely parallelises, sub-tasks that need isolated context or permissions, and independent checks that bring information the producer lacks.
• Every hand-off compounds error. Five steps at 95% each succeed about 77% of the time; Google Research measured error amplification of 17.2 times for independent agents against 4.4 times with central coordination.
• Multi-agent is expensive by design. Anthropic reports about 15 times the tokens of chat for its multi-agent system; compare against a single agent at the same budget.
• The evidence splits by task shape: about 81% better on parallel financial analysis, 70% worse on sequential planning in Google Research's study.
• Design rules make splits survivable: narrow contracts, structured hand-offs, a single owner of state, least-privilege permissions per agent, and hard budgets with stop conditions.

Frequently Asked Questions

When should an enterprise use multiple AI agents instead of one?

Use multiple agents when the task splits into independent parts that can run in parallel, when sub-tasks need isolated context or permissions (for example, separating an agent that reads untrusted content from one that can take actions), or when an independent checker can verify the output against a source, rule set or test. Otherwise, a single agent with well-designed tools is usually more accurate, cheaper and faster.

Are multi-agent systems more accurate than single agents?

Sometimes, and only on the right tasks. Anthropic reported a 90.2% improvement on its internal research evaluation for breadth-first research, and Google Research found an 81% gain on a parallelisable financial-analysis task but a 70% loss on a sequential planning task. Part of the multi-agent advantage comes from spending more tokens, so comparisons should hold the budget equal.

How much more do multi-agent systems cost to run?

Anthropic reported that its agents use about 4 times the tokens of a chat interaction and its multi-agent system about 15 times. Costs rise with each agent because each must be given instructions, tools and context and must reason at full price, and sequential designs add latency at every stage.

Why do multi-agent systems fail?

The MAST study of seven frameworks grouped 14 failure modes into system design issues, misalignment between agents and weak verification. Common examples are agents ignoring the task specification, repeating steps, not recognising when to stop, and finishing without verifying the result. Most are design failures that contracts, structured hand-offs, a single owner of state and budgets can prevent.

Is a second agent reviewing the first a good safeguard?

Only if the reviewer has something the producer did not: the source documents, a policy rule set, a deterministic test, or a different model. Research on multi-agent debate presented at ICML 2024 found that debate often failed to beat simpler single-agent methods while using more compute. A same-model reviewer tends to share the producer's blind spots.