Multi-agent AI systems in a mid-sized company: when one agent is not enough

An AI agent is a program based on a language model (like the one in ChatGPT). It does not just answer questions but carries out steps itself: it checks data in a system, fills in a form, prepares a document. It follows instructions and only has access to the tools you give it.

A multi-agent system is several such agents, each with its own role, instructions and permissions. The agents pass the case, or parts of it, to one another and put the result together. Think of the difference between one experienced employee with a checklist and a small team with a manager.

A team has the edge when tasks can be split up and done in parallel. But it has its costs too: every member has to understand the case, and passing information on takes time and leads to mistakes. It is exactly the same with AI agents.

In business practice you come across three set-ups. They differ in who decides the order of work and where a human steps in.

In a mid-sized company the first or second set-up in the table below usually makes the most sense, always combined with the third – a threshold for human review. You will also come across a set-up in which agents “negotiate” among themselves without supervision. It is worth leaving that for later, because it is the hardest to check and explain.

The signals below help you judge whether a process really needs more than one agent. One signal is usually not enough. When three or more apply, it is worth considering a multi-agent system.

Be wary of the argument that “one agent cannot hold all the documents”. Modern models accept very long texts in one go. The real reason for splitting the work is more likely clarity, and the fact that different roles should have different permissions.

In the documentation for its agent-building library, Microsoft gives simple advice: if a task can be written as an ordinary program function, do not use an AI agent for it. Still less should you use several agents.

Anthropic, for its part, describes multi-agent systems as struggling where all agents need to know the same full context or where the steps depend heavily on one another. Several agents work well when the work can be split into independent parts.

In a mid-sized company this means that most processes – replying to typical emails, reading documents, preparing a draft quote – can be handled by one agent or by automation in Power Automate. Several agents are a solution for a narrow group of complex cases.

The examples below are hypothetical and show a type of process, not the results of specific implementations. What they have in common is that one case requires several independent assessments, and a person makes the final decision.

Purchases above an agreed threshold. Take a manufacturing company where purchase requests above PLN 50,000 are checked by several people. One agent compares the request with the purchasing policy, a second checks the department’s budget, and a third reads the contract terms. A coordinating agent combines their comments into one note for the director, who approves or rejects the request.

Complex customer requests. An agent identifies the topic of a request and routes it to an agent for technical, billing or contract matters. The specialist agent prepares a proposed reply based on the customer’s history. When a case goes beyond its permissions, it passes it to an employee with a ready summary.

Preparing a salesperson for a meeting. One agent gathers public information about the customer’s company, a second checks the history of the relationship in the CRM, and a third suggests questions for the meeting. The salesperson gets one page of notes instead of looking for information in several places.

You usually pay for AI models by the number of tokens processed – fragments of text, roughly pieces of words. Each agent has to read its part of the case, and often the instructions and results of other agents too. That is why the cost grows faster than the number of agents.

A description of Anthropic’s research system shows the scale. A single agent there used about 4 times more tokens than an ordinary chat with AI, and the multi-agent system about 15 times more. Your process will have different proportions, but the direction is the same.

To estimate this for your own process, measure the usage of one agent on a few dozen real cases. Multiply by the number of agents that read the case and add a margin for the information they exchange. Multiply the result by the monthly number of cases and compare it with the current cost of people’s work.

The cost can be reduced in the ways listed below. It is worth asking the vendor about them before you sign the contract.

A single agent’s decision can be reconstructed: you can see what it received, what it used and what it answered. With several agents, the result depends on the work of each of them and on what they passed to one another. Without a record of every step, you will not be able to explain to a customer or an auditor where a decision came from.

The law reinforces this need. The GDPR gives people the right not to be subject to decisions with legal effects on them that are made solely by automated means. If agents are to take part in, for example, assessing job candidates, AI Act obligations for high-risk systems will also apply from 2 December 2027.

Below is a minimum set of oversight rules. It is best to agree them before go-live, not after the first problem.

You do not need to know these tools, but it helps to know what to ask the vendor. Some are ready-made platforms where agents are configured in a web browser. Others are libraries for developers – ready-made sets of code that give more freedom but need a technical team to maintain them.

For companies working in Microsoft 365, the natural starting point is Copilot Studio, where one agent can hand tasks over to other agents. When a process needs more control, the vendor usually turns to one of the libraries in the table below. This market changes quickly, so ask about the current version and the development roadmap of the chosen tool.

The most common mistake is building a system of several agents before a single agent has been tested on real cases. It is safer to go through the stages below and make a conscious decision after each one.

An important element is shadow mode. The agents see real cases and propose decisions but carry nothing out – people work as before and compare their decisions with the AI’s proposals.