The Single-Agent Ceiling
Give GPT-4 a task like "research this topic, write a report, fact-check it, and format it for publication" and watch the quality degrade in real time. The research is decent. The writing is passable. By the formatting step, it's forgotten half the research and the fact-checking is superficial. Context windows are large in 2026 — but attention quality degrades as they fill up, and a 128K-token window stuffed with intermediate reasoning is not the same as a fresh 8K-token window focused on one job.
Multi-agent systems fix this the same way engineering teams do: decomposition. One agent researches. One writes. One reviews. Each has a single role, a single goal, and a fresh context. Devin uses multiple agents for software engineering. Our own video pipeline runs six. CrewAI, LangGraph, and Microsoft AutoGen have made the pattern practical enough that the question isn't "should I use agents" but "how many, and how do I keep them from wasting money."
Why Simple Pipelines Break
Most people's first attempt at multi-agent looks like a chain: output of agent 1 becomes input of agent 2. Simple. It fails for three reasons.
No error recovery. If agent 2 produces garbage, agent 3 blindly processes garbage. There's no feedback loop, no quality gate between steps.
Context explosion. Each agent passes its entire output to the next. By agent 3, the context window is stuffed with irrelevant text from earlier stages. Quality collapses under accumulated noise.
No specialization. All agents use the same model, same temperature, same prompt structure. But a researcher needs high temperature (0.8-0.9) for creative exploration. A fact-checker needs low temperature (0.1-0.2) for precision. Identical settings for both means mediocre at everything.
The pipeline model treats agents like functions. Real teams work like graphs — with feedback, iteration, and role-specific configuration.
Five Components That Make It Work
The Orchestrator
The brain. It receives a task, breaks it into subtasks, assigns them to specialized agents, and manages the workflow. In CrewAI, this is the Crew. In LangGraph, it's the StateGraph. The orchestrator decides which agent runs next, whether the output passed quality, and whether to retry or advance.
The orchestrator matters more than any individual agent. A bad orchestrator with brilliant agents produces worse results than a good orchestrator with average ones. Get the routing logic right before you tune any agent prompts.
Specialized Agents
Each agent gets a role, a goal, a backstory that shapes its behavior, and a model configuration tuned to its task. A researcher runs at temperature 0.8-0.9 for creative exploration. A code reviewer runs at 0.1-0.2 for deterministic precision. An editor uses a smaller, faster model — it doesn't need deep reasoning, just grammar and consistency.
Different models for different jobs. Not every agent needs your most expensive model. Using Claude Opus for a formatting agent is like hiring a senior architect to move furniture.
Tools
Agents without tools can only generate text. With tools, they act. A researcher needs web search. A coder needs a code executor. A data analyst needs database access.
The quality of your tools often matters more than the quality of your model. A mediocre model with a reliable web search tool and a working code executor outperforms a frontier model with no tools. We've tested this repeatedly — tool quality beats model quality for task completion rates.
Shared Memory
Agents need to know what other agents have done. Short-term memory passes context within a single workflow run. Long-term memory persists across sessions — what worked, what failed, what the user prefers.
Vector databases handle this well. Store agent outputs as embeddings, retrieve relevant context for future runs. This is how multi-agent systems improve over time without retraining — the memory layer accumulates institutional knowledge.
The Router
Not every task needs every agent. A simple question goes straight to one agent. A complex research task spins up four in parallel. The router evaluates task complexity and assembles the right team dynamically. This prevents overspending on simple requests and underresourcing complex ones.
The Cost Problem Is Real
Every agent call is an LLM call. A 6-agent pipeline with 3 iterations each makes 18 LLM calls per request. At $0.01 per call, that's $0.18 per request. Sounds fine until you hit scale: 10,000 requests per day is $1,800 daily. $54,000/month.
This is where framework choice matters. CrewAI is the simplest — define agents and tasks, it handles orchestration. LangGraph gives you full control over the execution graph, better retry logic, and human-in-the-loop interrupts. Microsoft AutoGen excels at conversational patterns where agents debate and refine. Pick based on your control needs, not blog post hype.
How many agents is too many? Start with 2-3. Add an agent only when you can prove a single agent can't handle the subtask adequately. Every agent adds latency, cost, and a new failure mode. The best multi-agent systems use the minimum agent count — not the maximum.
What We Learned From Our Own Pipeline
We run CrewAI with 6 agents for video production — researcher, scriptwriter, diagram planner, SEO optimizer, voiceover prep, and publisher. Each has its own model configuration and temperature.
The biggest lesson: agent quality depends entirely on goal specificity. "Write a good script" produces generic content. "Write a 1200-word script in fast-paced explainer style with one sarcastic aside per section, a new diagram reveal every 20 seconds, and concrete numbers in every paragraph" produces something usable. The more specific the goal, the less the agent hallucinates its way through ambiguity.
The second lesson: the voice-edit pass matters more than the initial draft quality. We added a dedicated editor agent that runs after the writer, and article quality jumped measurably. Two passes with different prompts beat one pass with a perfect prompt, every time.
Key Takeaways
- Specialize ruthlessly — one role, one goal, one temperature per agent. The moment you add "and also do X" to an agent's goal, split it.
- Start with two agents, not six — add complexity only when you've proven it's needed. Every agent is a cost multiplier.
- Tools beat models — a mediocre model with good tools outperforms a frontier model with none.
- 18 LLM calls per request adds up fast at scale. Budget $0.10-0.20 per complex request and work backward from there.
- Goal specificity is everything — vague goals produce vague output. Quantify what you want: word count, style, structure, cadence.
Related
- Building Efficient RAG Pipelines with Vector Databases — the retrieval pattern agents use for grounded answers
- LLM Inference: How to Cut Your GPU Bill from $60K to $6K — optimizing the inference layer agents call into