Build a Multi-Agent Router with LangGraph and a Supervisor Agent

Put a single agent in front of a real product and it soon starts doing too much. Ask one agent to handle customer support, order tracking and refunds, and it ends up mediocre at all three. Its system prompt grows into a 2,000-word tangle of "if the user asks about X, do Y, but if it's also about Z then...". The cure isn't a cleverer agent. It's more, smaller agents plus one supervisor that routes between them. That is arguably the most widely used production pattern in agentic AI at the moment, and a working version fits in a short LangGraph program.

The trouble with one giant agent

Say you're building an assistant that answers general questions, does arithmetic and searches the web for current facts. The tempting design is one agent, one huge prompt and every tool attached. As tools and instructions pile up, the model gets unsure which tool to pick and its reasoning gets diluted. Debugging turns into a nightmare, because you can't isolate why it chose the wrong tool when everything lives in one prompt.

The remedy is to divide responsibilities: one agent for math, one for research, and a supervisor, which is just a router, deciding which specialist handles each part of a request.

The supervisor pattern

Think of a call centre. You phone with a question and don't get put through to the expert directly. A receptionist listens first, decides whether you need billing, technical support or sales, and transfers you. The receptionist is the supervisor and the three teams are the specialist agents. The supervisor never answers the question itself. Its sole job is choosing who should.

Visually it looks like this:

                    ┌─────────────┐
                    │    User     │
                    │   Request   │
                    └──────┬──────┘
                           │
                           ▼
                   ┌───────────────┐
                   │   Supervisor  │
                   │  (routes task)│
                   └───────┬───────┘
                    ┌──────┴──────┐
                    ▼             ▼
             ┌────────────┐ ┌────────────┐
             │  Research  │ │    Math    │
             │   Agent    │ │   Agent    │
             └──────┬─────┘ └──────┬─────┘
                    │              │
                    └──────┬───────┘
                           ▼
                   ┌───────────────┐
                   │   Supervisor  │
                   │ (final answer)│
                   └───────────────┘

The supervisor takes the task, passes it to the right specialist, receives the result, and then decides whether to route once more or to give the user the final answer.

The recommended build, step by step

A point worth knowing before writing code: the LangChain team now suggests building the supervisor pattern straight from tool calling, and not relying only on the older langgraph-supervisor helper library, because tool calling gives more control over context engineering. Below you'll find the quick prebuilt version first and then a manual one so it's clear what happens underneath.

Step 1: install the packages

pip install langgraph langchain-openai langgraph-supervisor

Step 2: build the specialists

Begin with the imports:

from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent

The first specialist handles math, and its tools are two small functions. Addition comes first:

# Specialist 1: handles math
def add(a: float, b: float) -> float:
    """Add two numbers together."""
    return a + b

Then multiplication:

def multiply(a: float, b: float) -> float:
    """Multiply two numbers together."""
    return a * b

Wrap them in a ReAct agent with a name and a prompt that limits its role:

math_agent = create_react_agent(
    model="openai:gpt-4o",
    tools=[add, multiply],
    name="math_expert",
    prompt="You are a math specialist. Solve numeric problems precisely."
)

The second specialist does research. Its one tool is a search function, stubbed out here:

# Specialist 2: handles research
def web_search(query: str) -> str:
    """Search the web for current information."""
    # In production, replace this with a real search API call
    return f"Search results for '{query}': [your real search integration here]"

Wrap that in an agent as well:

research_agent = create_react_agent(
    model="openai:gpt-4o",
    tools=[web_search],
    name="research_expert",
    prompt="You are a research specialist. Find accurate, current information."
)

Each agent has exactly one job and only the tools that job requires, so there's no confusion and no overloaded prompt. The examples use OpenAI models, though a free alternative such as Groq or OpenRouter would work as well.

Step 3: build the supervisor

Import the helper:

from langgraph_supervisor import create_supervisor

Then give it the two specialists, a model and a prompt telling it when to use each:

workflow = create_supervisor(
    [research_agent, math_agent],
    model=ChatOpenAI(model="gpt-4o"),
    prompt=(
        "You are a team supervisor managing a research expert and a math expert. "
        "For questions requiring current information or web lookups, use research_expert. "
        "For anything involving calculation, use math_expert. "
        "Route each request to exactly the right specialist."
    )
)

Compile the workflow into a runnable app:

app = workflow.compile()

Step 4: run it

Send a math question:

result = app.invoke({
    "messages": [
        {"role": "user", "content": "What's 1,847 multiplied by 293?"}
    ]
})

And print the final message:

print(result["messages"][-1].content)

Here is what happens under the hood:

1. The supervisor reads the request.
2. It sees a calculation and not a research question.
3. It hands the job to math_expert.
4. math_expert works out the answer with its multiply tool.
5. The supervisor gets control back and returns the answer.

For a research question, such as the current headcounts of big tech companies, and the same supervisor routes it to research_expert. The code is identical and only the routing decision differs, and the LLM makes that decision on its own.

What the handoff tools really are

This part removes the sense of magic. The supervisor doesn't route through some hidden LangGraph mechanism. It routes with tool calls, exactly like any ordinary tool-calling agent. Behind the scenes create_supervisor builds a handoff tool for each specialist, with names such as transfer_to_research_expert and transfer_to_math_expert. The supervisor model sees these as callable tools, no different from a weather lookup, so when it concludes research is needed it simply calls the transfer function. The consequence is reassuring: a supervisor isn't a new concept at all. It's the usual agent loop where some of the tools happen to be other agents instead of API calls.

Building it by hand

If you want full control over routing, which is advisable as you head toward production, here's the manual version on top of a raw StateGraph. Start with the imports:

from typing import Annotated, Sequence, TypedDict
import operator
from langchain_core.messages import BaseMessage
from langgraph.graph import StateGraph, END

Define the shared state, which holds the message list and the name of the next node:

class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], operator.add]
    next: str

The supervisor node looks at the latest message:

def supervisor_node(state: AgentState):
    """Decide which specialist should act next."""
    last_message = state["messages"][-1].content

In production the decision would come from an LLM call. Here it's reduced to simple keyword checks so the routing logic stays visible:

# In production, this decision comes from an LLM call —
    # simplified here to show the routing logic clearly
    if any(word in last_message.lower() for word in ["calculate", "multiply", "add", "sum"]):
        return {"next": "math_agent"}
    else:
        return {"next": "research_agent"}

Each specialist gets a node that invokes its agent and then hands control back to the supervisor. Math first:

def math_node(state: AgentState):
    result = math_agent.invoke(state)
    return {"messages": [result], "next": "supervisor"}

Then research:

def research_node(state: AgentState):
    result = research_agent.invoke(state)
    return {"messages": [result], "next": "supervisor"}

Next, wire the graph together by registering the nodes:

# Wire the graph together
workflow = StateGraph(AgentState)
workflow.add_node("supervisor", supervisor_node)
workflow.add_node("math_agent", math_node)
workflow.add_node("research_agent", research_node)

Set the entry point:

workflow.set_entry_point("supervisor")

Add the conditional edges that follow the supervisor's decision:

workflow.add_conditional_edges(
    "supervisor",
    lambda state: state["next"],
    {"math_agent": "math_agent", "research_agent": "research_agent", "end": END}
)

Send each specialist back to the supervisor afterwards:

workflow.add_edge("math_agent", "supervisor")
workflow.add_edge("research_agent", "supervisor")

And compile:

app = workflow.compile()

It's wordier, but nothing is hidden. You can see how state flows, how each routing decision is made, and where to add your own logic such as logging, guardrails or custom termination conditions.

The state design choice that ages worst

A warning from production experience: the state schema is the hardest decision to reverse later. It's tempting to toss every field you might want into one flat dictionary early on, covering messages, user context and intermediate results. That's fine with two agents. With eight or ten nodes all reading and writing the same shared state it becomes painful, because LangGraph's type checking is strict and untangling a bloated state means touching every node. Design the schema deliberately from the first day, even for a two-agent system. Decide what each agent really needs to read and write, and keep everything else out of shared state.

A failure you will meet

Supervisor systems that reach production tend to show this pattern. Imagine the supervisor delegating a job. The specialist completes it, yet the supervisor judges the result lacking and bounces it back. The specialist revises, and the new version is actually worse than the first. The supervisor is still unhappy and sends it back again, and unless you guard against it this can go round indefinitely.

The remedy is a cap on supervisor round-trips, which can be a simple counter in the state, together with a clear, deterministic stop condition. An LLM's opinion on whether something is good enough isn't a dependable sole stopping rule.

When not to use it

Be honest about fit. If the job is one retrieval followed by one generation, as in a plain RAG-style Q&A, a supervisor is overkill. The extra state management and routing overhead buys nothing when no real decision exists to route. Reach for the supervisor pattern once you have truly distinct specialists and a genuine routing choice, and don't adopt it by default.

Recap

• One overloaded agent breaks down as complexity rises.
• A supervisor is a router agent delegating to specialists.
• create_supervisor is the quick, prebuilt route.
• Underneath, handoffs are ordinary tool calls.
• A manual StateGraph gives complete routing control.
• Two production lessons: design the state schema early and cap revision loops.