Practical notes: Multi-Agent Orchestration: Building and Observing Multi-Agent

This walkthrough rebuilds the path from raw materials to a working system for: Multi-Agent Orchestration: Building and Observing Multi-Agent Systems with LangGraph and LangSmith. The focus is operable steps, explicit checks, and code that you can drop into a repo without guessing intent. For the Overview stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

Introduction: What is LangSmith Agent Orchestration?

When working through the Introduction What is LangSmith stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Checkpoint after expensive steps. Resume should not re-bill the same LLM call when an operator retries a later node.

The Architecture: How Orchestration Works in LangGraph

When working through the The Architecture How Orchestration stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Checkpoint after expensive steps. Resume should not re-bill the same LLM call when an operator retries a later node.

How the Orchestration is Set Up:

When working through the How the Orchestration is stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Checkpoint after expensive steps. Resume should not re-bill the same LLM call when an operator retries a later node. When working through the How the Orchestration is stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

Step-by-Step Reproducible Example

The Step-by-Step Reproducible Example stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Keep graph state flat and typed. Nested blobs hide which node wrote which field and break resume after interrupts.

Prerequisites

The Prerequisites stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Keep graph state flat and typed. Nested blobs hide which node wrote which field and break resume after interrupts.

Step 1: Environment Setup

The Step 1 Environment Setup stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Keep graph state flat and typed. Nested blobs hide which node wrote which field and break resume after interrupts. The Step 1 Environment Setup stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

uv add langgraph langchain-anthropic langsmith python-dotenv

Step 2: The Python Code

For the Step 2 The Python stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Separate client construction from the message loop so providers can be swapped without rewriting the conversation state machine.

import os
from typing import TypedDict, Literal
from dotenv import load_dotenv
from langchain_anthropic import ChatAnthropic
from langchain_core.messages import HumanMessage, SystemMessage
from langgraph.graph import StateGraph, END

# ==========================================
# 0. Load Environment Variables
# ==========================================
# This loads the API keys and LangSmith configs from the .env file
load_dotenv()

# ==========================================
# 1. Define the Shared State
# ==========================================
class AgentState(TypedDict):
    messages: list
    next_agent: str

# ==========================================
# 2. Define the Nodes (The Agents)
# ==========================================
# Initialize Claude 3.5 Sonnet
llm = ChatAnthropic(model="claude-sonnet-4-5-20250929", temperature=0)

def router_node(state: AgentState):
    """Acts as the router. Classifies the user query and directs it to the correct department."""
    system_prompt = SystemMessage(content=(
        "You are a router agent. Look at the user's message and classify it as either "
        "'billing' or 'technical'. Reply with ONLY the word 'billing' or 'technical'."
    ))
    response = llm.invoke([system_prompt] + state["messages"])
    classification = response.content.strip().lower()

    # Update state with the routing decision
    return {"next_agent": classification, "messages": [response]}

def billing_node(state: AgentState):
    """Handles billing-related queries."""
    system_prompt = SystemMessage(content=(
        "You are a billing support agent. Help the user with invoices, refunds, and payments. "
        "Be polite and professional."
    ))
    response = llm.invoke([system_prompt] + state["messages"])
    return {"messages": [response]}

def tech_support_node(state: AgentState):
    """Handles technical issues."""
    system_prompt = SystemMessage(content=(
        "You are a technical support agent. Help the user troubleshoot bugs, login issues, "
        "and software errors. Be analytical and helpful."
    ))
    response = llm.invoke([system_prompt] + state["messages"])
    return {"messages": [response]}

# ==========================================
# 3. Define the Routing Logic
# ==========================================
def route_decision(state: AgentState) -> Literal["billing", "technical"]:
    """Reads the state to decide which node to visit next."""
    next_agent = state.get("next_agent", "technical")
    # Claude is highly instruction-following, but we use 'in' to safely handle
    # any edge cases where it might add conversational filler.
    if "billing" in next_agent:
        return "billing"
    return "technical"

# ==========================================
# 4. Build and Compile the Graph
# ==========================================
workflow = StateGraph(AgentState)

# Add nodes
workflow.add_node("router", router_node)
workflow.add_node("billing", billing_node)
workflow.add_node("technical", tech_support_node)

# Define edges
workflow.set_entry_point("router")
# The magic of orchestration: Conditional routing based on state
workflow.add_conditional_edges(
    "router",
    route_decision,
    {
        "billing": "billing",
        "technical": "technical",
    }
)

# Both specialized agents end the workflow
workflow.add_edge("billing", END)
workflow.add_edge("technical", END)

# Compile the graph
app = workflow.compile()

# ==========================================
# 5. Run the Orchestration
# ==========================================
if __name__ == "__main__":
    # Test Case 1: Billing Query
    print("--- Running Billing Test ---")
    inputs = {"messages": [HumanMessage(content="I was charged twice for my subscription!")]}
    result = app.invoke(inputs)
    print(result["messages"][-1].content)
    print("\n")

    # Test Case 2: Tech Support Query
    print("--- Running Tech Support Test ---")
    inputs = {"messages": [HumanMessage(content="My app keeps crashing when I click the save button.")]}
    result = app.invoke(inputs)
    print(result["messages"][-1].content)

Step 3: Run the script to test

For the Step 3 Run the stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline. Put human approval on edges that spend money or change production data. Compile-time wiring does not equal business completeness.

uv run multiagent-orchestration.py

Step 4: View the Orchestration in LangSmith

For the Step 4 View the stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion. Put human approval on edges that spend money or change production data. Compile-time wiring does not equal business completeness. For the Step 4 View the stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph.

What you will see in LangSmith:

When working through the What you will see stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Checkpoint after expensive steps. Resume should not re-bill the same LLM call when an operator retries a later node.

Conclusion

Operational checklist