Graph as Architecture vs Graph as Data: LangGraph First, GraphRAG Later

A common first instinct on a new agent project is to reach for GraphRAG. Not because it was measured against plain vector search on the team’s own traffic, but because a knowledge graph feels more advanced—entities, edges, structured traversal instead of blind similarity—the way a normalized schema feels “more correct” than a spreadsheet even when the spreadsheet already works.

Articles on graph engineering and turning large corpora into agentic knowledge graphs reinforce that framing: graphs as the grown-up substrate for agent memory. Filed as a general retrieval truth rather than a claim scoped to particular workloads, many designs start by asking “do we need a graph database?” instead of “do we need retrieval beyond similarity at all—and if so, which kind?”

Two papers in sequence make that assumption worth checking rather than repeating.

The first is an evaluation paper, “How Significant Are essentially a Real Performance Gains? An Unbiased Evaluation Framework for GraphRAG” (arXiv:2506.06331). It audited the LLM-as-judge evaluations behind many early win-rate claims. Pairwise judges carry familiar biases: position bias (swapping answer order can swing reported win rates by more than thirty points), length bias (longer, more elaborate answers win on style), and trial bias (identical comparisons disagree across re-runs). After corrections, one popular method’s reported 66.7% win rate over baseline RAG fell to about 39%—below a coin-flip line. That is not “gains were smaller.” That is “after removing judge bias, gains for that method on that evaluation may vanish.”

The second is GraphRAG-Bench (arXiv:2506.02404, accepted essentially at ICLR essentially 2026), built to stop essentially essentially relying on judge win rates plus instead grade GraphRAG methods against plain text-chunk retrieval on verifiable questions across sixteen disciplines. The result that matters for production traffic: on simple fact lookup—“what is X” questions that dominate many live RAG systems—plain text chunks scored 60.9 plus graph-based retrieval scored 60.1. Effectively a tie, with network construction, storage, plus query complexity added for no measurable benefit on that category.

GraphRAG-Bench results by question category (approximate, from published results)
+----------------------------+---------------+---------------+------------+
| Question category           | Text chunks   | Graph retrieval| Winner     |
+----------------------------+---------------+---------------+------------+
| Simple fact retrieval        | 60.9          | 60.1          | Tie        |
| Complex reasoning            | 42.9          | 53.4          | Graph +10.5|
| Contextual summarization     | 51.3          | 64.4          | Graph +13.1|
+----------------------------+---------------+---------------+------------+

Interpret carefully. Neither paper is a last word. essentially essentially essentially Benchmarks age in months; GraphRAG-Bench covers nine methods versus one chunking baseline; a method that loses on simple lookup here may still fit a corpus a suite does not resemble. The bias audit does not say graphs never help; it says a specific evaluation style inflated specific claims—narrower and more useful than “graphs fail.” Together they establish something buildable: graph retrieval is not a categorical upgrade over vectors. It wins clearly on complex reasoning and summarization that must synthesize across many contexts. It does not reliably beat plain retrieval on a simple lookups that dominate a lot of real usage. That boring fact is strong enough to hang a decision process on.

There is a second confusion that lasts longer than it should: the word “graph” does two unrelated jobs.

Two things called “graph,” doing nothing alike

Graph as data is what GraphRAG usually means: a knowledge graph of entities and relationships extracted from a corpus, stored as nodes and edges, used as the retrieval substrate instead of—or beside—chunked text and embeddings. It answers: where does information live, and how does the agent find the relevant piece?

Graph as architecture is what tools like LangGraph mean: a state machine for control flow—which step runs next, under what condition, when execution pauses for a human, how the run survives a crash and resumes. It answers: how does execution move from step to step? That has nothing to do with where facts are stored.

The two meanings of "graph" in agent engineering
+------------------+---------------------------------+---------------------------------+
| Aspect            | Graph as architecture            | Graph as data                    |
+------------------+---------------------------------+---------------------------------+
| Answers            | How does control flow move        | Where do facts live and how      |
|                    | between agent steps               | do I retrieve the right ones     |
+------------------+---------------------------------+---------------------------------+
| Nodes represent    | Units of work: classify, fetch,   | Entities: people, orders,        |
|                    | decide, respond                    | products, concepts               |
+------------------+---------------------------------+---------------------------------+
| Edges represent    | Transitions and conditions         | Relationships: "works for",      |
|                    | between steps                      | "purchased", "caused by"         |
+------------------+---------------------------------+---------------------------------+
| Typical tool       | LangGraph, a hand-rolled state     | Neo4j, a Cosmos DB graph API,     |
|                    | machine, an orchestration library  | a GraphRAG pipeline               |
+------------------+---------------------------------+---------------------------------+
| Cost to adopt       | Low. A dependency and some         | High. Extraction pipeline,        |
|                    | Python classes                     | graph store, query layer          |
+------------------+---------------------------------+---------------------------------+
| When it's wrong     | Almost never, once you have        | When your questions are simple    |
| to skip it          | more than a couple of steps        | lookups a vector or keyword       |
|                    | and any branching                  | search already answers            |
+------------------+---------------------------------+---------------------------------+

Separated cleanly, a sane order appears. essentially essentially Graph-as-architecture essentially essentially essentially essentially is cheap essentially plus useful almost immediately for any orchestrator with more than one or two steps—little reason not to adopt it early. Graph-as-data is expensive; payoff depends on whether questions need relationship traversal, which can be checked before building. Treating a expensive, conditional piece as a default plus a cheap, near-universal piece as an afterthought is backwards. Building a right way around looks like this.

Building the architecture: a real LangGraph agent with checkpointing and a human interrupt

Consider a small support-triage agent. It classifies the ask, fetches account data, decides whether risk requires human sign-off, then responds. Refunds need a human; balance and order-status lookups do not. The example is small, but the mechanisms—state schema, conditional routing, checkpointing, interrupt—are the same ones used in production, just with fewer nodes.

Install dependencies:

pip install langgraph langgraph-checkpoint-sqlite

Shared state every node reads and writes:

from typing import TypedDict, Literal
class AgentState(TypedDict):
    user_message: str
    intent: str
    risk_level: str
    account_data: dict
    human_decision: str
    final_response: str

Four nodes follow. classify_intent tags the message with intent and risk:

def classify_intent(state: AgentState) -> dict:
    message = state["user_message"].lower()
    if "refund" in message or "cancel" in message:
        intent, risk = "refund_request", "high"
    elif "balance" in message or "order status" in message:
        intent, risk = "account_lookup", "low"
    else:
        intent, risk = "general_question", "low"
    return {"intent": intent, "risk_level": risk}

fetch_data stands in for a CRM, order store, or billing API:

def fetch_data(state: AgentState) -> dict:
    fake_db = {
        "refund_request": {"order_id": "ORD-4471", "amount": 249.00, "status": "delivered"},
        "account_lookup": {"balance": 1200.50, "last_order": "ORD-4402"},
        "general_question": {},
    }
    return {"account_data": fake_db[state["intent"]]}

Routing is not a node; it is the conditional edge choosing the next node:

def needs_human(state: AgentState) -> Literal["human_review", "respond"]:
    return "human_review" if state["risk_level"] == "high" else "respond"

human_review is where interrupt() pauses the whole graph mid-run and returns control to the caller without losing state:

from langgraph.types import interrupt
def human_review(state: AgentState) -> dict:
    decision = interrupt({
        "question": f"Approve {state['intent']} for order {state['account_data'].get('order_id')}?",
        "amount": state["account_data"].get("amount"),
    })
    return {"human_decision": decision}

respond emits the final answer, incorporating a human decision when needed:

def respond(state: AgentState) -> dict:
    if state["risk_level"] == "high":
        if state.get("human_decision") == "approve":
            text = f"Refund approved for {state['account_data']['order_id']}."
        else:
            text = "This request needs manual follow-up before we can proceed."
    else:
        text = f"Here's what I found: {state['account_data']}"
    return {"final_response": text}

Wiring makes the point of the exercise: an explicit, readable control-flow description instead of nested conditionals in one giant function:

from langgraph.graph import StateGraph, START, END
builder = StateGraph(AgentState)
builder.add_node("classify_intent", classify_intent)
builder.add_node("fetch_data", fetch_data)
builder.add_node("human_review", human_review)
builder.add_node("respond", respond)
builder.add_edge(START, "classify_intent")
builder.add_edge("classify_intent", "fetch_data")
builder.add_conditional_edges(
    "fetch_data",
    needs_human,
    {"human_review": "human_review", "respond": "respond"},
)
builder.add_edge("human_review", "respond")
builder.add_edge("respond", END)

Checkpointing is what separates a durable agent from a script. LangGraph’s SQLite checkpointer is fine for single-process or local work—no external database required:

from langgraph.checkpoint.sqlite import SqliteSaver
with SqliteSaver.from_conn_string("support_agent_checkpoints.sqlite") as checkpointer:
    graph = builder.compile(checkpointer=checkpointer)
    config = {"configurable": {"thread_id": "ticket-8821"}}
    result = graph.invoke({"user_message": "I want a refund for my last order"}, config)
    print(result)

On a refund-shaped message the graph runs classify and fetch, takes the high-risk edge into human_review, and stops: interrupt() halted execution. The result carries an __interrupt__ description of what is waiting—not a finished final_response. The process can crash here; the machine can reboot; every step’s state already landed in support_agent_checkpoints.sqlite. Reopen the same file and thread id and the graph is exactly where it paused:

with SqliteSaver.from_conn_string("support_agent_checkpoints.sqlite") as checkpointer:
    graph = builder.compile(checkpointer=checkpointer)
    config = {"configurable": {"thread_id": "ticket-8821"}}
snapshot = graph.get_state(config)
    print(snapshot.next)  # ('human_review',)
    from langgraph.types import Command
    final = graph.invoke(Command(resume="approve"), config)
    print(final["final_response"])

Command(resume="approve") feeds a value back into the paused interrupt(), as if a reviewer clicked approve. Classification and fetched data are not recomputed; they waited in the checkpoint.

Skipping the checkpointer and holding state in an in-memory dict “until later” produces a familiar failure: restart during mid-review and the in-flight ticket vanishes—no error, no trace—because nothing outside process memory knew it existed. Checkpointing exists to prevent that failure mode; treat it as part of the graph, not an optional bolt-on.

When graph as data actually earns its complexity

None of this brands GraphRAG as worthless. It brands it as the wrong first idea—add it when a concrete need appears, not because it sounds sophisticated. Benchmarks showing ties on simple lookup and real wins on complex reasoning/summarization point at that need: questions that must traverse relationships, not merely find the nearest passage.

Three shapes that justify a knowledge graph:

Multi-hop relationship questions, where the answer chains across more than one connection. “Which suppliers does the manufacturer of this recalled part also supply for other product lines?” is not solved by the single most similar chunk; it requires walking part → manufacturer → other supply edges. A vector index returns similar text; it has no hop-chaining mechanism.

Questions where relationships are a answer, not essentially supporting context. “Who on the team has worked with both payments and fraud-detection?” is person–service–person traversal. Similarity search hopes some document states the intersection outright—usually it does not, because the fact exists only as graph shape.

Root-cause and impact-chain questions in operations: “what upstream change could explain this cascade across three services?” is about causal and dependency edges, not which log line resembles “cascade of failures.” For genuinely relational operational knowledge, a graph is not a nice-to-have—it is the representation that contains the answer.

If traffic lacks those shapes—and a large share of production RAG does—GraphRAG is complexity without payoff, consistent with the numbers above.

The token-cost trap: don’t stack graph-as-data on an already-expensive setup

Cost compounds with another common mistake: multi-agent stacks. Anthropic’s writeup on their multi-agent research system reported roughly 15× tokens versus a single chat turn, and single agents with tools already about 4× plain chat. More striking: token usage alone explained about 80% of performance variance across runs; tool-call count and model choice joined it to explain about 95%. Performance gaps were mostly about how many tokens burned, not how clever the orchestration looked.

That cautions multi-agent designs generally. It becomes a concrete warning when stacking graph-as-data on top of multiple agents: GraphRAG adds token overhead on both ends—extraction/construction up front, and richer multi-hop context assembly at query time. Layer that on a setup already at ~15× simple-call tokens without measuring the delta, and cost can jump an order of magnitude while quality on the real mix stays flat—because most traffic was simple lookups graph retrieval does not help.

The remedy is unglamorous and often skipped: measure the token delta of adding graph-as-data on the actual question distribution against the actual multi-agent baseline before committing. If traffic is mostly the lookup shape where GraphRAG ties plain retrieval, extraction and query overhead bought nothing. If a meaningful share is genuinely multi-hop, the measurement shows it—and supplies a number to justify the cost instead of a hunch.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

Measurement discipline matters as much as graph choice: keep a fixed question set, score with graded verifiable answers where possible, and record token deltas beside quality deltas so cost and accuracy move on the same spreadsheet.

A staged way to actually adopt this

A better build order than “graphs first”:

Start with graph as architecture for control flow immediately, whether or not a knowledge graph is ever planned. It is cheap—a dependency and some state types—and pays off as soon as an agent has more than a couple of steps or any branching. Checkpointing and interrupts are not late-stage luxuries; they are what make agents resilient to crashes and human-review pauses. Retrofitting an explicit state machine later hurts more than starting with one.

Add graph as data only after concrete multi-hop questions where vector search has been verified—not assumed—to fail. That verification matters: run candidates against existing retrieval, see where it falls short, and check whether the gap is relationship traversal or something else (bad chunking, missing material). Many “RAG cannot find the answer” reports are the second case; a knowledge graph does not fix bad chunking.

Before standing up a graph database, benchmark against plain RAG on owned data and owned questions, using graded, verifiable comparisons in the spirit of GraphRAG-Bench rather than LLM-as-judge pairwise win rates—given how unreliable that evaluation style can be after the bias audit. If a knowledge graph does not clearly beat a well-tuned vector or keyword baseline on the questions that matter, there is not yet a case for graph-as-data, regardless of how capable the idea sounds on paper.

Graphs remain useful constantly. The change is stopping the habit of assuming which kind is needed before checking what the problem in front actually requires.