Loops vs. Graphs: Two Ways to Architect AI Agents

Two buzzwords, one architecture question

Anyone following AI agents that search the web, call tools, write code or run workflows on their own has probably met two terms lately: loop engineering and graph engineering. They sound like marketing, but they point at a real structural decision.

Consider an agent that burns 40,000 tokens and six minutes of real time on a question that three steps should have settled. Nothing is wrong with the model or the prompt. The design is at fault, because the agent can't tell that it has already attempted something. It simply cycles think, act, observe, with no record of what its own process looks like.

That kind of failure pushes agent design out of the prompting category and into software architecture. Once you look at it that way, nearly every shipped agent system rests on one of two patterns: loops and graphs. What follows covers what each one is, how it's built, where it breaks and why the move from one to the other changes how engineers approach these systems, beyond a swap of tooling.

What an agent is actually doing

At any given moment an agent does one of three things. It thinks (an LLM call), it acts (a tool call), or it decides what comes next. Agent engineering is largely the work of wiring those three together.

For the first couple of years of LLM agents, almost everybody wired them the same way, as a loop. It was the obvious choice. ReAct-style agents worked like that from the beginning, and it still matches how Anthropic describes the simplest agent: a model using tools autonomously in a loop.

Later, agents got longer and messier assignments: several specialized sub-agents, retries, human sign-offs and branching business rules. A lone loop began to buckle under that. Teams wanted to look at the shape of the process rather than trust that the loop would converge in the end, and that need gave rise to graph engineering, which spells the process out as nodes and edges and doesn't leave it implicit in repeated iterations.

Neither pattern is wrong. They answer different problems, so each deserves a proper look.

An analogy from the kitchen

Imagine teaching someone to cook. Loop engineering is a single instruction repeated: taste the soup, add salt if it needs it, taste again, and keep going until it's right. It's one cycle of act, check, repeat if necessary. Simple, predictable and easy to picture.

Graph engineering resembles a full recipe with branches. Chop the vegetables while someone else boils water. When both are ready, combine them. If the taste-tester wants more spice, send the pot back to the seasoning step, and otherwise plate it up. There's no single line of steps here. Several paths exist, some steps run at once, some depend on a decision, and only certain ones loop back when something is off.

What loop engineering means

A loop is the original and simplest way to build an agent. It runs in one continuous cycle:

1. Take in the input, meaning the user's question or goal.
2. Think about what to do next, which is the model's reasoning step.
3. Act, which normally means calling a tool, running code or looking something up.
4. Observe what the action produced.
5. Return to step 2 and keep going until the agent judges the goal reached.

People often call this a think, act, observe cycle, and many of the earliest agents were built on it. Put as a practice, loop engineering treats an agent as a single repeating cycle: the model reasons, selects an action, runs it, studies the result and writes that result back into its own context, then goes around again until it concludes the job is finished or hits a cap.

A few points explain its popularity:

• It underlies the well-known ReAct pattern (reasoning combined with acting) from Yao and colleagues, where reasoning traces are interleaved with actions and a model doesn't have to finish all its thinking beforehand.
• Most agent frameworks define an "agent" this way too, as one control loop with a shared context window and a tool-calling interface.
• Writing one takes only a few lines. The state is the conversation to date, and there's a single exit condition. A coding assistant that edits a file or a research agent that searches until it's satisfied needs nothing more elaborate.
• Anthropic's guidance says as much: begin with the simplest approach that works, and add structure only when the task demands it.

The catch is that everything the loop must reason about, including what it already tried, what failed and what the goal was, has to sit inside one ever-growing context window. The process has no separate memory. All it keeps is a transcript.

How the loop is put together

The flow runs end to end like this.

Stepping through it:

• A user task starts the loop with a goal.
• In the thinking step, the model reads the system prompt, the conversation history and the latest observation, then decides what happens next.
• The "is the task complete?" check isn't a separate system. The model itself decides, based on its context, whether it's finished.
• If it's not finished, the model picks a tool and calls it.
• The tool's result or error returns as an observation.
• That observation gets appended to the context, and the cycle restarts from thinking with a slightly longer history each pass.
• When the model considers the work done, the loop ends and hands back the final answer.

A compact Python version shows how little code this needs.

def run_agent(task: str, tools: list, max_iterations: int = 10) -> str:
    context = [{"role": "user", "content": task}]

    for step in range(max_iterations):
        response = llm.generate(context, tools=tools)

        if response.is_final_answer:
            return response.content

        tool_result = execute_tool(response.tool_call)
        context.append({"role": "assistant", "content": response.content})
        context.append({"role": "tool", "content": tool_result})

    return "Stopped: max iterations reached"

What graph engineering means

Graph engineering represents the agent's process explicitly. It's a set of nodes, each a unit of work such as an LLM call, a tool call or a piece of business logic, joined by edges that are either fixed or conditional. All nodes read from and write to one shared, typed state object. In short, the workflow becomes a map of connected steps with paths between them, replacing a single repeating cycle.

A node can be any of these:

• an agent, a specialized AI with one job such as research, writing or analysis
• a tool call, for example a web search, a calculation or a database query
• a decision point that chooses the next path
• a merge point that combines results from several paths

Several paths may run in parallel, and others are conditional and followed only if a test passes. Loops remain possible, but they appear only where they're needed, as a controlled edge that returns work a few steps and doesn't blindly redo everything.

Why graphs are attractive

• Parallelism. Independent tasks can run side by side and don't wait in a queue.
• Branching. The workflow can go in different directions depending on the situation, and doesn't just repeat the same steps.
• Specialization. Each node can be its own expert agent with a single focus, which is closer to the way a human team splits work.
• Targeted retries. When a step such as a quality check fails, only that step sends work back to be redone, and everything else is untouched.
• Visibility. Since the workflow is a map, it's easier to see where something went wrong and which path was taken.

LangGraph is a framework built on this idea, modeling workflows as stateful, cyclic directed graphs. A graph can still loop, since a node may have an edge back to an earlier one, so retries and iteration survive. What changes is that the loop becomes one edge among many, drawn on a map you can look at, and stops being the whole architecture.

This lines up closely with a distinction Anthropic draws between workflows, where LLMs and tools follow predefined code paths that developers control, and agents, where the model directs its own process dynamically. A graph can express both in one structure, with fixed paths where predictability matters and model-directed branches where flexibility pays off, and the difference shows on the diagram.

How the graph is put together

Take the same kind of problem, an agent that must research, write code and review its own output, and rebuild it as a graph.

Stepping through this one:

• START is the entry point into the graph.
• A router node reads the current state and picks a path. That can be a plain if/else, or an LLM call that classifies the request.
• Three specialized nodes, a research agent, a code agent and a review agent, are each self-contained: they read state, do their job and write results back.
• An aggregator or merge node folds whatever the executed branch produced into one consistent state object.
• A retry edge handles review failures. When the review agent's validation fails, an explicit edge routes back to the router, and the same model isn't simply re-prompted in place. The path is visible on the graph and not buried in a prompt line like "if this fails, try again."
• END is reached once the state passes validation.

Here's simplified pseudocode in the style most graph frameworks settle on.

from graph_engine import StateGraph, START, END

def router(state: dict) -> str:
    return state["task_type"]  # "research" | "code" | "review"

def research_agent(state: dict) -> dict:
    state["results"]["research"] = do_research(state["query"])
    return state

def code_agent(state: dict) -> dict:
    state["results"]["code"] = write_code(state["query"])
    return state

def review_agent(state: dict) -> dict:
    state["valid"] = validate(state["results"])
    return state

def aggregator(state: dict) -> dict:
    state["final_output"] = merge(state["results"])
    return state

graph = StateGraph(schema=AgentState)
graph.add_node("router", router)
graph.add_node("research_agent", research_agent)
graph.add_node("code_agent", code_agent)
graph.add_node("review_agent", review_agent)
graph.add_node("aggregator", aggregator)

graph.add_conditional_edges("router", {
    "research": "research_agent",
    "code": "code_agent",
    "review": "review_agent",
})
graph.add_edge(["research_agent", "code_agent", "review_agent"], "aggregator")
graph.add_conditional_edges("aggregator", {
    "valid": END,
    "invalid": "router",   # explicit retry edge
})

app = graph.compile()
app.invoke({"query": "...", "results": {}, "task_type": "code"})

Putting the two patterns next to each other makes the contrast easier to see.

Why the industry is moving from loops to graphs

The shift wasn't sudden. Loops ran into real limits as people tried to build more ambitious systems, for several reasons.

Real work doesn't run in a straight line. Research, customer support, financial analysis and content pipelines all branch. If the question is about billing, go one way, and if it's a technical problem, go another. A loop squeezes all of that into one long chain of conditionals, which gets messy fast.

Multi-agent systems needed a better shape. Once a system had several agents, say a researcher, a summarizer and a fact-checker, one loop couldn't express that two agents should work at the same time or that one should run only if a check fails. A graph captures this naturally.

Visibility and trust matter more over time. Once agents handled high-stakes work, teams needed a step-by-step view of what occurred. A graph can be drawn and pointed to, showing exactly where a run went well or badly, while a loop is an opaque block that executed identical code twenty times.

Cost and speed. Every wasted iteration burns time and often real money, since each thinking step is a language model call. Running independent steps in parallel and looping back only the part that needs redoing is plain more efficient.

Testing and maintenance get simpler. You can build, test and change a graph node by node. A tightly wound loop is usually all-or-nothing logic, and changing it safely after launch is harder.

Choosing between them

A rough decision framework follows.

Reach for a loop when:

• the job has one purpose: a single agent, one goal and one set of tools
• an early prototype needs checking before any architecture money is spent
• the step count is low and fairly predictable
• one person owns the whole thing and nothing has to be handed to another team

Reach for a graph when:

• two or more specialized agents or roles are involved
• retries, fallbacks or human approvals need to be auditable
• parts of the process can run side by side and later have to merge
• execution has to be traced, checkpointed or resumed from a given point, which a replayed transcript can't offer
• separate people or teams own separate segments, with the graph serving as the agreement between them

Here's a useful gut check. If a loop's code contains a comment like "at this point we should already have tried X," that's a hint you're modeling a graph inside a loop, badly, and it's probably time to draw the graph.

It isn't truly an either/or call. It's about fitting the tool to the task. A basic chatbot answering questions one at a time, or a small script repeating one clear action until done, suits a loop. A task with several branching steps, multiple collaborating specialist agents, parallelizable work or a need for clear visibility into what happened and why suits a graph. A sensible default is to start with a loop and move to a graph the moment the if-this-then-that logic gets hard to follow.

A production checklist

Before shipping either pattern, these habits pay off:

• Cap the steps. Both patterns need a circuit breaker, since an agent with no limit has no ceiling on the bill.
• Make "done" testable. The exit condition should be something a test can check, never a hunch the model forms.
• Instrument every step. Give each LLM call and each tool call its own OpenTelemetry span or similar, and avoid a single span for the whole run.
• Version the graph. Keep definitions in version control like any code, because a quiet edge change quietly changes behavior.
• Validate state at node boundaries. Check typed schemas on state entering and leaving every node so malformed data is stopped early.
• Log decisions. Capture why the model took a path along with what it did, since that's what post-mortems rely on.
• Exercise the retry path. Trigger a validation failure on purpose and confirm the retry edge or loop pass does what you expect, including how latency and spend change.
• Measure cost and time per piece of work. Count per node in a graph and per pass in a loop to spot where a slow or pricey run spends its effort.

What it comes down to

A loop is the minimal design, with one context, one cycle and the model steering control flow. It fits narrow, single-purpose agents. A graph spells the process out through nodes, typed state and edges that you can draw, branch, run in parallel and retry deliberately.

The deeper change isn't that graphs are better. It's a move from procedural thinking, asking what comes next, to declarative thinking, asking what shape the system has. Graphs pair well with the tracing and observability discipline production systems need, while loops pair well with fast iteration.

Most production systems end up as graphs built from loops, essentially state machines whose nodes are small ReAct-style cycles. That makes the two patterns different zoom levels of one architecture and not competitors. For any current agent project, a fair question is whether everything still runs as one long loop or whether it's time to sketch it as a graph.