Escalate Nodes, Not Tasks: A Six-Level Ladder for LLM Workflow Costs

Many teams building LLM-powered pipelines start with a single architectural question for every new feature: should this run as a fixed DAG or as an autonomous agent? It sounds like a sensible design review, but answering it at the level of a whole task quietly locks in the cost of the most uncertain step for every other step. This article walks through how that happens, then presents a six-level escalation ladder in which each node starts at the cheapest viable level and climbs only when an explicit contract fails. By the end you should be able to decompose your own "agent" tasks, see how much of them is actually deterministic, and put hard bounds on what the rest may spend.

The scenario used throughout is a product that does two things: it converts everyday file formats, and it runs files-in, files-out workflows, both as one-click pipelines and through a builder that generates a pipeline from a request written in everyday language. The conversion half works reliably. The workflow half is where the DAG-or-agent question kept coming up, and where it took most of a year to see that the question itself was the problem.

The starting point: classify each task up front

The initial process for every one-click workflow was methodical. Research the use case, write a specification, and then have a person decide whether the pipeline would be implemented as a DAG or as an agent.

Open-ended tasks went to an agent

Tasks judged genuinely open-ended were implemented as a single ReAct agent on LangGraph. The reference case was job search driven by a resume. A user uploads a CV, and the system must find roles worth applying for and produce a tailored resume for each. That involves resumes written in several languages, matching candidates to roles on dimensions that keyword overlap cannot capture (commute time from home, the makeup of the team, what the job involves day to day, the salary band) and finally generating a customised document. No one can say in advance how many steps that takes, so it looked like the textbook case for an agent.

Predictable tasks went to a static DAG

Sequential, predictable work became a fixed graph. The reference case was turning a video into subtitles: extract the audio track, run speech recognition, split the transcript into timed segments, write the subtitle file. Even the optional extras can be listed ahead of time, such as cleaning up the verbatim transcript for readability or checking whether factual statements in the talk hold up. Because every decision can be made before execution begins, the whole graph can be drawn before execution begins.

The division looked principled, and it shipped. The problem was that costs never fell.

Why the agent tax never goes away

The mechanism behind the flat cost curve is mundane, which is precisely why it is easy to overlook.

When a task is labelled "agent", every step inside it runs inside the agent loop and pays for it. A ReAct loop can only choose an action if the schemas of its tools are in the context window. You can compress the conversation, summarise intermediate state and trim retrieved documents, and the team did all of those things, but the minimum cost per turn stays where it is, because that minimum is the block of tool schemas and it is resent on every turn. Nor can you simply give the agent fewer tools if you want it to stay flexible: the reason it is an agent in the first place is that nobody knows ahead of time which tools a given run will need.

One way out is to predict the relevant tool subset for each task. That is a legitimate research direction, but it requires its own investment and a model that guesses well. What the team wanted instead was something that could be specified: an architecture whose cost is bounded by how it is built, not by the accuracy of a predictor. If the prediction route interests you, progressive tool discovery for agents at scale explores it in depth.

Rereading the literature alongside months of production logs produced an almost embarrassingly simple conclusion:

Uncertainty belongs to individual nodes, not to whole tasks.

The DAG-or-agent decision was being made per task. But a task is just a collection of steps, and in nearly every task that had been filed as "agent", most steps were entirely deterministic. Making the call at the task level means every node pays the price of the most uncertain node in the graph.

Take the job-search workflow apart and the pattern is obvious:

• Extracting structured fields from a PDF resume needs no model reasoning at all.
• Identifying which language the resume uses is equally mechanical.
• Locating the home address and working out the commute is one tool invocation.
• Retrieving open positions means hitting a jobs API.
• Rating how well a given role fits a given candidate takes one model request, a fixed prompt and zero tools.
• Only something like "find out what this particular team has been shipping recently" has no predictable number of steps.

That last item is one node. The whole graph had been paying agent prices to serve it.

The escalation ladder

The fix was to stop deciding in advance. Under the new rule, every workflow begins at the cheapest level that could conceivably work, and only the individual nodes that fail move up. There are six levels.

• L0: tool calls only, no LLM. If a request can be satisfied entirely by deterministic operations, no model runs. In the product this is the existing fast path and it remains a separate surface. Cost is simply the tool calls.
• L1: static DAG of mechanical nodes. A genuine graph with branching and fan-out, but every node is ordinary code. Still no model calls.
• L2: static DAG with single-shot LLM nodes. The graph shape is unchanged, but certain nodes now invoke a model once, with a narrowly scoped context. In practice the node sees its direct inputs and nothing more: no conversation history, no global state and, crucially, no tool schemas. At this level the model behaves like a pure function rather than an agent. Cost is O(n) calls, with n known before the run starts.
• L3: bounded refinement loops. An L2 node may retry against a contract up to a fixed maximum of N attempts. The contract, not the model, decides when the output is acceptable. Cost is O(n·N), which is still known before the run.
• L4: an opaque node becomes a sub-agent. A node that cannot be specified ahead of time gets a real ReAct loop, but its tools are limited to that node and it has a hard step budget. Nobody can predict what it will spend, and that unpredictability is the reason it must have an explicit ceiling.
• L5: full replan. The plan itself was wrong, so the graph is rebuilt. This is the most expensive move available and should be rare.

Contracts, scope and budgets

A taxonomy alone would just be nicer vocabulary. Three mechanisms make the ladder actually work:

• Contracts trigger escalation. Every node states the shape of an acceptable result, and only a failed check against that statement justifies moving up a level.
• Scope keeps L4 affordable. A sub-agent's tool schemas live inside that one node's context and appear nowhere else in the run, so the schema tax is paid locally.
• Budgets cap the top of the ladder. Every level above L2 has a maximum number of attempts or steps, after which the node either gives up or escalates further.

A practical way to think about it: the contract answers "is this good enough?", the scope answers "what may this node see and call?", and the budget answers "how much may it spend trying?". If any of the three is missing, the corresponding level stops being predictable. For more on keeping loops like L3 and L4 under control in code, see bounded agentic loops in TypeScript.

Walking the subtitle pipeline up the ladder

Most runs never leave L1. The pipeline extracts audio, runs speech recognition, segments by timestamp and writes an SRT file, all without any model call. The output is then checked against its contract: cues must not overlap, no line may exceed the character limit and reading speed must stay below a threshold. A typical talking-head video with clean audio passes, and the job costs no more than ffmpeg plus one recognition pass.

When a particular cue violates the line-length or readability rule, only that cue moves to L2. It gets a single model call whose context is the cue and its immediate neighbours. The full transcript, the video metadata and every tool schema stay out of the prompt, and the rest of the graph is unaffected.

L3 is justified by terminology. In a technical talk that comes with a glossary, terminology checks can keep failing after a single corrective call, so that node may refine its output, exactly twice, with the glossary check deciding when the result is acceptable.

L4 appears in only one place: verifying that the claims made in the talk are correct. That requires searching, and nobody can say how many searches. So that single node becomes a sub-agent whose tools are limited to search and fetch and whose steps are capped. The subtitle pipeline surrounding it remains mechanical.

L5 handles the case where the plan was wrong from the outset. Suppose the file turns out to be a screencast in which the meaning lives in on-screen text and the audio is incidental. No amount of refining individual nodes will rescue a plan built around speech recognition, so the graph is rebuilt around OCR instead.

Walking job search up the ladder

This is the example that changed the team's thinking, because it seemed so obviously shaped like an agent.

L1 covers more ground than expected: parsing the resume, detecting its language, geocoding and computing the commute, and fetching listings. All of it is plain code.

L2 handles matching. For each candidate job there is one scoped model call that returns a structured score across the relevant dimensions, with the resume summary and that single job description as its entire context. That is O(n) calls to an inexpensive model with no tool schemas at all, and it replaced an agent that had been reasoning through the same list with the full toolset reloaded on every turn.

L3 handles rewriting the resume, and here contracts shift from being a cost control to being a safety control. The contract requires every statement in the rewritten resume to be traceable to the original, forbids inventing any employer or date, and caps the length. If a draft breaks those rules, it is refined, twice at most, and then the loop stops. This is a useful pattern beyond cost: a contract that checks provenance is a cheap, deterministic guard against a model embellishing someone's career history.

L4 is a single node: researching the specific company's team and its recent direction. It is open-ended by nature but bounded by construction.

L5 triggers when the categories themselves do not fit. Imagine a physicist applying for quantitative finance roles: the parsed job categories map poorly onto the candidate's real background, and the matching plan needs to be rebuilt rather than tweaked.

The headline result is that the task once classified as an agent turned out to be roughly 80% L1 and L2 work. That is not a matter of tuning a prompt; it puts the workflow on a different cost curve altogether.

What the ladder buys a files-to-files product

Files-in, files-out workflows overlap heavily. The first 80% of almost any two pipelines looks nearly the same. Yet the quality that users notice lives entirely in the remaining 20%, and that part differs every time. This is the customisation trap: either engineers hand-build the details for each task and the product never scales, or the details are skipped and the result is mediocre.

The ladder is a way out of that trap. Customisation still happens, but it happens through escalation decisions driven by contracts rather than through engineering hours. The outcome is per-task tuning without per-task human effort.

An obvious comparison is with the general-purpose agents offered by Anthropic and OpenAI, which likely do something structurally similar. Those systems are closed, so this is inference rather than knowledge, but a plausible guess is that much of their effort goes into inducing consistent structure across tasks, and they have far more data to induce it from. The ladder gets comparable structure out of how the workflow is shaped rather than out of massive datasets, making it an affordable version of the same idea.

When the ladder is not worth it

The approach assumes that you can write meaningful contracts. If a node's output quality can only be judged by a human, escalation has no reliable trigger, and the ladder degenerates into guesswork. It also adds orchestration machinery; for a pipeline with two or three steps and low volume, a single well-scoped model call may be simpler and cheap enough.

Shipping the bottom rung first

When every workflow takes files in and returns files, the file-handling layer sits beneath everything else. Every rung described above eventually reduces to something mechanical: open the container, extract the text, keep the tables intact. Nothing in that layer is uncertain, so paying for inference there would be pure waste, and its predictability also made it the obvious first component to release.

It was released as an open-source Python SDK, published on PyPI under the Apache-2.0 license and usable as an MCP server inside Claude Code. Installation is a single command with either uv or pip.

uv add convilyn          # or: pip install convilyn

The SDK converts documents to Markdown, converts images between 26 formats and reorders PDF pages, all locally, with no account and no network access. When a job truly requires a model to read something, such as a scanned page, a photo or an audio file, it goes to a hosted cloud service, and that is where usage starts to be billed.

That division is the ladder expressed as product design rather than internal architecture. The free local path is L0: when plain deterministic code can finish a job, no model gets loaded, and work only moves to a paid path after the inexpensive route has demonstrably failed. According to the project, the offline converter needs no account, key or quota and sends no telemetry; check the repository's current README for installation details and the exact feature set, since both are likely to evolve.

Current status of the workflow engine

At the time of writing, the workflow and builder features were still stabilising while the rebuild described here was being completed underneath them, with the team expecting to finish within roughly a month. The reasoning is worth copying: it is better to hold back a workflow engine that is known to be on its way out than to promote it and migrate users twice.

Prior art and related work

None of the individual rungs is new, and each has been described before. What does not appear to have been published is this specific combination: escalation per node triggered by contracts, tool scope used as the cost lever, recursive expansion permitted but budgeted, applied to a files-to-files workload. The following work covers the pieces.

Start simple and add complexity only when needed

• Building effective agents, published by Anthropic in December 2024, distinguishes workflows from agents and recommends the simplest workable solution, adding complexity only when it is needed. The ladder differs by using that principle while the pipeline executes, node by node, instead of once per task during design.

Escalate only the component that failed

• The ADaPT paper, whose name stands for as-needed decomposition and planning (Findings of NAACL 2024), begins with a high-level plan and breaks a sub-task down further, recursively, only after execution of that sub-task fails, leaving successful parts alone. It is the closest published description of the L4 trigger, although it is not framed in terms of cost.

Contracts, bounded recovery and escalation rules

• A scheduler-theoretic framework for executing LLM agents as structured graphs, posted to arXiv in April 2026, uses a static DAG with output contracts per node and a three-stage recovery protocol of retry, local patch and full replan, with explicit escalation invariants. Its position is that model-backed nodes should be validated against contracts, since type checks are not enough, and that retrying non-idempotent steps requires bounded budgets. It intentionally leaves out recursive sub-graph expansion, which is where L4 sits.
• Task-Decoupled Planning (TDP), surveyed in the same paper, splits work into a graph of sub-goals, each with its own narrow context, and limits any replanning to whichever sub-task is currently active. It reports token reductions of up to 82%, the closest published measurement supporting the L2 scoping argument.

Tool scope as the main cost lever

• Skillflow defines a DAG in YAML that the engine, not the model, walks. Its I/O is gated by capability: a step receives only the context it asks for, and any tool not covered by its contract simply does not appear in its schema. Its stated conclusion, that small role-specific context is what lets cheap models suffice, matches the L2 argument.

Staged ladders already in production

• PraisonAI ships an escalation feature in its agents SDK that defines four progressive stages: a direct answer with no tools or planning, heuristic tool use without an extra model call, one constrained model call, and finally a fully autonomous loop with tools, sub-agents and verification. That is roughly L0 to L4 compressed into four steps.
• NVIDIA NeMo Switchyard's escalation router applies the same shape to model choice instead of architecture: begin with a weaker model and switch to a stronger one when a judge detects persistent trouble.

The same shape elsewhere

• Agentic Design Patterns (systemdesign.one, April 2026) uses the phrase "escalation ladder" for the workflow-versus-agent choice and recommends defaulting to the least elaborate arrangement that gets the job done.
• Vercel's guide to AI agent evaluation frameworks for production (July 2026) applies cheapest-first ordering to evaluation: run the least expensive check capable of catching a given failure, and escalate only when that check structurally cannot see the problem.

Key takeaways

• Deciding "DAG or agent" per task makes every step pay for the single most uncertain step.
• The irreducible cost of an agent loop is the tool-schema block resent on every turn, which history compression cannot remove.
• Start every node at the cheapest level and escalate only on a failed, explicit contract.
• Scope tool schemas to the node that needs them and give every level above L2 a hard budget.
• Expect most "agent" workloads to be largely deterministic once decomposed; in the job-search case, about 80% was L1 and L2.