Home / Articles

Agents Pay Dearly for Waiting: Polling, Blocking and Tasks Compared

A benchmark of three ways an agent can wait on a slow cloud operation: polling, blocking and MCP tasks, the token cost of each, and rules for tool designers.

Publisher: REACTAPP.TOP8 min read1807 words

This article is published in English.

Sources: Your Agent Doesn’t Know How to Wait

Nearly every cloud infrastructure API contains the same pause. A create call does not return a finished resource. It returns a receipt:

{
  "name": "operations/abc-123",
  "status": "RUNNING",
  "done": false
}

The database needs minutes to come up, and in the meantime all the caller holds is a tracking ID. This is the long-running operation, or LRO, pattern, and it sits under infrastructure provisioning, model training, database migrations and video transcoding alike. For a human at a terminal it is routine: run the command, wait, get a coffee.

For an AI agent it is a serious design fork. Benchmarks comparing ways of handling the wait showed a 5x difference in cost, an 11x penalty in generated output tokens and heavy concurrency bottlenecks. Most tool authors do not realize they are picking a path when they write their tool definitions. The comparison covered three approaches on identical workloads with the same model: polling inside the agent loop, blocking on the server, and handing the wait to a harness-managed task.

Why waiting is expensive

An agent is a language model running in a loop. It receives a goal and some tools, reasons, calls a tool, inspects the result and decides what to do. Each cycle is a step, and a step is not an isolated HTTP request. At every step the whole conversation is fed back into the context window: the system prompt, every tool definition, the goal and all earlier calls and responses. Total task cost is therefore roughly the number of steps multiplied by the context size at each step.

Now picture the agent polling an operation. It calls get_operation and sees RUNNING, and deciding what to do next needs a full inference pass over everything accumulated. It calls again, sees RUNNING again, and pays for another full pass. A frontier model, priced in dollars per million tokens, is being used to implement sleep.

Four terms recur below:

  • Steps: how many tool calls the agent made. Fewer is better.
  • Billed tokens: the cost metric, which weights cached context more cheaply, at about a tenth of the price of fresh input. That is the figure a cloud bill reflects.
  • Tool surface: whatever the agent is allowed to call, either through an MCP server or by typing CLI commands into a shell.
  • Tasks: background work handed off to the harness or the MCP runtime, which keeps the agent's main thread unblocked.

How the benchmark ran

The test used Evalbench against the same cloud database control plane in three setups:

  • MCP Toolbox, a server with a prebuilt database configuration.
  • A basic MCP server exposing plain create, read, update and delete tools plus operation status.
  • A fallback CLI setup (gcloud), with no MCP tools, where the agent resorts to shell commands.

There were 12 tasks per difficulty tier, run with Claude Opus on a pinned CLI version with an automated judge doing the scoring. Among the hard tasks, two ran for a long time: cloning an instance and provisioning a high-availability one. The three setups handle waiting in three different ways.

The three waiting patterns

Polling inside the loop

The server returns an operation ID and the agent keeps asking inside its reasoning loop:

create_instance()      → operations/abc-123
get_operation(abc-123) → RUNNING
get_operation(abc-123) → RUNNING
... ×10 ...
get_operation(abc-123) → DONE

Ten full trips through the model context just to learn that a clone is still in progress.

Blocking on the server

The server provides a tool that keeps the connection open until the operation reaches a final state:

create_instance()           → operations/abc-123
wait_for_operation(abc-123) → DONE   (one call, eleven minutes later)

That is one step. The idle time is spent on a server billed at a tiny fraction of a cent per hour, and not inside a model context billed in dollars per million tokens. The catch is that blocking is strictly serial, and while the call is held open the agent's thread is frozen.

Tasks handed to the harness

Rather than blocking the agent or burning tokens on polls, the agent passes the operation to a background task runner:

Bash(cmd, run_in_background: true)  → task_id: task-8921
... agent configures network rules, audits schemas, writes migration scripts ...
TaskOutput(task-8921, block: true)  → done

Through the MCP Tasks extension, starting work and consuming its result become separate events. The agent launches the job, receives a light task handle and moves on to other useful work. When it finally needs the database, it checks the handle or waits for a completion signal. Only this pattern lets an agent wait on several operations at once or overlap a long wait with productive computation.

What the clone test cost

In the clone evaluation (clone an instance, wait until ready, confirm health), the underlying operation takes about 13 minutes of real time in every environment. Wall-clock differences were small, but token economics diverged sharply:

  • MCP Toolbox with its blocking wait did best on every metric: 4 steps, 211,854 billed tokens, 765 seconds.
  • The basic MCP server polling ten times took longest, at 864 seconds across 23 steps.
  • The fallback CLI had the heaviest overhead: 328,316 billed tokens and 14,108 output tokens over 19 steps.

The single blocking wait cut steps by 80%, billed tokens by 26% versus basic polling and cost by 35% versus the fallback CLI. It also finished first, since polling agents kept waking up for extra reasoning passes.

The most striking figure is output tokens, 11x higher on the CLI. With no MCP tools, the agent had to write its own retry script in bash:

while true; do
  st=$(cloud-cli operations list --project=production-db \
       --instance=staging-replica --format='value(status)' | head -1)
  [ "$st" = "DONE" ] && break
  sleep 20
done

It wrote that loop three times in slightly different forms, because each backgrounded attempt failed to monitor cleanly. In the end it gave up and set a wake-up timer five minutes out. An autonomous model spent 14,000 output tokens reinventing wait_for_operation from scratch, and did it badly. An agent denied a way to wait will build one, at high cost, and the result is worse than the tool that was left unshipped.

Guidance for API and tool designers

Designing for agents means rethinking REST and gRPC conventions around a constraint they never faced: every byte exposed is re-read at every step. Each practice below traces to something measured.

Offer a tiered waiting model

Different situations call for different waits, so provide all three tiers and let the agent choose:

create_instance()                       → { "operation": "operations/abc-123", "done": false }
get_operation(id)                       → Tier 1: inspect status, resume after interruption
wait_for_operation(id, timeout_seconds) → Tier 2: the 90% single-instance case
create_instance(async)                  → Tier 3: durable handle for parallel, fleet-wide work

Offering all three adds almost no overhead, yet the gap between them accounted for 55% of total cost in one clone task. Limiting the surface to the first tier forced the CLI agent to build its own retry mechanism at a price of 14,108 output tokens.

Put the operation ID in the JSON body

Classic REST places operation links in headers such as Location or Operation-Location. Modern tool-calling layers, including MCP, OpenAI Tools and LangChain, often drop HTTP headers and pass only the JSON payload to the model.

When the ID is absent from the body, the model does not fail loudly. It falls back to querying the target resource directly instead of checking the operation. That is costlier and murkier, since resource endpoints often return 404s or stale data during initialization, making active processing indistinguishable from failure. In one run this produced 221 repeated get_instance calls.

Add explicit timeouts with structured partial results

Indefinite blocking carries real risk: dropped connections, 300-second load balancer cutoffs and tasks frozen for good. The wait tool with the lowest overall cost had no timeout and blocked for sixteen minutes in one case, leaving the agent with no contingency had the call stalled.

The remedy is a timeout_seconds parameter, defaulting to around 300. At the limit, return a structured status that is not an error, instead of throwing an exception:

{
  "done": false,
  "operation": "operations/abc-123",
  "elapsed_seconds": 300,
  "status": "RUNNING",
  "metadata": { "percent_complete": 65 }
}

Now the model can look at progress and choose whether to wait again, background the work or tell the user, without triggering error-handling routines. That one field separates a blocking wait from a gamble.

Design for tasks and asynchronous harnesses

A server author implementing MCP gets the standardized form of the third pattern through the Tasks extension. Several points matter:

  • Let the server decide the execution path. Tasks are server-directed. Advertise io.modelcontextprotocol/tasks and return a CreateTaskResult instead of a normal result whenever a request will run long. Do not require callers to pass an async flag, because the client opts in during capability negotiation.
  • Return structured handle metadata: a taskId, an initial status, ttlMs and pollIntervalMs, and make sure the task is durably created before the response goes out. Harnesses can then watch progress without waking the model, and clients can recover after a crash.
  • Support completion notifications. Where the transport allows, push notifications/tasks instead of relying on client polling. Polling stays as the fallback, but notifications remove needless round trips.
  • Keep state for finished tasks and operations for at least 24 to 48 hours. If an agent's context is compacted during a wait and it later polls a completed job, it should receive the result and not a 404.

Count the schemas

Every tool mounted taxes every step of every task, including tasks that never use it. Cost is roughly steps times context size. A blocking wait improves the first factor, while schema bloat can erase the gain through the second. A server with the fewest steps can still spend the most tokens if large tool schemas sit in context at each of them. With a big tool surface, loading definitions on demand, using skills or adding tool search keeps the always-present schema small.

The takeaway

Waiting is an architectural decision that shapes the economics of an agent, and not a minor optimization. Replacing token-heavy polling with tiered, blocking or task-based waits cuts overhead, and leaner tool surfaces help agents scale without paying a frontier model to count seconds.