Fewer Tokens Isn't Cheaper: Why Coding Agents Waste Work, Not Tokens

Since coding agents took off, developers have fixated on one figure: tokens. How many did the model generate, how many did a tool return, how big is the context, how many did the agent burn through? The reasoning feels airtight. Fewer tokens should mean a smaller bill.

GitHub's analysis of the Copilot coding agent shows why that assumption is risky. Trimming each tool response can make the whole task more expensive if the trimmed text held something important. The agent then reruns commands, digs the original output back out, spends extra reasoning turns, and drags a bigger context through everything that follows.

So the cheapest interaction isn't automatically the one with the fewest tokens. What you actually want is the finished software task with the least wasted work, and that changes how you should judge agent performance.

The local optimization trap

Say an agent runs a test command:

npm test

It prints 50,000 tokens. That looks wasteful, so you write a wrapper that squeezes the output down to 2,000. On paper that's a win: 48,000 tokens gone.

Now suppose those 2,000 tokens leave out what the agent needed. It answers with something like:

I need more information about the failing test.

So it reruns the command, searches the saved output, opens the log file, or tries a narrower command. The flow becomes this:

Original command
      ↓
Compressed output
      ↓
Missing information
      ↓
Agent searches again
      ↓
Command runs again
      ↓
More context
      ↓
More reasoning

The single step got cheaper while the task got pricier. GitHub ran into this while testing ways to cut Copilot's output. One experiment used a tool named RTK to shorten shell output, and agents frequently reopened or reran commands whenever the missing detail mattered. The result was more turns and, on average, more tokens and slower completion. It sounds obvious in hindsight, and it's easy to miss when you're optimizing the wrong number.

Judge the task, not the call

An agent exists to finish a software task, not to minimize one API request's cost. The gap between those goals is bigger than it sounds. Compare two agents. Agent A:

Task
↓
8 tool calls
↓
100K tokens
↓
Completed in 4 minutes

and Agent B:

Task
↓
15 tool calls
↓
80K tokens
↓
Completed in 9 minutes

Looking only at tokens, B wins. Run an engineering organization, though, and the answer is murkier. A finished sooner, probably needed fewer retries, may have used less compute, may have produced a better patch, and kept developer waiting time down. If someone had to babysit B for another five minutes, B's real price climbs further.

That's why GitHub argues that per-interaction token counts say little about agent efficiency. The thing to optimize is the outcome of the whole task.

An end-to-end cost model

Experienced engineers already know this from other work. Cut a database query from 200 ms to 100 ms and it sounds like a clear gain. But if the new query makes another service fire three extra requests, the system can end up slower overall. You tune the system, not a single operation.

Agents follow the same rule. A useful mental model of total cost:

Total agent cost =
model inference
+ tool execution
+ retries
+ repeated work
+ context growth
+ developer waiting
+ infrastructure

And a pattern that shows up often:

smaller response
+
missing context
=
more recovery work

This is why AI engineering looks more and more like systems engineering.

Not all output is worth the same

A notable finding is that command output varies in information density. Repetitive install, build, test and progress logs can be packed with noise. Source-like output and arbitrary command results, in contrast, may contain exactly what the agent needs. Rather than one compression rule for everything, GitHub built a selective approach. In concept it works like this:

Command output
      │
      ▼
┌──────────────────────┐
│ What kind of output? │
└──────────┬───────────┘
           │
     ┌─────┼─────┐
     │     │     │
     ▼     ▼     ▼
 Source  Search  Noise
 output  results logs
     │     │     │
     ▼     ▼     ▼
  Keep   Reorganize Compress
  all     without   selectively
          losing
          content

That beats the blunt alternative:

if output.length > X:
    truncate()

The crude rule optimizes a number. The selective one optimizes the agent's ability to keep working.

What git diff teaches

Picture an agent that has edited five files and now runs:

git diff

The output is long. A naive compressor would say it's big and shrink it. But the diff may hold exactly what the agent needs to judge whether its changes are right. GitHub reports that early versions of its compressor did squeeze git diff, and benchmarks showed agents reopening the original output to get the information back. So that filter was dropped.

It's a great AI-specific lesson: you can't call something useless just because it's big. Some large outputs are high-value context, and some tiny ones are worthless. Token count can't tell them apart.

Designing your own tools

If you're building a coding agent, you might write this:

const result = await exec(command);

followed by a hard cap:

return result.stdout.slice(0, 5000);

It looks reasonable, but it quietly assumes the first 5,000 characters contain everything the agent needs. A smarter design classifies the output first. Start the same way:

const result = await exec(command);

Then branch on the kind of output:

const outputType = classifyOutput(result);switch (outputType) {
  case "source":
    return result.stdout;  case "search":
    return reorganize(result.stdout);  case "repetitive-log":
    return compress(result.stdout);  default:
    return result.stdout;
}

Even that needs an escape route, so the agent can get the original back:

return {
  summary,
  compressedOutput,
  fullOutputAvailable: true
};

The agent gets a compact view, and nothing has actually disappeared. For autonomous systems that's a much better design.

Compression should delay, not delete

Compression shouldn't mean deleting information. It should mean postponing it until it's needed. Compare the two approaches. Bad compression looks like this:

Original output
       ↓
Throw away 90%
       ↓
Agent cannot recover it

Better compression keeps the way back open:

Original output
       ↓
Create useful summary
       ↓
Agent continues
       ↓
Needs more detail?
       ↓
Retrieve original

That's closer to how caching works in ordinary software. The agent doesn't need everything right away, but it shouldn't lose access to anything important. GitHub's shipped approach gives a direct recovery path once output has been compressed.

Sometimes more context is cheaper

This sounds backwards. Developers have spent years shrinking context. But more context can prevent more work. Say the agent sees a compact failure:

Test failed:
Expected 401
Received 500

The full output might also have shown:

Database connection refused
JWT middleware initialized correctly
Request reached /api/auth/login
PostgreSQL timeout after 5 seconds

The short version tells the agent that something failed. The complete one says the authentication logic probably isn't at fault and the database connection is. The second costs more tokens and may save five tool calls. Context has a cost, and so does missing context. The engineering challenge is balancing the two.

Why agents seem to go in circles

You've probably watched an agent do something like this:

Run command
↓
Read partial output
↓
Run another command
↓
Search same file
↓
Run original command again
↓
Read more output
↓
Change code
↓
Run test
↓
Go back to original command

It looks like confusion, and sometimes it is. But sometimes the infrastructure just never gave the model enough information, and the agent is compensating for an incomplete interface. If an agent keeps asking for the same thing, the fault may lie in tool design instead of the model.

Tool design is UX for models

Developers usually judge APIs by inputs, outputs, latency, errors and authentication. Tools for AI need one more question: how does this output affect the model's next decision? Take two responses. API A:

{
  "status": "failed"
}

and API B:

{
  "status": "failed",
  "error": "PostgreSQL connection timeout",
  "service": "auth-api",
  "retryable": true,
  "attempt": 2,
  "duration_ms": 5040
}

B contains more, yet it can lower total system cost, because the agent can decide correctly on the spot. The target isn't the smallest possible JSON. It's the information that lets the agent make the next right decision.

Tuning the whole loop

GitHub isn't treating this as a single compression problem. Its write-up lists several harness changes: keep useful context while cutting repetition, drop formatting that doesn't help the task, shorten instructions without changing behavior, and avoid needless retrieval steps for background work. Each change was evaluated offline on agentic coding benchmarks and then validated in controlled online experiments before shipping.

That's performance engineering for agents. The question shifts from how to make a response shorter to how to help the agent finish with less wasted work, and it's a far better question.

Observability for agents

This changes what you track in production. For ordinary services you'd watch:

Latency
Error rate
CPU
Memory
Throughput

For agents, add more:

Task completion rate
Tool-call count
Retry rate
Repeated-command rate
Context growth
Model escalation rate
Time to completion
Cost per completed task
Human intervention rate

Most of those are workflow metrics, not token metrics. Tokens still matter, but they feed the economics rather than define efficiency.

Cost per successful task

The simplest change available is to stop tracking this:

cost per request

and track this instead:

cost per successfully completed task

They aren't equivalent. Consider an agent with these numbers:

Agent A
$0.20/task
70% success rate

compared with this one:

Agent B
$0.35/task
95% success rate

The cheaper request isn't necessarily the cheaper system. Normalize it:

Effective cost
= inference cost / successful tasks

For A:

$0.20 / 0.70
≈ $0.286

For B:

$0.35 / 0.95
≈ $0.368

A still wins on this simple metric. But if A needs far more developer supervision, the economics shift again, which is why teams eventually need a broader definition of task cost.

The same idea applies to picking models

This links to the move toward model routing. GitHub's HydraFusion research preview already treats coding workflows as an optimization problem, picking per task between a single model, escalation to stronger ones, or a second model acting as independent critic. Offline tests under controlled conditions reportedly showed big estimated savings, with quality on some benchmarks equal to or above the Opus 5 baseline they used.

Put that together with the output lesson. A smart agent doesn't merely ask which model is cheapest. It asks which workflow finishes this task with the least waste. That workflow might combine:

Cheap model
+
Good context
+
Efficient tools
+
Verification

and beat this one:

Expensive model
+
Poor context
+
Repeated tool calls

The intelligence no longer lives only in the model. It's spread across the whole system.

If you're building AI products

It's tempting to optimize the obvious numbers first: lower token use, smaller prompts, shorter outputs, cheaper models. Don't tune those alone. Instrument the entire workflow:

User request
    ↓
Model inference
    ↓
Tool call
    ↓
Tool response
    ↓
Model reasoning
    ↓
Another tool call
    ↓
Verification
    ↓
Final result

Measure every stage, then ask where the agent does unnecessary work. Maybe the database tool returns too little context, or the search tool returns too much irrelevant material. Maybe the agent retries needlessly, or your prompt forces redundant planning. Maybe a stronger model would solve it in half the turns, or a cheaper one is plenty for a first attempt. These are software engineering problems.

The future isn't fewer tokens

It's tempting to compress the whole cost problem into:

tokens ↓
cost ↓

The real relationship looks more like this:

Useful context
      ↓
Better decisions
      ↓
Fewer retries
      ↓
Fewer unnecessary tool calls
      ↓
Shorter task duration
      ↓
Lower total cost

Sometimes the road to lower cost begins by giving the model more information. That's the counterintuitive part, and it's why AI engineering is turning into a systems discipline rather than an exercise in writing prompts.

Measure completed work

When you evaluate an agent, don't ask only how many tokens it used. Ask how much useful software it produced, and how much work it wasted getting there. That means tracking correctness, latency, tool calls, retries, context size, inference cost, human intervention and successful completion.

The winning coding system won't necessarily emit the fewest tokens. Expect it to be whichever setup converts each token, tool call and model choice into real progress. GitHub's recent engineering shows the direction: optimize the whole task, keep the information the agent needs, and cut work only when you know it doesn't help. The point isn't to make agents do less. It's to make them waste less.