Finding Outputs Nobody Reads: A Consumer Audit for Automated Systems

Every automated system accumulates work that runs correctly and matters to no one: a report regenerated nightly that nobody opens, a validation gate that can fail but is never invoked, a rule added to a checklist that no process actually reads. None of these break anything, which is exactly why they survive. This article describes how to audit a system for output without a consumer, what such an audit typically uncovers, the three mistakes that make the audit destroy healthy components, and the single piece of metadata that makes the whole exercise almost free.

The rule behind the audit: every step has a sender and a receiver

A useful discipline for pipelines, whether they are made of cron jobs, CI steps or chained AI agent sessions, is to insist that every unit of work is connected on both sides. Something upstream hands it context and inputs; something downstream takes what it produces. A step that receives properly but hands off to nobody is not doing useful work, even if everything it does is correct.

Stating the rule is easy. Checking it is harder, because the violation leaves no trace. The audit exists to look for steps that do something sensible whose result is never picked up. It pays off most when it runs regularly, for example as part of every retrospective, rather than once after a cleanup scare.

What an honest audit tends to find

Consider a team running a heavily automated, agent-driven workflow that ran this audit on itself. The numbers were unflattering, which is precisely what made them useful.

• Gates that never fire. They found 95 checks capable of turning red that nothing ever called. Each was implemented and tested properly, and each had seemed essential on the day it was written, yet none was wired into any path.
• Rules without a caller. Of 25 recently adopted rules, 19, or 76%, had no reference from any code path, checklist or routine. The problem was not disobedience; nothing pointed at them at all. One debugging procedure stayed in the team's canonical guidance for 42 days without being applied once. Nobody had forgotten it. Writing it down had simply been mistaken for adopting it.
• A task registry full of silent items. A live count showed 417 open tasks, 146 of them with no activity whatsoever and 104 with no definition of done. The oldest untouched item was 37 days old.
• A dashboard nobody could see. A publishing-pipeline dashboard was refreshed on every release and stored in the shared location where it belonged. But the share on the producing machine was configured as receive-only, so nothing written there ever propagated. For weeks it was accurate and current, and no one else ever saw it.

The last case captures the lesson in one line: putting something in the right place is not the same as delivering it.

Why the failure always happens at the seams

Losses tend to occur at the boundaries between systems, and this is not specific to AI. Something is handed over and not accepted, or accepted and not noticed in time. In every case above, the producing side worked perfectly. The writer wrote, the counter counted, the gate was able to fire. What was missing was a named consumer.

The core difficulty is that nothing fails when a consumer is absent. Missing consumption emits no error, no alert and no log line. Producers get monitored because their failures are loud; consumers are rarely monitored because their absence is silent.

Two further cases show the cost when that silence persists:

• An alert relay buffered messages for a human operator and promised delivery on the next successful send. On one node it exited before running its first line of logic, every single time. 552 messages sat waiting for 16 days. The producer was healthy and the route was correct, but the consumer was never reached.
• A public dashboard showed 17 publications sourced from a mirror while the ledger it supposedly reflected did not exist anywhere in the fleet. The display faithfully showed what it had. Nobody had asked what it was actually reading from.

Both would have been caught by a check on the receiving end: did the operator get anything, does the dashboard's source exist.

Three ways a usage audit goes wrong

The naive version of this audit, "count how often each component is used and remove what scores zero", actively damages a system. These three corrections address the most common ways it misfires.

1. Measure the use, not the invocation

It is tempting to hook a counter to the obvious entry point, such as the tool call that loads a skill or runs a script. But components are often consumed indirectly. If automated routines simply read a skill's file rather than calling the tool that wraps it, an invocation counter reports zero for a component that is used constantly, and the audit declares it dead.

Count the effect a component has downstream rather than the entry point that triggers it. If a report is consumed, some decision, ticket or downstream file should change as a result; that change is the evidence worth counting.

2. Print the age of the data and respect the window

A component created nine days ago cannot win a thirty-day usage comparison. An audit must report how far back its data goes and must never pass judgment on anything younger than its observation window. A verdict that does not state the period it covers has no real authority, and it systematically penalizes whatever was built most recently.

3. Exempt things whose value is availability

Some artifacts exist to be ready, not to be used. A runbook for an incident that has never occurred should have zero invocations. For that class the right question is whether it still works, not how often it was called. Apply a utilization metric to it and you delete precisely the material you will need on your worst day. Disaster recovery procedures, fallback configurations and rarely used admin tools belong in the same bucket; test them periodically instead of counting calls.

The deeper trap: metrics that can only confirm

Beyond counters, there is a more fundamental problem. A metric that can only ever move in favor of the thing you built is not evidence. An audit that can never recommend turning a pipeline off is really just a status report under another name. Before trusting the result, ask what outcome would lead you to delete something, and check that such an outcome is actually reachable.

The fix: declare the consumer when the artifact is born

The most effective change is small. For every artifact, whether a gate, rule, report, dashboard or task, record one line at creation time rather than at audit time: who consumes this, and which state change proves it was consumed.

Every failure described above traces back to that question going unanswered on day one, and every retroactive fix costs more than answering it would have. A few practical ways to enforce it:

• add a required "consumer" field to templates for new tasks, rules and scheduled jobs
• reject definitions of done that describe only the producer's action ("dashboard updated") without a receiver ("dashboard visible to the on-call reviewer")
• for anything delivered over a boundary, such as a share, queue or relay, add a check on the receiving side, not just the sending side

Once that field exists, auditing no longer requires detective work. You simply query for artifacts with a blank consumer entry; that list is the result, and it could have been read off at any point.

Key takeaways

• Output without a consumer is invisible by default, because nothing fails when no one reads it.
• Failures cluster at handoffs, so verify delivery on the receiving side.
• Count state changes caused by consumption, not invocations of an entry point.
• Report the data window, never judge components younger than it, and test availability-only artifacts instead of counting their use.
• Record "who consumes this and how we know" when an artifact is created; the audit then costs almost nothing.