Agent Maturity Model: Human in the Loop, Harness Engineering, Orchestration

An agent maturity model is easy to draw and hard to live in. On paper it's clean: a ladder that starts with simple tool-callers and ends with self-improving production systems. Frameworks always look tidy before anyone tries to run one under load. Once real code is behind them, the same model reads differently. It wasn't wrong, but knowing something in theory and knowing it in production are different kinds of knowledge. A map can't tell you how the road feels at three in the morning when a circuit breaker won't close and the on-call rotation is you.

This is a look back at that model after the layers behind it have actually been built: how the levels line up with the engineering work, which beliefs didn't survive contact with production, what a stacked reference architecture looks like, and what remains unsolved. Code is minimal here. The aim is to step back and see the whole stack.

Five levels, each one resting on the last

The maturity model describes five levels of agent capability. Each level is more than a stronger version of the one below, because it also depends on everything beneath it. You can't skip steps. An agent can't be reliably autonomous without evaluation infrastructure, and it can't improve itself without memory. The levels compound.

Here is how the engineering topics line up against them:

+---------+---------------------+------------------------------------------+
| Level   | Capability          | Articles That Build It                   |
+---------+---------------------+------------------------------------------+
| Level 1 | Reactive            | Article 4: Protocols (MCP, A2A)          |
|         | Responds to inputs, | Basic tool use, standard interfaces      |
|         | calls tools         |                                          |
+---------+---------------------+------------------------------------------+
| Level 2 | Assisted            | Article 5: Harness Engineering           |
|         | Shaped behavior,    | Article 13: Human-in-the-Loop            |
|         | human oversight     |                                          |
+---------+---------------------+------------------------------------------+
| Level 3 | Supervised          | Article 6: Multi-Agent Orchestration     |
|         | Coordinated,        | Article 8: Evaluation Pipelines          |
|         | evaluated, secured  | Article 9: Security and Red-Teaming      |
+---------+---------------------+------------------------------------------+
| Level 4 | Autonomous          | Article 7: Memory Architectures          |
|         | Operates            | Article 10: Cost Optimization            |
|         | independently at    | Article 11: Deployment Patterns          |
|         | scale               | Article 14: Goal-Directed Loops          |
+---------+---------------------+------------------------------------------+
| Level 5 | Self-Improving      | Articles 7 + 8 combined                  |
|         | Learns from its own | Article 12: The frontier (not yet here)  |
|         | operation           |                                          |
+---------+---------------------+------------------------------------------+

The shape matters more than the mapping. Level 5, self-improvement, has no dedicated component, because it isn't something you build directly. It emerges when memory and evaluation work together, and even then the field isn't fully there yet. The top of the model is still partly aspirational. Self-improvement can be approximated today but not fully achieved.

That wasn't obvious when the model was first sketched. Level 5 looked like a destination you reached by assembling the right parts. In practice it behaves more like a horizon that recedes as you approach it.

Beliefs that production corrected, from human in the loop to orchestration

This is the part a framework document can never capture, because it only comes from building. Several early assumptions turned out to be wrong or at least incomplete.

Memory was badly underrated. In the first sketch, memory was a bullet point, a feature that makes agents feel more continuous. Building it showed that memory is infrastructure, with everything that word implies: schema design, retrieval quality, decay policies and tenant isolation. A more uncomfortable lesson followed. Even the infrastructure we know how to build isn't quite the right abstraction. Teams adapt databases designed for other jobs because nothing purpose-built exists. Memory was the topic most underestimated in the original model.

The goal is predictability, and autonomy is secondary. This is the deepest shift. Early on, progress was measured by how much an agent could do without a person, so every added bit of independence looked like growth. Yet an agent that acts alone but unpredictably simply does harm sooner. Reliability under supervision turned out to count for more than raw independence in every serious deployment. One illustrative failure involved an agent that autonomously created dozens of duplicate tickets. What production systems need is the right amount of autonomy: bounded, auditable and predictable, with humans owning the decisions whose cost deserves human attention. A reader once argued that orchestration should aim for predictability, not autonomy, and that point took a long time to concede.

Security is a property of every layer. The original plan treated security as a topic near the end, something bolted on once the system worked. Prompt injection corrects that. Its targets are elsewhere: the point where tools get called, the route by which memories are written, the way goals get revised, and the traffic between agents. Every component has a security dimension. Authentication at the tool boundary, isolation in memory and controls on goal revision are all security, spread through the system and not parked in one box.

The loop outranks the prompt. This lesson came last and would have been more useful first. Much of the field's energy goes into prompt engineering and the search for magic wording. But an agent isn't a prompt. It's a loop that pursues a goal across many steps, and a perfect prompt inside a broken loop gives you an eloquent agent that never arrives. Once you think in loops, many problems that looked like model limitations turn out to be loop design failures.

A human in the loop isn't a flaw to engineer away. It's tempting to assume human involvement is a temporary crutch to remove as the technology matures. The best production agent systems aren't the most autonomous ones. They place human judgement exactly where it's needed and nowhere else. Human oversight isn't a stage you graduate from. It's a deliberate, lasting feature of any system whose errors carry a real cost.

The stack in one picture

Stack everything above and a mature agent system looks like this:

+==========================================================================+
|                          HUMAN OVERSIGHT LAYER                           |
|          Approval gates, escalation, audit trail  (Article 13)          |
+==========================================================================+
              |                    |                    |
+==========================================================================+
|                            GOAL & LOOP LAYER                            |
|      ReAct / Plan-Execute / Reflexion, goal revision  (Article 14)     |
+==========================================================================+
              |                    |                    |
+==========================================================================+
|                          ORCHESTRATION LAYER                            |
|      Supervisor-worker, pipeline, fan-out, debate  (Article 6)         |
+==========================================================================+
              |                    |                    |
+==========================================================================+
|                            HARNESS LAYER                                |
|   Self-verification, loop detection, reasoning routing  (Article 5)    |
+==========================================================================+
              |                    |                    |
+--------------------------------------------------------------------------+
|                             MODEL LAYER                                 |
|              AWS Bedrock: Claude 3.7 / 3.5 / Haiku                      |
+--------------------------------------------------------------------------+

  CROSS-CUTTING CONCERNS (touch every layer above):

  +----------------+  +----------------+  +----------------+  +----------------+
  | MEMORY         |  | EVALUATION     |  | SECURITY       |  | COST           |
  | Article 7      |  | Article 8      |  | Article 9      |  | Article 10     |
  | episodic,      |  | offline+online |  | injection def, |  | model routing, |
  | semantic,      |  | LLM judge,     |  | guardrails,    |  | caching,       |
  | procedural     |  | regression     |  | red-teaming    |  | batch, budget  |
  +----------------+  +----------------+  +----------------+  +----------------+

  +--------------------------------------------------------------------------+
  |                          DEPLOYMENT LAYER (Article 11)                   |
  |     Blue-green, canary, state migration, model pinning, IaC             |
  +--------------------------------------------------------------------------+

The vertical layers form the request path. A task enters through human oversight, gets shaped by goals and loops, is coordinated across agents, refined by the harness and finally executed against the model. Concerns that cut across the stack reach every layer. Memory supplies both the loop and the harness. Security guards the tool boundary and goal revision. Cost governs model routing at each step. Deployment wraps the lot.

The level-by-level view hides how tightly these pieces are woven together. A change in one ripples into the others. You can't optimise cost without understanding the loop, since an expensive run is often a loop that won't terminate. Security work needs the same awareness of memory, since one corrupted entry keeps attacking long after it was written. Treat the layers as one interwoven fabric and not a set of separate modules.

What nobody has solved yet

This may be the most useful section, since everything above describes what can be built and this describes what can't, yet. An honest map of the frontier is worth more than a polished one.

Agent-native memory doesn't exist. Today's memory is assembled from databases and vector stores built for other purposes. A real agent-native system, that tracks time, retrieves by association, forgets deliberately and lets the agent decide what to write, is by some estimates still a few years off. What exists works. It isn't what's needed.

Agents can't reliably tell when their knowledge applies. This is the deeper form of the stale-memory problem. An agent may retrieve a relevant procedure and apply it with confidence to a case where it doesn't fit, because where a piece of knowledge stops being valid is hard to write down and even harder to test when something is retrieved. A procedure that works in one jurisdiction gets misapplied in another, and neither the agent nor, often, the user notices. That's a fundamental limitation and not merely an engineering gap.

Self-assessment is unreliable. Escalation logic and confidence scoring both rely on an agent judging its own certainty accurately. At times it manages that, and at times it is wrong while sounding certain, the most damaging case because it defeats every safeguard that depends on the agent recognising its limits. There is no dependable answer yet for calibrated self-assessment.

Harnesses don't transfer between models. Each harness is tuned to the behaviour of a particular model. Change the model and you re-tune prompts, loop-detection thresholds and verification rubrics. It needn't be this way in principle, but nobody has built a harness abstraction that survives a model swap without adjustment. A model upgrade remains a project and not a config change.

There is no true continual learning. Procedural memory approximates learning: successful approaches are stored and reused. Yet the weights don't change and the agent doesn't develop new abilities from experience. It builds a library of past solutions and fetches from it. An agent that genuinely improves at a class of problems through experience isn't something production systems can do today.

Cost is unpredictable at the margin. Cost management tools exist and they work, but the tail of the distribution is stubborn. A small share of runs, those that hit pathological loops or awkward edge cases, still consume disproportionate resources in ways that are hard to foresee. You can cap that spend, though you can't always predict it.

None of this is a reason to hold back. These are the edges of the current map, and knowing where they lie is what separates engineering from wishful thinking.

Where to start: harness engineering before agent orchestration

If you're building an agent system, don't treat the topics as a build order. They're grouped by concept, not by construction sequence. Build in that order and you'll create memory before you have a harness to use it, and evaluation before there's anything to evaluate.

Begin from where your system sits on the maturity ladder.

• A basic tool-calling agent that's unreliable should start with harness engineering, which is where most reliability problems actually live.
• A solid single agent that has hit a ceiling is ready for multi-agent orchestration.
• A system headed for production needs evaluation, security and deployment work, none of it optional, and it's wise to build them in roughly that order.
• Memory pays off once you have a working, evaluated, secured agent and you see the same work repeated across runs. Adding it earlier is premature.
• Cost optimisation becomes urgent when usage moves from test volumes to production volumes, though it should shape the design from day one.
• Goal-directed loops and human-in-the-loop patterns cut across everything. You'll meet them at every stage, so read about them early even if you adopt them gradually.

Tools are interchangeable, ideas aren't

The examples behind this discussion were built on AWS Bedrock, LangGraph and LangChain. Those were practical choices, made because they were the stack in production, and they're not the only correct ones. Most of what matters isn't tied to them. Authentication belongs at the tool boundary on any cloud. The loop matters more than the prompt in any framework. The tiers of memory, the patterns of multi-agent coordination, the categories of attack and the matrix of reversibility against cost for human oversight all port to other stacks.

That's also why a local alternative with Ollama and Docker is worth having. It makes the patterns runnable without a cloud bill, and it proves the concepts are separable from the implementation. If a memory architecture runs against a local model and a local vector store, then the design owed nothing to Bedrock in the first place. What it captured was the underlying problem, and that looks alike on any platform. The same holds for the orchestration patterns, since swapping LangGraph for another framework changes the syntax of the loop but not the failure modes it has to survive, and the same holds for evaluation harnesses that can be pointed at whichever model sits behind them.

What writing it down exposed

Explaining a topic clearly, in order, to someone who wasn't there when you learned it, exposes gaps in your own understanding. Plenty of topics feel mastered right up until you try to write them down.

The honest state of the field is that it's exciting and immature at once. Things are possible now that weren't two years ago, and it's easy to fool yourself because fluent output feels more capable than it is. The distance between an agent that demos well and one that runs reliably in production is enormous, and most of the hard engineering lives in that gap.

If there is one message worth sending back to the start, it's that the interesting work isn't making agents more powerful. Models are already powerful and keep improving on a schedule you don't control. The interesting work is making powerful models reliable, predictable, safe and affordable enough to trust with real decisions. That's engineering and not magic: harnesses, loops, memory, evaluation and all the unglamorous infrastructure that turns a capable model into something you can depend on.

The intelligence is borrowed from the model. The architecture is what you have to build yourself.