Self-Improving Agents: Healing, Learning and Controlled Change
Self-improving agents without the danger: runtime self-healing, learning across runs, and changes that ship only after evals and canaries, on LangGraph.
This article is published in English.
Sources: Self-improving Agentic AI Applications — Feedback loop integration
A ReAct-style agent plans well, calls tools, pulls in information and finishes multi-step jobs. The trouble starts once it's in production and someone asks what happens when it gets something wrong. Can it notice? Can it recover without restarting the whole workflow? Can it remember what worked, learn from user corrections, and eventually improve its own behavior?
People bundle those questions under the label "agent feedback loop", and the label misleads. The architecture below pairs LangGraph with MongoDB, Gemini on Vertex AI and Arize Phoenix, and covers runtime recovery, learning from experience, continuous evaluation and controlled improvement.
Why the naive loop is dangerous
The tempting version of a feedback loop is short:
Agent produces answer
↓
LLM reflects on answer
↓
Agent learns
It's attractive because it's simple, and it's also risky. When an LLM judges its own output and immediately alters future behavior, a mistake can turn into a lesson, the lesson can turn into memory, and that memory can then shape thousands of later decisions.
A safer design rests on a different principle: the agent heals the current execution, the platform learns across many executions, and a control layer decides what becomes production behavior.
A concrete case: an agent recommends payroll earning or deduction codes. In one run it picks the wrong code because it never retrieved a state-specific payroll policy. The naive learning path goes like this:
Agent made mistake
↓
Agent reflects
↓
"Always retrieve state policy"
↓
Write lesson to memory
↓
Future agents use lesson
Several things can go wrong with that. The diagnosis might be incorrect. The rule might hold for only one type of payroll transaction. A later policy change could make the lesson obsolete. Malicious user input might steer the reflection. And the lesson might help some cases while hurting others.
So the better design treats a lesson as a hypothesis and not as truth.
Production failure
↓
Capture evidence
↓
Evaluate the run
↓
Identify recurring failure
↓
Generate lesson candidate
↓
Gather supporting and contradicting evidence
↓
Validate
↓
Canary test
↓
Activate
The shift is a subtle one. The agent isn't constantly rewriting itself. The platform around it is constantly evaluating and evolving the agent.
Three planes with different authority
It helps to picture three connected planes.
+------------------------------------------------+
| RUNTIME PLANE |
| |
| plan -> act -> validate -> repair -> respond |
+-----------------------+------------------------+
|
v
+------------------------------------------------+
| LEARNING PLANE |
| |
| evaluate -> diagnose -> cluster -> learn |
+-----------------------+------------------------+
|
v
+------------------------------------------------+
| CONTROL PLANE |
| |
| test -> approve -> canary -> rollout -> rollback|
+------------------------------------------------+
Each plane runs at its own pace and holds a different degree of authority. The runtime plane works within milliseconds or seconds, and its single task is to complete the present request reliably. The learning plane studies many runs spread over minutes, hours or days, and decides what the pooled evidence shows. The control plane rules on whether a proposed improvement is safe enough to turn into production behavior. Few design choices in an adaptive agent platform matter more than keeping these three apart.
1. Runtime self-healing
Start with the immediate problem: a tool call fails. A simplistic implementation just retries.
Tool failed
↓
Retry
↓
Retry
↓
Retry
Retrying isn't self-healing, though. Healing requires knowing what failed and choosing a fitting recovery strategy. A stronger LangGraph workflow looks closer to this:
User Request
|
v
Clarification Gate
|
v
Retrieve Context
|
v
Plan
|
v
Proposed Action
|
v
Action Guard
|
v
Execute Tool
|
v
Sanitize Tool Result
|
v
Validate Tool Result
|
v
Reason
|
v
Validate Answer
|
+------ uncertain ------> Critic
| |
| v
| Policy Router
| / | | \
| PASS REPAIR HUMAN FAIL
| |
+---------------------------+
|
v
Response
A few ideas in that graph deserve attention.
Validate before acting
Imagine an agent that can take side-effecting actions: updating employee data, creating a payroll deduction, submitting a transaction, modifying an account or sending a notification. Checking the result after the fact is too late. The architecture needs an action guard ahead of any consequential change. The guard weighs permissions, business rules, available evidence, whether the action is reversible, risk level, and whether a person must approve. Only then may the tool run.
That reframes validation. The question stops being "was the action correct?" and becomes the much safer "do we have enough evidence and authority to allow this?"
Tool output is untrusted too
Another risk gets overlooked. Agents read content from websites, documents, APIs, search engines, databases and other agents, and that material flows into the LLM's context. Tool results are therefore one more route for prompt injection, so a deterministic tool-result sanitizer belongs between tool execution and reasoning.
External Tool
|
v
Tool Result
|
v
Sanitizer
|
v
Validator
|
v
LLM
Typical sanitizer duties include enforcing output type and length, rejecting malformed structures, escaping text that looks like instructions and flagging probable injection attempts. One constraint is firm: no LLM may be used to inspect text that might be adversarial.
Keep the validator apart from the critic
A common pattern asks an LLM "is this answer correct?" on every request. That adds latency and cost, and it hands the model too much authority. Deterministic validation should come first, for example:
Schema correct?
Required fields present?
Allowed value?
Business invariant satisfied?
Evidence exists?
Policy satisfied?
Known contradiction detected?
If software can prove an answer is wrong, there's no reason to ask another model. Only genuine semantic ambiguity reaches the critic, and even then the critic doesn't steer the workflow. Its role is diagnosis, along these lines:
{
"correctness": 0.61,
"groundedness": 0.92,
"uncertainty": 0.73,
"defects": [
"missing_authoritative_evidence"
]
}
A deterministic policy router then picks among four outcomes:
PASS
REPAIR
HUMAN REVIEW
SAFE FAIL
The split is clean: the LLM supplies evidence and judgment, while software policy holds the authority.
Repair should change the strategy
A retry only has value when something is different. Take an agent doing this:
Search policy
↓
Generate recommendation
↓
Fail validation
If repair repeats the sequence below with the same query, the same evidence and the same reasoning approach, that isn't recovery. It's repetition.
Search policy
↓
Generate recommendation
A sturdy agent keeps track of a few fields.
failure signature
strategy fingerprint
attempt ID
repair strategy
remaining budget
quality delta
Then a simple rule applies.
same failure
+
same strategy
+
same evidence
=
do not retry
Whenever the same failure meets the same strategy a second time, the agent should switch strategies, hand off to a person or fail safely. A bare retry_count doesn't cover this, so production self-healing needs repair budgets. These can limit LLM calls, tool calls, elapsed time, spend and semantic repair attempts. Persistence until something works isn't the goal. A limited amount of resources, spent on recovery strategies that actually differ, is.
Check that repair helped
Different attempts can still fail to move the needle. Say an evaluator scores the initial result:
quality score = 0.54
After the first repair:
quality score = 0.55
After the second:
quality score = 0.56
On paper that's improvement, and in practice it's a stall. A repair delta gate catches it.
delta = score_after_repair - score_before_repair
When several repairs each bring negligible gains, resources stop flowing.
small delta
+
small delta
=
human review or safe failure
That guards against one of the commonest agent failures, the expensive reasoning loop that looks busy and achieves almost nothing.
Human review is a routing choice
Human review shouldn't count as a failure. For some tasks it's simply the right final decision. And "human review" covers several different outcomes.
Human
|
+-----------+-----------+
| | |
Approve Edit Reject
| | |
Continue Validate Safe Fail
Request Repair
|
v
Repair
Each outcome continues differently, so each should appear explicitly in the graph. LangGraph's interrupt and resume model suits this well, since the workflow can be checkpointed, suspended, then resumed when the reviewer responds.
2. Learning from experience across runs
Runtime healing answers how to rescue this request. Experience learning asks what the request teaches about future ones. It generally belongs in the background. Once the agent has replied:
run_completed
|
v
Evaluation worker
|
v
Gather feedback
|
v
Reflection
|
v
Candidate lesson
Nobody has to wait while one LLM reflects on what another LLM might have learned. Latency stays low, and the offline evaluation can be far richer.
Feedback isn't truth
Signals arrive from many directions. A user taps thumbs down. A subject matter expert fixes an answer. A downstream API turns a transaction away. A compliance audit spots an error two weeks on. These aren't equal. Consider the first pair of equivalences that fail:
thumbs up != correct
thumbs down != incorrectuser correction != authoritative rule
Feedback should therefore be kept as raw evidence and ranked by reliability, roughly like so:
Authoritative business outcome
>
Expert human label
>
Deterministic rule
>
Calibrated evaluator
>
User feedback
>
Agent self-confidence
That stops sentiment from being mistaken for ground truth.
Some of the best feedback comes late
Agents often start workflows whose success can't be seen right away.
Agent recommends payroll code
|
v
Payroll system accepts
|
v
Two weeks later
|
v
Audit rejects transaction
The useful signal arrives well after the LangGraph run has finished, which means each important decision needs causal lineage. Pass the run_id into downstream systems so later outcomes can be traced to the decision behind them. Otherwise the learning system learns that something failed but can't tell which agent decision earns the credit or the blame. This is the classic credit-assignment problem.
Memory needs governance
Once agents learn across runs, memory turns powerful and dangerous at once. Three classes are worth keeping apart.
- Profile memory stores preferences of a user or entity, such as a preference for concise explanations.
- Episodic memory records something that happened, for instance that run 182 failed because the state policy wasn't retrieved.
- Procedural memory holds a reusable strategy, such as resolving the work state before choosing a code in state-sensitive payroll classification.
Procedural memory changes behavior the most, so it deserves the tightest governance. A single failed run must never mint a production procedural rule. The path should look more like this:
Observation
↓
Candidate lesson
↓
Supporting evidence
+
Counterexamples
↓
Validation
↓
Active lesson
A lesson is better modeled as hypothesis plus evidence plus scope plus trust plus lifecycle, and not as text dropped into vector memory.
Learned memory can't outrank authoritative knowledge
Agents retrieve two kinds of information. Authoritative knowledge means policies, regulations, configuration, reference data and contractual business rules. Learned knowledge means preferences, experiences, earlier strategies, observations and lessons. They should stay logically separate, with a sensible precedence:
Authoritative policy
>
Tenant configuration
>
Approved procedural lesson
>
Episodic example
>
User preference
>
Unverified claim
>
LLM reflection
Otherwise a confidently remembered lesson can quietly overrule actual policy.
Lessons go stale
Something useful six months ago may be wrong now. Policies, products, APIs, user behavior and models all change. Active lessons need lifecycle management.
candidate
↓
validated
↓
active
↓
pending revalidation
↓
deprecated
↓
retired
Give every lesson a validation timestamp and, when it makes sense, an expiry or a revalidation policy. A policy change can then trigger re-evaluation of all lessons built on that policy, so memory turns into governed knowledge and not a bucket of embeddings that only grows.
3. The offline learning plane
The next layer aggregates evidence across many production runs.
Runs
+
User Feedback
+
Human Reviews
+
Downstream Outcomes
+
Evaluation Scores
|
v
Failure Classification
|
v
Failure Clustering
Say a week of runs reveals this distribution:
missing state policy 178
wrong tool selected 63
bad tool argument 52
unsupported inference 41
output schema failure 11
Now the platform can see which problems are systemic. That's a different thing from asking a single failed agent to reflect on a single failed run. The learning plane sees patterns.
Not every failure is a prompt problem
Prompt engineering isn't always the answer. If an agent keeps forgetting to fetch an authoritative policy before recommending something, adding a line to the system prompt asking it to remember might help. A stronger fix could be a structural rule:
Graph rule:
Policy retrieval must occur before this decision.
Other recurring failures may call for changes to these areas:
retrieval
tool schema
validator rule
routing
memory policy
clarification logic
action guard
model configuration
A mature improvement engine proposes several candidate fixes and doesn't automatically rewrite the prompt.
Evaluation sits at the center
If a system can generate improvements but can't judge whether they're better, there's no learning loop, only a mutation loop. Evaluation is the anchor of this architecture.
Correctness
Groundedness
Retrieval relevance
Completeness
Tool selection
Tool arguments
Trajectory efficiency
Repair effectiveness
Safety
Latency
Cost
Business outcome
Evaluation must cover the final answer and also the path taken to reach it. Two agents may produce the same reply. Agent A does this:
Agent A
search -> answer
while Agent B does this:
Agent B
search
-> wrong tool
-> retry
-> timeout
-> second search
-> repair
-> answer
Judged by the final answer alone, they're equivalent. Trajectory evaluation shows they clearly aren't.
One LLM judge isn't enough
Having Gemini write an answer and another Gemini prompt grade it is useful but carries correlated-error risk, since the evaluator may share the generator's blind spots. A stronger evaluation layer combines several sources.
Deterministic checks
+
Business outcomes
+
Human labels
+
Multiple evaluator rubrics
+
LLM judges
LLM evaluators should also be compared with human-labeled examples from time to time. That answers how far the evaluator can be trusted before its scores influence which lessons get promoted.
Observability belongs to learning
Traditional monitoring asks whether the service is up. Agent observability must also ask why the agent made a given decision. A good production setup records a trace across the whole LangGraph execution.
Agent Run
|
+-- Retrieve Context
|
+-- Planner
|
+-- Tool Call
|
+-- Tool Result Validator
|
+-- Repair
|
+-- Critic
|
+-- Final Answer
Each run should also carry a stable identity tying together several records.
LangGraph execution
MongoDB events
Vertex AI calls
Evaluation results
Human feedback
Downstream outcomes
That correlation is the raw material of the loop. Observability stops being only a debugging aid and becomes part of the learning system.
Why append-only events help
Stuffing the whole execution trajectory into one MongoDB run document is tempting. It works at first and gets awkward at scale. Keep the run compact instead.
run_id
agent
release
outcome
latency
tool count
repair count
cost
Then record execution as append-only events.
run.started
retrieval.completed
tool.called
validation.failed
repair.started
human_review.requested
run.completed
That preserves causal ordering, avoids huge growing documents, simplifies distributed tracing and makes failure analysis more precise.
Reproducibility needs more than a prompt version
An agent's behavior depends on more than its system prompt.
model
generation settings
graph version
tool definitions
validator rules
critic rubric
retrieval configuration
memory rules
security rules
Each production run should therefore pin an immutable agent release. Conceptually:
agent_release_43
|
+-- graph v12
+-- prompt v43
+-- Gemini configuration
+-- tools v17
+-- validator v11
+-- retrieval config v9
+-- memory policy v5
+-- evaluation suite v8
A failed run can then be reproduced later. Without it, debugging collapses into asking which version produced an answer, followed by silence.
The control plane: improvements earn production access
Eventually the learning system spots a recurring failure and drafts an improvement. The worst response is to deploy it at once. Run it through gates instead:
Candidate Improvement
|
v
Historical Replay
|
v
Regression Evaluation
|
v
Shadow Production
|
v
Canary
|
v
Progressive Rollout
|
v
Production
At every stage, compare the candidate with the current release. It might lift correctness and hurt latency, cut hallucination and raise human escalation, or help one task type while harming another. Promotion therefore needs several guardrails.
Lessons need canaries too
Long-term memory has an interesting consequence. When a new procedural lesson gets activated and injected into future agents' context, their behavior changes, so injecting a lesson is effectively a prompt change. Activation should be treated as an experiment, not handed to every eligible run immediately.
Historical replay
↓
5% canary
↓
Measure outcome
↓
25%
↓
Measure
↓
100%
If the lesson degrades performance, roll it back. Memory then stops being a mysterious adaptive mechanism and becomes a measurable software release.
The final architecture
Pulled together, the adaptive agent platform looks roughly like this:
USER
|
v
+------------------------------------------------------+
| RUNTIME |
| |
| clarify -> retrieve -> plan -> action guard |
| | |
| v |
| tool |
| | |
| sanitize |
| | |
| validate |
| | |
| reason |
| | |
| answer validate |
| | |
| critic if needed |
| | |
| policy router |
| / | \ |
| repair human pass |
+--------------------------+---------------------------+
|
agent events
|
v
+------------------------------------------------------+
| LEARNING |
| |
| traces + feedback + outcomes |
| | |
| v |
| evaluate |
| | |
| classify failures |
| | |
| cluster |
| / \ |
| lessons improvement candidates |
+--------+----------------------+----------------------+
| |
v v
+------------------------------------------------------+
| CONTROL |
| |
| validate -> replay -> shadow -> canary -> rollout |
| | |
| monitor |
| | |
| rollback |
+------------------------------------------------------+
Responsibility boundaries are intentionally sharp. The runtime agent repairs execution. The learning system generates hypotheses. The control plane decides what becomes trusted behavior.
Rethinking "self-improving"
Hearing "self-improving agent", many people picture an AI rewriting its own system prompt or updating after every chat. For production systems that's the wrong mental model. A better one is a cycle:
Observe
↓
Measure
↓
Diagnose
↓
Propose
↓
Test
↓
Promote
↓
Monitor
Intelligence here isn't confined to the LLM. Different parts of the system carry it:
- The LangGraph graph decides the order of execution.
- Plain code validators enforce rules that are already known.
- The LLM handles whatever is ambiguous.
- MongoDB keeps state and accumulated experience.
- Tracing gathers the evidence.
- Evaluators score quality.
- People make the call when the risk is high and the answer unclear.
- The control plane governs any change in behavior.
That distribution turns an agent into an adaptive agent system.
A sensible build order
Nobody needs to build the whole loop in one go. A workable sequence starts with dependable execution and a trace on every run. On top of that go action guards, result and answer validation, bounded repairs and human review. Measurement of quality is next: traces, failure evaluation and classification, and a regression dataset worth having. Experience learning enters only when quality can be measured with confidence, bringing lesson candidates, scoped memory, evidence aggregation, staleness controls and delayed feedback. Controlled improvement is the final stage, with failure clustering, candidate generation, replay evaluation, shadow traffic and canary rollout.
The order matters:
Reliability
↓
Observability
↓
Evaluation
↓
Learning
↓
Controlled adaptation
Jumping to "self-learning" before evaluation exists usually yields a system that changes faster than engineers can follow.
The goal
Nobody should aim for an agent that edits itself. The better target is a platform able to demonstrate when an agent ought to change. Remembering earlier conversations isn't enough for a loop to deserve trust. It needs a disciplined chain between these stages:
production behavior
↓
evidence
↓
evaluation
↓
learning
↓
experimentation
↓
controlled production change
That chain is what an evidence-driven adaptive agent system stands on, and it's the architecture agentic AI needs if it's going to graduate from striking demos to dependable production software.