Home / Articles

SkillClaw: When One Agent Fails, Every Agent Learns

How SkillClaw gathers interaction traces from many users, evolves shared agent skills with an agentic evolver, and validates changes overnight before deploying.

Publisher: REACTAPP.TOP5 min read1209 words

This article is published in English.

Sources: SkillClaw: One Agent Fails, Every Agent Learns

Letting agent skills improve over time is one of the more promising ideas in the field right now. SkillClaw, a research framework from an April 2026 arXiv paper, applies it to a setting where many users share one agent ecosystem (OpenClaw), and it's worth walking through how it works and where it may fall short.

Why current skill setups stall

Agents talk to users all day and pile up experience, yet very little of it makes the system better overall. A few reasons stand out.

  • Skills are mostly static. Somebody installs and maintains them by hand, and afterward they rarely change. Even if an agent finds a better method in the middle of a task, the improvement usually stays inside that conversation and never reaches the skill repository.
  • Users keep hitting the same walls. People follow similar workflows, call the same tools and repeat the same mistakes: wrong parameter formats, tool calls in the wrong order, missing validation. Nothing links these patterns across users or over time, so many people rediscover the same fix on their own.
  • Existing approaches only cover half the problem. Memory systems store history so it can be searched, but they struggle to turn it into skills that can be reused or refined. Skill libraries do package experience, but they tend to be frozen and don't evolve through use.

Imagine turning scattered experiences from many sessions into skills that the whole system shares and keeps improving. A validated fix found in one place would travel through the shared repository, and any future agent using that skill would benefit.

A closed loop for collective evolution

SkillClaw collects interaction traces from everyday use across users, evolves them into better skills, and syncs the improved skills back to every agent. The cycle has four steps.

1. Record the interaction traces

Every exchange is captured in full: not only final answers but intermediate prompts, actions, tool calls, errors, feedback from the environment and the closing response. The detail matters because many skill problems hide in the middle steps and not in the outcome.

2. Build shared evidence

Traces are structured and grouped by the skills they involved. That makes it possible to compare successes and failures of one skill across different users and settings, which shows when and why it works. Sessions that used no existing skill go into a special group, written G(∅), which the evolver mines for gaps: concrete procedures that could be taught and probably come up again.

3. Evolve skills with an agentic evolver

The central piece is an agentic evolver. It doesn't follow hand-written rules. It reasons openly over a group's evidence together with the current skill definitions, then chooses among three outcomes: refine the existing skill, create a new one, or leave things alone.

It looks at wins and losses together. Successes mark the parts of a skill that must not be touched, while failures show what needs work. The aim is to avoid fixing one failure while breaking something that already worked, and the next step adds another safeguard.

4. Verify before deploying

Nothing goes live right away. Candidate skills are tested overnight in realistic environments, comparing new against old on relevant tasks.

  • Only candidates that beat the currently deployed skill on the matching validation tasks are accepted, merged into the shared repository and rolled out to all agents the next day.
  • Candidates that perform worse are discarded, giving what the authors call monotonic deployment: the skill pool never gets worse.

That safety check has a price. Validation consumes extra tokens because candidates must run in real environments with full tool interaction. The payoff is more stable behavior for users compared with deploying changes directly.

Evaluation

The question was whether skill evolution driven by multi-user traces improves performance on later tasks that resemble real agent work.

Setup

The benchmark, WildClawBench, has 60 complex tasks across six capability domains. Tasks run end to end inside real Linux containers with real toolchains and multimodal inputs, not just text output. Scoring is strict, and a critical error can drop a task's score to zero.

The test also wasn't a single offline run. It simulated six days of deployment. Each day, eight concurrent users ran tasks through agents. Each night the system generated candidate skill updates from that day's traces and validated them in the realistic environment, and only updates that passed went live the following day. The same model, Qwen3-Max, was used throughout.

Results

Results were reported for four representative task categories, and the improvement paths differed. Creative synthesis first gained from more reliable environment and input setup. Search and retrieval improved in stages, beginning with file validation and then moving to retrieval planning that respected constraints better.

Compared with day one, the curve wasn't noisy. It rose step by step and then leveled off in most categories, as validated updates were folded into the best skill pool so far.

Keep the scale in mind, though: eight users, six days and four reported categories. It's a small experiment. The steady gains are encouraging for collective skill evolution, but larger deployments are the real test.

Assessment

SkillClaw is heavily engineered. It aggregates execution traces from existing agents, uses an agentic evolver to reason over successes and failures, edits skills based on what it learns, then tests asynchronously and keeps the best results. All of that depends on having plenty of interaction data, both good and bad, from a large user base.

Two concerns stand out.

  • Evolution, execution and validation all rely on one backbone model. In validation especially, the model compares old and new results itself. That can create a loop in which one set of biases keeps reinforcing itself. Benchmark scores might climb steadily without real business metrics following.
  • Deciding which interactions are actually good is an unsolved problem. Without a trustworthy evaluation loop, building a high-quality trace repository is hard to begin with.

The framework also resembles Trace2Skill, a related approach that distills agent experience into transferable skills. Both share the insight that agents can improve without parameter fine-tuning, by turning experience traces into reusable skills. The differences are notable:

  • SkillClaw treats skill evolution mainly as an online systems problem: how distributed agents can gather fragmented daily interactions into shared skills and iterate on them safely in a simulated realistic setting.
  • Trace2Skill treats it as experience distillation: break down many individual traces, synthesize high-quality general lessons and compose them into transferable, declarative skill artifacts.

The source paper is titled "SkillClaw: Let Skills Evolve Collectively with Agentic Evolver" and is available on arXiv.