Building a Good Claude Agent Skill: Lessons From a Running Coach Project

Picture a runner who, three years in a row, set a goal of 1,000 km and missed it each time, finishing the three years at roughly 2,500 km in total. Effort wasn't the problem. Injuries, restarts and busy weeks all happened. What was missing was a plan, because motivation works well until the day it doesn't.

One answer is to stop relying on willpower and build a system. That is how running-coach came about, a Claude Agent Skill that plans individual runs and whole weeks from a person's own history, turns each week into an editorial-style email digest, and even sizes a playlist to the length of the run ahead.

Running is only the backdrop here. The more useful subject is what it takes to build a good skill, meaning one that triggers reliably, uses context sparingly, behaves identically on every platform, and doesn't decay over time. Below, each of Anthropic's published authoring practices is matched with the way this project applies it.

What an Agent Skill is

A skill is a folder. In its root is a SKILL.md file that begins with YAML frontmatter holding a name and a description, then continues with instructions in Markdown. Reference documents, scripts, templates and other assets can sit alongside it.

The interesting part is how a skill is loaded, using a three-tier model called progressive disclosure:

• Level 1, metadata. Always loaded at startup, at about 100 tokens per skill. It consists of the name and description only.
• Level 2, instructions. Loaded only once the skill triggers, and kept under roughly 5k tokens. This is the body of SKILL.md.
• Level 3, resources. Pulled in on demand through bash, and free until touched. Reference files, scripts and assets live here.

Anthropic likens a skill to an onboarding guide written for a new colleague. The principle underneath everything else is that the context window is shared, a public good. Whatever a skill spends is unavailable to the rest of what the agent is trying to keep in mind.

1. The description works as the API

At Level 1, all the model ever sees of a skill is its description. It acts less as documentation and more as a routing table, since the model matches an incoming request against it to decide whether to fire the skill. It must state both what the skill does and when to use it, be written in the third person, and stay under 1,024 characters, with about 200 as the practical limit if you want it shown untruncated in claude.ai's picker. This is the description running-coach uses:

name: running-coach
description: >-
  Plans a run or a whole week as Google Calendar blocks, builds a weekly
  running digest email, and makes playlists — from Strava, Spotify, AccuWeather
  and Calendar. Use for pacing or planning a week.

Look at how it's built. It opens with what the skill can do (plan a run, build a digest, make playlists), follows with when to reach for it (pacing, or planning a week), and spells out the data sources so that a Strava-style question links to this skill. The name is kebab-case and identical to the directory name, a hard rule and not a style preference, and one the project's build enforces, as shown later.

Whether a description works can't be known in advance. It has to be tested, which is why the first thing in the repository is a set of evaluations and not documentation.

2. Write evals before polishing the prose

Anthropic's advice is to build evaluations first, so you solve problems that exist and not ones you imagined. The project carries four scenario files in an evals/ folder, one per capability. They aren't unit tests. They are behavioural contracts written in plain language. This is the one for planning a run:

{
  "skills": ["running-coach"],
  "query": "I'm heading out for 8k this evening, how long is that going to take me?",
  "expected_behavior": [
    "Triggers the running-coach skill without being named, and routes to the planning capability",
    "Reads references/run-planning.md and does not read the digest or playlist references",
    "Runs scripts/cache.py status before projecting…",
    "Executes scripts/planning/plan_run.py 8 rather than reading its source or reimplementing Riegel inline",
    "Reports the projected finish time, the pace, and the recommended effort band…",
    "Never asks for or uses a Strava access token"
  ]
}

Each assertion encodes a best practice. Triggering without being named tests description quality. Reading only the relevant reference tests progressive disclosure. Running the script and not reimplementing it tests the preference for scripts over prose. Never touching a token tests the credential boundary. Together they turn the whole design philosophy into observable behaviour, and the remainder of the build is about making them pass.

3. Keeping SKILL.md as a router

The most common way to burn a skill's context budget is to pour everything into SKILL.md. A better approach is to treat that file as a table of contents that points to deeper material when needed. Keep it short, and keep references one level deep, because agents tend to read deeply nested files only partially and end up with half a picture.

Four capabilities here would each take a page to explain, so SKILL.md explains none of them. It routes:

| Capability     | Reference                        | Entry point                       |
| -------------- | -------------------------------- | --------------------------------- |
| Plan a run     | references/run-planning.md       | scripts/planning/plan_run.py      |
| Plan the week  | references/week-planning.md      | scripts/planning/plan_week.py     |
| Weekly digest  | references/weekly-digest.md      | scripts/digest/run_pipeline.py    |
| Run playlist   | references/run-playlist.md       | scripts/playlist/build_playlist.py|

Work out which one is wanted, then read only that capability's reference.

Ask for a playlist and the agent opens run-playlist.md and nothing else. The digest's workflow spec, its gotchas and its template notes never enter context, since unused bundled content costs nothing. Level 2 stays lean and Level 3 contributes exactly one file. The "reads X, does not read Y" line in the eval verifies exactly this.

4. Scripts over prose: run them, don't read them

This may be the most valuable idea in skill design. When the agent runs a script through bash, only the output enters context and the source never does. Pre-written scripts are also more dependable than code produced on the fly, and they give the same result every time.

So the logic of running-coach lives in Python, and the instructions explicitly tell the agent to leave the source alone:

**Run the scripts, don't read them.** They print everything needed; every one
takes --help and most take --json. Reading their source wastes context.

The Riegel finish-time projection, the effort-band calculation and the digest analysis all run as deterministic Python and return a compact answer. The agent orchestrates and doesn't recompute. Compare an agent that reasons about your pace, which is slow, variable and token-hungry, with one that runs a pace calculator and reports the number, which is fast and identical every time.

5. Every script is a proper CLI with shared conventions

For "run it, don't read it" to work, the agent has to drive a script it is forbidden to open. That depends on a uniform, self-describing interface, so each script follows the same conventions:

• one positional argument for whatever is always required (distance, for planning and playlists);
• --data DIR everywhere, so any script can point at a different cache, which is what enables offline testing and per-session sandboxes;
• --json on nearly all of them, so one script's output can feed the next;
• --help on every one, the agent's fallback when unsure about a flag. It reads the help text, which is cheap, and not the source, which is costly and off limits.

Here is the surface, capability by capability.

Planning a run uses plan_run.py:

python3 scripts/planning/plan_run.py 8                 # 8 km, defaults
python3 scripts/planning/plan_run.py 8 --band steady   # force an effort band
python3 scripts/planning/plan_run.py 8 --start-in 3    # start 3h from now (weather-aware)
python3 scripts/planning/plan_run.py 8 --json          # machine-readable plan

The one required argument is distance_km. Optional flags are --exp, which adjusts the Riegel fatigue exponent, and --band, which forces a particular effort band. With no override, it derives the band from the runner's trailing training load. The heuristic part gets freedom while the exact projection stays deterministic.

The playlist is handled by build_playlist.py, which deliberately mirrors the planner's shape:

python3 scripts/playlist/build_playlist.py 8                    # size to the 8 km finish
python3 scripts/playlist/build_playlist.py 8 --genre "lo-fi"
python3 scripts/playlist/build_playlist.py 8 --minutes 55       # size by time instead
python3 scripts/playlist/build_playlist.py 8 --band hard

Because both take the same positional distance, the two capabilities compose: the finish time plan_run projects is the duration build_playlist sizes the playlist to. The script emits a prompt and not a playlist, and the agent passes that prompt to the Spotify connector. The credential boundary shows up again here, with the script doing arithmetic and the agent doing I/O.

The weekly digest is a small pipeline and not a monolith. run_pipeline.py validates the cache and then runs two stages:

python3 scripts/digest/run_pipeline.py                 # validate → analyse → render
python3 scripts/digest/run_pipeline.py --plan-km 8     # lead the digest with a planned run
python3 scripts/digest/run_pipeline.py --seed 3        # pin the masthead title choice
python3 scripts/digest/run_pipeline.py --skip-validate # skip the pre-flight cache check

Beneath it, analyze.py aggregates the cached week into one digest_data.json, and build_email.py renders that JSON as a self-contained HTML email. Separating analysis from rendering means the email can be restyled without touching analysis, and a --seed flag makes the single random choice, the masthead title, reproducible, handy for screenshotting the same week twice.

Planning the week is the most interesting CLI because it is a plan, validate, execute pipeline expressed as subcommands that the agent steps through in order:

python3 scripts/planning/plan_week.py goal       # show the weekly goal in force
python3 scripts/planning/plan_week.py propose    # propose the split of runs (as JSON)
python3 scripts/planning/plan_week.py draft   --split -   # lay the split across the days
python3 scripts/planning/plan_week.py brief   --draft -   # hand the agent a per-run contract
python3 scripts/planning/plan_week.py events  --draft - --content -   # emit calendar blocks

Each stage prints JSON that the next one consumes on stdin. Where they apply, the stages also accept common overrides (--km, --runs) and scheduling options such as --start-date, --time, --weekend-time, --location and --tz. The guidance recommends this "verifiable intermediate outputs" shape for fragile work: the agent never free-writes a week. It emits a structured plan, the script checks and shapes it, and only afterwards does it turn into calendar events.

Midway there's a deliberate handoff. At the brief stage the script works out distance, pace window and day, then gives the agent a contract for each run. The agent writes the human-sounding words that fill it, so no two run descriptions read alike. The script owns what must be exact and the model owns what should feel written. Low freedom for fragile maths and high freedom for prose is the "appropriate degrees of freedom" principle in a single command.

6. Solve problems, don't punt them

A weak skill pushes hard cases onto the agent with vague orders such as "if the data looks stale, work something out." Anthropic's wording is to solve and not punt: handle error conditions in the script and don't delegate them. running-coach applies this through a freshness model. For every piece of cached data it records how old it is, and status names the exact MCP call that would close each gap:

python3 scripts/cache.py status                 # what's cached, how stale, what to call
python3 scripts/cache.py put streams --file -    # ingest an MCP response from stdin
python3 scripts/cache.py show week --json        # a compact summary, not the raw blob
python3 scripts/cache.py validate --for digest   # shape-check before a run

cache.py has four verbs (status, put, show, validate), each a rung on the refresh ladder. status lists every key with its freshness budget, tight for the current week since it changes with every run, and weekly for completed weeks and athlete data, and for anything stale or missing it prints the specific MCP tool to call. The agent makes the call and pipes the response straight into put. Then validate --for <capability> confirms the result is usable before a downstream script runs, so a bad fetch stops right there with an error, rather than surfacing as a broken digest later.

Further guardrails: put accepts only keys from a fixed allow-list, show prints a summary and not raw JSON so inspecting the cache doesn't dump a megabyte of GPS data into context, and the agent is instructed not to edit those JSON files manually. The upshot is that the agent doesn't have to decide whether to refetch or what to fetch, because the script says. A fuzzy judgement ("is this fresh enough?") becomes a deterministic instruction, which is exactly what the eval means by running cache.py status before projecting.

7. The credential boundary

This is the central architectural choice and also a security practice: audit for unexpected network calls and file-access patterns. The skill makes that audit trivial by construction, since nothing inside it holds a credential or opens a socket.

agent --(Strava / Spotify / AccuWeather MCP)--> cache.py put --> the cache
the cache --> plan_run.py / run_pipeline.py / build_playlist.py --> output

The agent alone speaks to MCP connectors, and cache.py alone writes the cache. All remaining scripts just transform cached data offline, with no HTTP client, no API keys and no access tokens anywhere in the package.

# cache.py — the only gateway to the running-coach data cache.
# The agent is the only thing that talks to MCP; this script is the only thing
# that reads or writes the cache. Nothing here needs an access token.

Three benefits follow. For security, there's nothing to leak, and the network surface to audit is empty. For portability, the skill doesn't care whether connectors are local, remote or missing, because it only consumes a cache. For testability, a populated cache lets every downstream script run fully offline, which is what makes a CI smoke test possible.

8. One runtime requirement: Python 3.10

Skills don't sync across surfaces, and each surface has its own runtime limits. The Claude API offers no network access and no runtime package installation, whereas Claude Code has everything. To run one skill everywhere, design for the most constrained environment.

That is why running-coach pulls in nothing external. There's no requirements.txt, no virtualenv and no install stage. Its instructions say Python 3.10 or newer is all that's required, and plain python3 works as it stands, even in a hosted sandbox where installing packages is impossible.

Everything the skill needs sits inside the skills/running-coach/ directory. Dropping that folder into ~/.claude/skills/ serves Claude Code, and a zipped upload serves Cowork or claude.ai, with identical behaviour. Paths use forward slashes throughout, a quiet cross-platform pitfall avoided. Cache and output files go to a location outside the skill folder, keeping the installed copy read-only, and are never committed to git, because the cache contains real GPS polylines that decode to routes starting at someone's front door.

9. Packaging checks and CI that back up the claims

Packaging is the step people skip. A skill that installs wrongly looks broken in ways that aren't obvious: an archive with loose files at its root is silently rejected on upload. So the project includes package.py, a validator with no dependencies that repeats the checks found in Anthropic's own skills repository. It verifies the allowed frontmatter keys, a kebab-case name that matches the directory, a description length within spec (with a warning at claude.ai's roughly 200-character picker limit), and an archive containing exactly one top-level directory with SKILL.md at its root.

GitHub Actions then gates every release on those checks. When a v* tag is pushed, the workflow:

1. byte-compiles every script (python -m compileall) to catch syntax errors;
2. checks the frontmatter against the spec;
3. runs every entry point with --help, which shows at minimum that each script imports and parses on a clean Python 3.10;
4. verifies that running-coach/SKILL.md is at the root of the archive;
5. and only after all that, publishes the zip.

# 3.10 is the version the skill claims to support, so building on it is
# also how that claim stays honest.
- uses: actions/setup-python@v5
  with:
    python-version: "3.10"

Lessons that generalise

A handful of points stand out from the build:

A description is about 100 tokens that do 90% of the work. Because it's the only part always loaded, a failure to trigger makes everything else irrelevant. Give it outsized attention, and add evals that test triggering without mentioning the skill by name.

Shift logic from prompts into deterministic scripts. Text the agent reads varies and costs tokens, while a script it runs is cheap and gives the same answer each time. The slogan is to run, not read.

Trust a boundary you can say in one sentence. "Only the agent speaks to MCP, one script writes the cache, all else is offline" is worth more than piles of defensive code, since whole classes of bugs and leaks become structurally impossible.

Have the build check your claims. A skill that promises Python 3.10 and zero dependencies should be built exactly that way in CI at every release. Without that, the promise is only a hope.

The running goal itself stays open, but there's now a coach that plans the week, gives an honest read on Monday morning and recomputes the target from trailing mileage every week. Running turned into a solvable engineering problem once it was treated as one.

Anyone installing a skill like this needs code execution enabled and their own Strava connector, since the skill carries no credentials. The same goes for the sources worth reading: Anthropic's Agent Skills overview and its skill-authoring best-practices page, both on the Claude Platform docs.