Structured System Prompts for Rust AI Agents: Layers, Caching, Drift Checks

A first agent usually gets by with a system prompt that is one string literal, maybe sixty words: answer questions about a Rust project, and call read_file when you need to see a file's contents. That works because there's exactly one tool, one user and no rule about how answers should look. Then things pile up. A second tool means the model must learn when to list files and when to read them. A third means teaching an order. An output rule such as "cite the file you read" needs a home. A regression test asking whether the prompt still yields the same answer on fixed input needs a way to notice that somebody changed the prompt by accident. Prompt caching makes you care which parts change per turn and which don't. A bare literal handles none of this gracefully.

What follows is a structured prompt for an agent called Eugene that does handle it: four layers, a typed builder, per-request context, a single few-shot example, a hash that flags drift and a static/dynamic boundary so Anthropic's prompt cache pays for the persona once instead of every turn. The agent loop stays the same. Only the system message changes shape.

Why structure helps

A prompt is code, since it steers behavior, and every sentence in it competes for the model's attention. A long undifferentiated block gets partially ignored, while a short set of labeled sections gets read carefully.

It sounds superstitious but it's observed behavior. Claude and its peers are trained on instructions in familiar forms: sectioned system prompts, user turns that make a plain request, assistant turns that give a plain answer. Echo those forms and the model is less unsure what each piece means, giving fewer hallucinations, tighter output and better tool choice.

The four layers below aren't the only valid structure. They are the smallest one that survives the changes a real agent goes through. Nearly the same anatomy appears in the system prompts the Claude Code CLI builds for itself, and that's no accident: agents that survive real users tend to converge on the same layout.

The layers

Identity is a paragraph about who is speaking, which fixes tone, expertise and stance. A sample: Eugene is a careful research assistant for a Rust project who would rather read the source than guess. No rules and no facts belong in it.

Instructions are behavioral rules, each one a single testable statement: use list_files to find out what's in the project before reading, or don't retry a call that returned an error. They accumulate as the agent grows. Keep them as a list, since bullets are easier to add and remove than paragraphs.

Output constraints describe the shape of the answer and not its content: plain prose with no markdown, a parenthetical citation of the file read, replies under 200 words. They apply no matter what the user asked. Splitting them from instructions lets you change the format without touching behavior, and the reverse.

Examples show the persona at work. A few-shot example does in one line what an instruction needs three for, because it displays good output, citation style included. Examples are part of who the agent is, so they sit in the static prefix next to identity and instructions.

Context holds whatever varies by run: the date, the user's locale, the project root, the skills on offer right now. Because it changes, it can't sit in a static string. It gets its own layer below the cache boundary, rebuilt on every request.

A typed builder in Rust

In Rust the structure becomes a builder where each method stands for a layer. It tracks which layers are static and which are dynamic and renders the boundary between them.

let prompt = SystemPromptBuilder::new()
    .identity("You are Eugene, a careful research assistant who answers \
               questions about a Rust project. You prefer reading the source \
               over guessing.")
    .instruction("Use `list_files` to discover what is in the project before reading.")
    .instruction("Use `read_file` to inspect a specific file. Do not call it on \
                  paths you have not seen listed.")
    .instruction("If a tool returns an error, do not retry the same call.")
    .output_constraints("Answer in plain prose. Cite the file you read in parentheses, \
                         for example: (src/main.rs).")
    .example("What edition does Cargo.toml use?",
             "I'll check Cargo.toml directly. (Cargo.toml) The project uses Rust edition 2024.")
    .context(format!("<env>\ntoday: {today}\nproject_root: {sandbox}\n</env>"))
    .build();

The compiler can't check much here, but readability gains a lot. You can scan the chain and know what the agent will see. The first five methods form the static prefix and the last builds the dynamic suffix. The rendered prefix looks like this:

## Identity

You are Eugene, a careful research assistant ...

## Instructions

- Use `list_files` to discover what is in the project before reading.
- Use `read_file` to inspect a specific file. Do not call it on paths ...
- If a tool returns an error, do not retry the same call.

## Output

Answer in plain prose. Cite the file you read in parentheses ...

## Examples

Example 1:
User: What edition does Cargo.toml use?
Assistant: I'll check Cargo.toml directly. (Cargo.toml) ...

The suffix is shorter and entirely dynamic:

## Context

<env>
today: 2026-05-22
project_root: /Users/me/code/eugene
</env>

The headers carry meaning. "## Instructions" tells the model the bullets underneath are rules, and "## Context" says the lines underneath are facts. Without them the two layers blur, and the model may read facts as commands or the reverse. With headers, tool selection is visibly better.

The XML-style tags around the environment info work the same way. The <env> wrapper is a marker the model has met thousands of times in training, so it reads what's inside as runtime state and not as guidance. Claude Code CLI passes its current directory, git state, operating system and model id into its own prompt in just this way, and it's a convention worth borrowing.

Identity gives voice, not truth

This layer is the one people overwrite most. Beginners fill it with claims, such as an expert Rust engineer with twenty years at major tech companies. Telling a model it's an expert doesn't make it one. Identity controls voice: a careful research assistant hedges before speaking, a fast no-nonsense engineer answers tersely, and a patient teacher explains.

It also sets posture toward uncertainty. A line telling the agent to prefer reading source over guessing is a real lever, and the model does call read_file more often with that sentence present, since it changes which reply feels in character. Treat identity as direction for an actor, not a CV.

Instructions are a list

Every instruction should pass or fail by itself. "Use list_files before reading" is checkable, since the model either called it first or didn't. "Be helpful to the user" can't be checked, so drop it.

Order matters when rules clash, because the model reads top to bottom, with earlier rules anchoring and later ones clarifying. If two rules truly conflict, the prompt is wrong and the model will pick at random. Fix the clash in the prompt, not at inference time.

Three or four rules suit a small agent. Ten is a smell. Twenty is a failure in itself, because the model can't hold them all and quietly drops some. A long list usually means a missing output constraint or context fact that would make several rules unnecessary.

Output constraints separate format from behavior

Take a prompt that says to answer politely in plain prose, cite the file in parentheses and stay under 200 words. It mixes three concerns. Politeness is voice, so it goes in identity. Citation and length belong to output. When someone asks to change the citation format, you should edit one line in one layer without rereading everything else.

This is also where structured output attaches. If a downstream system wants JSON, the output layer says to respond in JSON matching the supplied schema. If a user asks for steps, the layer says to number each one. Behavior keeps doing what it did while format adapts.

Examples show, they don't explain

One example often replaces several instructions:

.example(
    "What edition does Cargo.toml use?",
    "I'll check Cargo.toml directly. (Cargo.toml) The project uses Rust edition 2024.",
)

That single example demonstrates the citation format, shows that a short acknowledgment is fine, sets the expected length and models the link between a tool call and a citation. None of this appears in the instructions, and none needs to.

The risk is overfitting. If every example involves Cargo.toml, the model grows oddly fixated on it. Use one or two examples spanning the real range of inputs. If you can't write a representative one, something else in the prompt is probably under-specified.

Context is rebuilt on every request

Everyone forgets this layer at first. The model has a knowledge cutoff and doesn't know today's date, your project's name or which files exist. It will guess, and the guesses sound plausible while being wrong. "Probably mid-2024" isn't a useful date.

The fix is mechanical: pass the facts in each time.

let today = OffsetDateTime::now_utc()
    .date()
    .format(&Iso8601::DATE)
    .unwrap_or_else(|_| "unknown".into());
let sandbox = sandbox_root()?.display().to_string();

let prompt = system_prompt(&today, &sandbox);

Two facts, about half a kilobyte of tokens, and the agent now knows when and where it is. The cost is negligible and accuracy on date-sensitive or path-sensitive questions improves a lot.

A real agent's context layer grows fast: user and project names, active skills, the page being viewed, recent error messages, the latest commit hash. None of it is opinion or instruction. It's data scoped to one request and supplied by code.

The cache boundary

This is what divides a hobby agent from one that's affordable to run. Anthropic's API offers prompt caching: pass the system field as an array of text blocks rather than a single string, tag a block with cache_control: {"type": "ephemeral"}, and everything through that block is cached for five minutes. Any later request sending the identical prefix hits the cache, and those cached input tokens bill at about a tenth of the usual price.

For an agent running a six-turn loop with a thousand-token persona, that's about a 5x cut in input billing on each turn after the first, plus lower latency. Savings scale with how much of the prompt is stable. The persona nearly always is, and today's date isn't.

let blocks = prompt.into_system_blocks();
// blocks[0] = { type: "text", text: <static prefix>, cache_control: {type: "ephemeral"} }
// blocks[1] = { type: "text", text: <dynamic suffix> }   // no cache_control

The builder splits naturally. Identity, instructions, output constraints and examples are static, context is dynamic, and each half renders on its own with its own headers. Only the first carries the cache marker, so the second gets re-tokenized each turn while the persona stays cached.

That arithmetic is why Claude Code's source has a literal SYSTEM_PROMPT_DYNAMIC_BOUNDARY marker that its cache logic splits on. Anything that shifts between turns, such as mid-session MCP connections, model overrides or language preferences, sits below the line. The static persona above it can be cached across users where the build permits.

Two rules keep the cache warm. The prefix has to be byte-identical between requests, so an extra space, a reordered instruction or a different date format inside identity changes the hash and you pay to create the cache again. And the prefix must reach 1024 tokens before Anthropic will cache it. Small personas gain nothing and large ones gain a lot. If yours is short, add content that earns its place and not filler.

The response includes a receipt:

[turn 0] in=4 cache_read=0 cache_create=1247 out=89
[turn 1] in=3 cache_read=1247 cache_create=0 out=42

Turn zero creates the cache, and every turn after reads from it. Print usage stats while developing, because the cache stays invisible until measured.

Memoizing dynamic sections

The cache boundary saves tokens. A smaller in-process version of the idea saves compute. Some dynamic context is expensive to build, like scanning the filesystem, loading a memory snapshot or calling a remote settings store. Redoing that every turn wastes time and clutters traces.

Memoize each dynamic section by name and recompute only on explicit invalidation, such as a new user request, a /clear or a /compact. Claude Code's source separates sections that may be cached (systemPromptSection) from volatile ones (DANGEROUS_uncachedSystemPromptSection), and the latter demands a reason argument so the author must justify breaking the cache. The loud name is deserved: one volatile section that changes every turn invalidates everything after it for that request.

In a Rust agent the equivalent is a registry of Section { name, compute_fn, cache_break } entries resolved when the prompt is built, with each result keyed to a turn id. For now the eugene-prompt crate stays simpler and recomputes every dynamic section on each build. If one is costly, wrap it in your own cache and pass the value in, since the crate's job is layout and not memoization.

Versioning and regression checks

The hardest prompt bug is the unintended one. Someone edits a sentence to fix a different problem, behavior shifts and breaks a use case nobody was watching, and no test fails because no test exists.

A fingerprint helps. The builder hashes the rendered text and exposes it twice: prefix_fingerprint for the prefix alone, which is what the cache keys on, and fingerprint for the whole prompt.

const EXPECTED_PROMPT_FINGERPRINT: u64 = 0; // set after first successful run

if EXPECTED_PROMPT_FINGERPRINT != 0
    && prompt.fingerprint() != EXPECTED_PROMPT_FINGERPRINT
{
    eprintln!(
        "warning: system prompt fingerprint drifted (was {EXPECTED_PROMPT_FINGERPRINT}, \
         is {}). Update the constant if the change was intentional.",
        prompt.fingerprint()
    );
}

In CI, compare the prefix fingerprint to a constant. When they diverge, CI complains, and you either accept the change (update the constant and tell the team why) or reject it (revert). Either way it's visible. The same fingerprint that guards regressions also shows exactly when the cache prefix changed, which is when your next request misses the cache.

A fingerprint is necessary but not enough. The other half is behavioral testing: fixed inputs, fixed expected behaviors and an evaluator comparing output with expectation. The fingerprint catches the trivial 90% of drift.

The result in Eugene

The loop is untouched. What differs is the structured prompt and how it goes onto the wire:

let prompt = system_prompt(&today, &sandbox);
let system_blocks = prompt.into_system_blocks();

let response = send(&http, &api_key, &system_blocks, &tools, &messages).await?;

With list_files and read_file available, the instructions and the example teach the agent to chain them. Asked what's in the src folder, it lists the directory, chooses a likely file, reads it and replies with a citation. Asked about the edition in Cargo.toml, the agent first says it will open the manifest, then states the edition (2024) with the file name in parentheses. Citation placement is the output layer's work, checking instead of guessing is identity and instructions, and the citation style is the example's doing. Cost is a fraction of what it'd be without the cache boundary.

Takeaways

A hardcoded literal is fine until it isn't. When you have several tools, several output formats, several engineers editing the prompt or any budget expectation, structure pays off. The layers aren't arbitrary: identity is who, instructions are what, output is how, and context is now. If something doesn't fit one of them, the agent's requirements are fuzzy and the structure isn't at fault.

The cache boundary turns the structure into savings. The typed builder is the shape Rust offers for free, and the fingerprint is cheap regression testing. Together they give you a system prompt you can edit on a Friday afternoon without dread.

The two tools here are still ad-hoc functions in a match statement, with no shared interface, no schema generation, no place for retries and no way for a third party to ship a tool the agent can pick up. The natural next step is a Skill trait, a registry that owns the dispatch table and schemars to derive JSON Schema from a Rust struct, so the agent grows by composition instead of accretion.