Why a Coding Agent's Memory Should Hold Only the Delta, and How to Store It

Picture a personal memory store kept for a coding agent on a single machine. It holds 181 topic files behind a 134-line index, and almost none of it is knowledge in the usual sense. The agent already understands Express middleware, the shape of a Playwright test and what a Dockerfile does, having read far more about all three than any person could. Every line in the store exists for the opposite reason: it marks a spot where one particular world differs from the field average.

Two examples show the idea. A deploy verifier blocks a release whenever a check can't run, and that's intentional. A release train's self-approve gate reads .status rather than .conclusion, a detail that cost an afternoon before it became a single line in a file.

That's the shape of good agent memory. It isn't a copy of what the agent knows. It's the delta, the leftover part. Everyone agrees an agent needs memory, and nobody has specified one, so the rest of this piece takes two steps: why an agent needs memory at all, then how to organise it. The second half breaks into four: staying simple, the demands on each memory unit and on the store, the storage that meets them, and the routine of living with it.

Agent memory isn't human memory

The natural instinct is to reason by analogy. People remember things, so agents should too. The analogy fails at the first step. A person starts empty, so human memory has to hold both the average and the exception. A model arrives with the average pre-loaded, on a scale beyond any single person. Telling it what middleware is adds no information. It adds tokens.

Tokens cost something, and that's where many people go wrong. Context that wasn't needed isn't neutral, it's a net loss. In one study, researchers counted how frequently a model made API calls correctly. When the model received documentation for APIs it already knew well, and the retrieval was imperfect, its rate of correct calls fell by 39 percentage points. Fetching that kind of documentation for rare APIs the model hadn't seen helped instead, adding nine points of accuracy. Same content and same pipeline, yet the effect flipped sign.

Seen this way, a memory file is a diff laid over the model's existing assumptions. Every line that restates a familiar convention fights for attention with the one line that really matters, and attention is what runs short.

Record only what the model couldn't have guessed.

Run real context files through that filter and most of their contents drop out. Naming rules, "composition over inheritance" and the stock folder layout are all average knowledge that a linter handles and the model already holds. What stays is usually awkward stuff: a workaround, an abandoned path, something true in this codebase and nowhere else.

In the sample store, about 3,700 lines long (3,831 once the index is counted), no line says what middleware is or how to structure a test. Lines that slid into explaining a famous tool get removed, since the model knew it already and they burned tokens on every request for nothing.

The cheapest memory is the one you don't need

If memory holds the delta, then its size depends on how far a project strays from the common pattern. Follow the usual architecture and the agent is owed almost nothing, because it knows the territory. Invent a private module system and each of its rules becomes a line that has to be written, kept true and paid for on every request.

No benchmark is needed to feel the effect. In a mainstream stack, an agent simply gets on with the job. In a niche language or a homegrown internal framework, it pauses, guesses or asks questions. To close that distance you carry the context yourself: write it, version it, keep it current for good.

There's no clean A/B test for this claim, and it would be dishonest to pretend otherwise. The research cited here supports the mechanism, while one person's files support only the practice, which isn't proof of the mechanism. The experience is consistent, though: new work in ordinary stacks moves fast, and work in large old systems, where most context was never written down, moves slower, sometimes slower than doing it by hand.

So architecture choices have become memory choices. Pick the boring, widely used pattern and there's nothing extra to maintain forever. Every departure from the common pattern is a line someone must write down and keep accurate, and a custom path costs more on every later request, not just the one that created it.

The hard part is staying small

That's the why. The how begins with an uncomfortable fact: what kills memory systems in production isn't forgetting, it's accumulation.

Usefulness lags behind size. Twice the memory is not twice the value, since clutter accumulates faster than signal, and past a certain point a larger store gives worse answers than a smaller one did, even on the questions it was built for.

Conflict grows faster than size. Any new fact can contradict any existing one, so the number of possible clashes climbs much faster than the number of files. A contradiction also costs far more than a gap. When a fact is missing, the agent is unsure, and unsure is acceptable: it asks, or it fails where people can see. When two facts contradict, the agent is confident and wrong, and since nothing in the loop looks broken, nobody notices.

Worse, most memory products can't see the problem. Vector similarity registers two sentences on the same topic, but not that one says yes while the other says no. So value flattens, conflict keeps rising, and past some size the memory system is making the agent worse.

Memory rarely fails loudly. It fails by turning confidently wrong while every query still succeeds.

The difficulty, then, isn't piling things up, since every product will happily do that. It's staying small while remaining sufficient. That takes deleting, and deleting means judging your own old writing.

People tend to reach for structure first. Faced with a messy pile, they impose a taxonomy of hierarchies, categories, graphs or formal knowledge schemes. That reliably backfires. In one 2026 evaluation, added depth to an on-demand context index was tested with the same model and harness, and accuracy dropped from 0.91 to 0.64. Flat beat hierarchical wherever it was tried. Elaborate structure doesn't cure complexity. It's a second complexity stacked on the first, and now both need maintenance.

Organising memory is therefore less about drawing a better diagram and more about setting a bar that each line must clear, plus a store that makes clearing it cheap.

Four tests for every memory unit

The bar applies to the unit, not the system. On a team, these are also what a reviewer checks.

• Simple and dense. Only seed information. No sentence exists merely to sound complete. A line that doesn't change what the agent would do is decoration, and decoration has a price.
• Non-conflicting. No two facts that can't both be true. This one needs active effort, since conflicts accumulate slowly and never on purpose.
• Measurably useful. Memory doesn't make an agent smarter. It makes the agent quicker and cheaper at the same quality, giving the same answers without the rediscovery. A file that doesn't visibly cut time or tokens on a real task isn't earning its upkeep.
• Classified. A verified fact, an assumption, a preference and a rumour shouldn't look alike. In the sample store the filename carries the kind. Checked facts live in 117 reference_ files, standing rules in 50 feedback_ files, and perishable state in 14 project_ files. A reader knows what a file claims before opening it.

One published result seems to undermine the third test and actually supports it. Researchers at ETH Zurich ran a comparison of repository context files against no files at all. It found no general gain in task success and more than twenty percent added inference cost, and machine-written files scored slightly below having nothing. Read through the delta rule, that's not a counter-result. A file full of average knowledge is precisely what the rule predicts will cost something and gain nothing. The finding doesn't say context files fail, only that files restating the average fail, which describes most of them.

Discipline therefore means less writing and more pruning: every line stays dense, and no pair of lines may disagree.

Addy Osmani came at the same issue from the upkeep side in August, arguing that agent files go stale as models improve (his post is titled as an audit of agent files), so they deserve periodic pruning and rebuilding rather than endless growth. Memory that only expands is on a countdown.

Then slice what survives by task, as your own recall does. What gets loaded to write prose differs from what gets loaded to debug a build.

Nobody needs one memory. What's needed is the smallest correct slice for the task at hand.

Three requirements for the store

Those four tests say nothing about where files live, which is the other half of organising.

A hint comes from what existing tools already share. AGENTS.md is an ordinary markdown file at a predictable path, and over twenty tools read it, even though their makers compete on nearly everything else. Rival vendors have never both implemented the same memory API.

A survey of shipping coding agents (published on Substack) reported seven of them converging on one design without coordinating: a small markdown file always in context, plus a larger archive on disk opened when a task calls for it. None used vector search. Though they don't share a benchmark, all seven meet the same three requirements.

Searchable with nothing installed. "Search" tends to suggest semantic search, which isn't even the common case. Anthropic's own memory tool exposes view, create, replace, insert, delete and rename, and has no query operation at all. The agent lists a directory and reads what's there. Memory that works only when the index is built, the embeddings are fresh and a vendor is up puts a weak link in front of the context. It must keep working when all of that is down.

Changeable while a person watches. Memory that updates itself is where the field is taking damage. An audit of 10,134 automatically extracted entries in production reportedly judged 97.8% of them junk. A separate study had models save their own successful runs back into memory and found results worse than no memory, across all four agents tested. Self-writing memory drifts with confidence, not politely, so any change must be visible as a diff, rejectable and undoable.

Maintainable with tools that are already open. Corruption rarely ends a memory system. Neglect does: the file gets written once, nobody revisits it, and it slowly turns false. According to a write-up of one 2026 study, half of context files never saw an edit after the first commit, and the 6% that kept getting revised were owned by people who reported real benefit from the agent. Maintenance has to happen in the editor, search and version control that people use every day anyway.

That's the full specification. What remains is which storage satisfies it.

Why markdown files in git fit

The case for files isn't new. Writers on Medium have argued "markdown beats vector databases for agent memory" at least six times this past year, and Hacker News threads on it keep producing a wisecrack that boils down to using plain text. The part rarely supplied is the reasoning.

Three vendors selling memory databases published against files within three months, while the two vendors shipping the leading agents had already chosen files. The argument here has nothing to sell, only a folder.

What the field stores

What a system stores determines everything downstream.

• Mem0 stores an LLM-extracted sentence together with its embedding, one row in a vector index. What comes back isn't the original document but an extractor's paraphrase, and it can't be opened.
• Zep keeps a typed graph edge for each fact and follows two timelines per fact, so every one carries a validity window. Fixes work by replacement: the old edge is flagged as dead instead of being edited.
• A vector-native memory startup operated vector archives for years. In early 2026 it switched the default store for its coding agents to a tree of markdown files in a git repo, committing once per edit.
• Anthropic ships a memory tool that is only file operations on a /memories directory, executed by the customer's own client. Its hosted, still-beta Memory Stores API names things by path in filesystem style and resolves concurrent writes with a content hash, which amounts to git's data model with an HTTP skin.
• Consumer assistants hold a summary that users can only view as displayed. A documented machine-readable export doesn't exist for any of them as of September 2026.

Lined up, the real dividing line is whether memory can be read, and the files-versus-databases framing misses it. All of the non-file options have one thing in common: memory can't be diffed between two moments. Corrections happen with a toggle, not a commit.

Each camp ships, and each is partly right

MatrixOrigin claimed in March 2026 that static markdown decays and enforces nothing. Mem0 followed five weeks later with a post called "Your AI Agent's Memory Is Just a File? That's the Problem", saying flat files repeat the pre-database errors of the 1970s: no semantic search, no temporal reasoning, no safe concurrent writes and no forgetting. Zep added "Markdown is not agent memory" in June, with a strong central point: a markdown file shows what was written, never what it replaced or why.

Meanwhile Anthropic's own coding agent keeps its memory as plain markdown on disk, and Windsurf's documentation quietly advises moving anything worth keeping out of its memory store and into markdown under version control. It's worth noticing who is selling what.

Database vendors do have two valid points. Vectors win when question and note share no vocabulary, as with a request for "the thing about feeling burned out" against a note that never says burnout. Only an embedding bridges that, at consistent speed over millions of rows. Grep cannot find something you cannot name.

Graphs win on time. Answering what someone believed last Tuesday, and what replaced it, needs facts stamped with start and end dates, and a similarity score cannot tell which of two clashing facts is current. A third-party rerun on the temporal questions of one long-memory benchmark reportedly gave the graph system 63.8% against 49.0% for the vector one. Files lose on both.

The evidence, with its caveat

Retrieval results in 2026 surprised the field. Agents that explored a plain pile of files with grep, find and file reads outperformed every keyword, embedding and reranking baseline set against them. One agentic-search benchmark found that removing the embedding retriever and letting the agent read files lifted accuracy from 69% to 80%, with nearly a third less cost. Another study on long-term memory questions saw plain keyword search beat semantic search in each inline tool-calling configuration. When results came back as files, however, the order reversed for half the pairs, because the questions rewarded locating exact things such as a date, a count, an identifier or an error string.

The agentic-search paper also carries the caveat, which belongs in plain view: scale the pile from 100,000 to 400,000 documents and the same method slides from 80% to under 40% accuracy while using three times the tool calls. Grep works inside a size envelope and is no law of nature. Within it, grep wins on quality and cost, and beyond it, it collapses.

The honest question is therefore which side of the envelope a system sits on, not which approach wins in general. A store of 181 files is hundreds of times smaller than the floor where trouble reportedly starts. A production memory system at a large company like LinkedIn is not, and nobody is arguing it should use markdown.

Scorecard: files against the alternatives

Against the three requirements, markdown files meet each one at nearly zero cost. Grep serves as the fallback, any change is a diff (a reversible commit inside a repo), and the maintenance tooling is already open on the desktop.

Two more benefits arrive for nothing. Contradictions show up, because two conflicting facts are two lines of text instead of two float arrays, and the stored format doubles as the export format.

The trade-off deserves naming rather than hiding. Files lose to embeddings when fuzzy recall over many documents is needed. The answer is to place an index on top of the files, rebuildable at any time, instead of in place of them. Basic-memory builds that layer on SQLite, memsearch builds it on Milvus, and Karpathy's compounding-wiki idea gets by with just an index page, no embeddings whatsoever. When the index is the storage, inspection, diffing, ownership and portability are given up for recall that was available anyway.

If you can't grep it, diff it and delete a line from it, it isn't your memory. It's somebody's cache of you.

What a running system looks like

Principles are cheap, so here is one working setup.

• Layers of pointers, with no manifesto. The first file the agent opens is 21 lines long. It works as a map with three top-level areas, one line each, pointing at deeper files. One level down, a projects file holds what is genuinely personal: the repo that's live versus the one that died in a rename, the versions everything is pinned to, the communication rules to obey. Twenty-eight such files sit across the projects tree, all hand-written, each loading only if the work heads there.
• A store the agent writes, a human signs off. Next to that is the auto-memory, with 181 topic files behind a 134-line index. Only the index loads when a session starts, and a topic file is opened the moment a task needs it.
• An entry rule. Only things the model got wrong or had no way to know. These are not rules guessed at in advance. They are actual failures noted after they bit. Three real ones: the live web repo sits in a different directory than the old docs say, so confirm the git remote before touching anything; pruning after a deploy is only best-effort since concurrent deploys fight over the lock; avoid em dashes. Nothing here can be inferred from code, and nothing is in a model's weights, which is the whole admission test.
• Grouped by job rather than subject. Design work and deploy work call for different context, and combining them thins out both.
• A separate knowledge layer. Specs, decisions, plans and research live in Obsidian vaults, around seventeen hundred markdown files the agent can read and a human writes. Keeping them apart matters. Agent memory is what the work left behind, while the vault is what was intended. Mix the two and the store begins to misstate what was decided.

Layers for loading, not reasoning

Count the levels above and there are four, which is the depth the earlier 0.91-to-0.64 result warns against. But the variable in that study was depth, and an index as such stayed in every setup. The best configuration was a flat index per task, the agent reads in one pass before deciding what to open, which nearly doubled accuracy over raw navigation at about half the cost. Results fell apart only when the index sprouted branches that the model had to climb down, guessing from labels alone which one held the answer.

This setup's index is the flat sort: a single level, read whole when a session begins. It has 134 lines, roughly one per topic, each naming the files it points to with a short clause of summary. Nothing branches, so nothing needs guessing. It is a small routing table kept by hand, not a taxonomy for the model to reason through, and it may be imperfect: eleven of the 181 files are missing from it. Topic files below are small, flat and read in full, and a layer opens because work moved into its area, not because some query went hunting. That is progressive disclosure.

The test is simple. If the model must guess which branch holds the answer from labels alone, a tree has been built and the 0.64 is coming. If it reads the whole index and opens a file by name, it's a loading order.

Structure exists to decide what to load. It isn't there for the model to reason through.

A safeguard against drift

The agent proposes, the write appears as a diff, and a person accepts it or doesn't. Files that live in repositories bring their own history, which means the question of why the agent believes something can be answered by reading, which no vector store or vendor memory page can offer. A memory nobody reviewed is a rumour with a filename.

Occasionally there is a dull cleanup pass: delete lines that were true in June, files the model has since absorbed, and rules superseded by better ones. Stores shrink more often than people predict.

Everything is plain markdown on local disk and works without the assistant that helped write it. When a stronger one arrives next year, aim it at the same folder and continue.

A check you can run today

The organisation fits on a card: four tests for every line kept, three requirements for where it's kept, and a human between the agent and the file.

To find your own position, open the project's context file, whether it's CLAUDE.md, AGENTS.md or whatever the tool reads, and delete every line the model would have guessed unaided. Cutting half is a realistic outcome. Then give the next real task to the trimmed file and see if anything gets worse. After that, whenever the agent errs on something it had no way of knowing, add a single line and have it reviewed like code. That's the discipline, and it doesn't scale by adding more.

The agent already knows the average. The only job is the delta.