CodeBuddy: Smarter Context Retrieval for AI Coding Agents

The overlooked pain point in agentic coding

The excitement around AI coding agents is well deserved — Claude, Codex, Cursor, and the rest have become genuinely useful tools. But spend more than a week pointing one at a large, real-world codebase, and a familiar problem tends to surface.

The agent itself isn't the limiting factor. Model capability isn't what breaks down. What breaks down is context.

Two recurring failure patterns show up again and again:

1. Starving the agent of context — you hand it a task, and it has no idea what conventions your project follows, no memory of a similar bug that was fixed and then rolled back, no sense of which files depend on the one it's about to touch. The result is an edit that looks reasonable in isolation but is wrong for the system as a whole.
2. Drowning the agent in context — to sidestep that first problem, you feed it the entire repository, or let it search through everything indiscriminately. Now each task takes longer, costs more, and the prompt is full of noise. The model burns its attention budget on files that don't matter instead of the handful that do.

Neither of these is really about model intelligence. They're both symptoms of poor context engineering. That's the actual problem CodeBuddy sets out to fix.

Born as a workaround, not a planned product

Long before CodeBuddy existed as a tool, its core idea was already being practiced by hand.

Whenever a real change needed to be made in a project using Claude, the routine was always the same: track down the relevant functions, find the tests that covered that part of the code, pull up any notes explaining why things were built that way, and paste all of it into the conversation before even describing the change. It worked, but it was tedious, repetitive, and relied entirely on remembering the codebase — memory that inevitably gets worse as a project grows larger.

Eventually the obvious question came up: why was a person acting as the context-retrieval mechanism? That's a mechanical, repeatable task. It belongs in automation, not in someone's head.

That's really where CodeBuddy came from — not a decision to build "an AI dev tool," but a decision to stop performing that lookup manually.

The false shortcut: avoiding Graphify

Automating that retrieval step meant deciding how CodeBuddy would understand the shape of a project — which files rely on which, what calls what, and where the real architectural boundaries sit.

A separate project, Graphify, already tackled this by constructing an actual dependency graph of a codebase, mapping real relationships between files instead of inferring them from naming patterns or proximity. Bringing it in as a dependency felt like unnecessary complexity — one more moving part, one more install step, one more potential point of failure. The initial plan was to skip it entirely and have CodeBuddy build its own lightweight architectural index instead, using symbol extraction, file co-occurrence, and a handful of heuristics — just enough to nudge the agent in a reasonable direction.

That approach fell short.

The lightweight index could show what was located near what, but it couldn't reliably explain why two files were actually connected, and it couldn't trace a dependency chain three or four steps deep — exactly the kind of information that matters most before touching code with a wide blast radius. Repeatedly, the agent would make a change that looked safe on the surface but broke something two layers away, simply because the index hadn't captured that relationship with enough precision.

That led to a reversal. Rather than treating Graphify as optional overhead to be designed around, it became a core part of the system: CodeBuddy still works on its own, using its internal index, for anyone who prefers not to add extra setup. But if Graphify is installed, CodeBuddy defers to its graph as the authoritative source for architectural relationships instead of relying on guesswork.

That change of direction is the real story behind the current workflow — not a new feature bolted on, but an acknowledgment that the simpler design was actually worse, followed by rebuilding around the very dependency that had originally been avoided.

What CodeBuddy looks like in practice today

1. Getting set up

npm install -g @ayushkumar320/codebuddy
codebuddy

Running codebuddy inside a project handles the unglamorous but essential setup steps:

• Connects to a local PostgreSQL database
• Applies whatever schema and setup that specific project needs
• Generates a private configuration file scoped to that project
• Integrates with Claude and Codex, adding instructions that tell the agent how to make use of CodeBuddy's tools
• Prompts you to decide whether to turn on Graphify

Choosing to enable Graphify connects its MCP server, after which running /graphify . once builds the initial architectural map for the project. Declining it means CodeBuddy falls back to its own lightweight index — the tool still functions fully, but the resulting architectural picture is less precise.

2. The core loop: context_pack

This is where the real work happens. When you hand the agent a task — say, "add OAuth login" — it doesn't start browsing files at random or swallowing the entire repository. Instead it calls CodeBuddy's context_pack tool, which puts together a tightly scoped bundle containing:

• The files that actually matter for the task
• The specific functions and classes involved, rather than whole files whenever that can be avoided
• Tests that already exercise that area of code
• Project-specific conventions or rules that apply here
• Past bugs or regressions tied to that section of the code
• Any files that would be affected once the targeted ones change
• Existing plans or decisions relevant to the task
• The architectural relationships surrounding those files, pulled from Graphify when it's active, or from the internal index when it isn't

The result is a small, dense, on-topic package instead of a sprawling, thin one. That distinction is what separates genuinely context-aware behavior from something that merely looks context-aware.

3. The agent implements the change

Armed with that package, Claude or Codex has enough to reason about downstream effects rather than just the immediate diff — things like which callers depend on this function, which tests need to keep passing, and whether this exact approach was already attempted and rolled back before.

4. CodeBuddy remembers

Once the task wraps up, the agent can write what it learned back into CodeBuddy: decisions it made, rules it uncovered along the way, regressions it ran into, approaches that turned out to work. That knowledge lives locally, split across Postgres and Markdown memory, and gets folded into the context package for the next related task. Instead of resetting to zero every session, the system keeps getting more precise.

5. PRs get checked automatically

A GitHub workflow can automatically report on each pull request, covering:

• How risky the change appears to be
• Where test coverage is missing
• What the change touches architecturally
• Whether the implementation drifted from the original plan
• Whether verification actually succeeded

Why this matters more than it might seem

It's tempting to file this under "another dev tool wrapped around Claude." That framing misses a few things:

It addresses the real bottleneck. Models keep getting more capable at a fast clip. The quality of the context feeding them hasn't kept up at the tooling level — most setups still default to reading entire files or grepping and hoping for the best. CodeBuddy is built on the premise that the next meaningful gains in agentic coding will come from improving what feeds the model, not from the model itself.

It builds on itself over time. A context window has no memory by default — every session starts fresh unless the surrounding system deliberately preserves state. Because CodeBuddy keeps track of decisions, rules, and regressions, the tenth task on a given codebase should go more smoothly than the first, without you needing to re-explain the same things.

It's upfront about the tradeoffs rather than papering over them. Making Graphify optional is a direct response to an earlier attempt to eliminate that tradeoff altogether, which didn't hold up. You get to pick: no extra setup and a serviceable lightweight index, or a genuinely precise architecture graph if you're willing to add one more component.

Everything stays on your machine. PostgreSQL, the Markdown memory store, the internal index, and Graphify's graph all run locally. Credentials sit in a private config file. Improving the agent's context doesn't require shipping your codebase off anywhere new.

It fits into tools people are already using. This isn't a new editor or a new agent to learn — it plugs directly into Claude and Codex, meeting you where you already work, and simply makes them more effective at understanding the codebase in front of them.

Where this goes from here

This is still an early-stage project under active iteration, and the memory system in particular has the most obvious room to improve — right now it behaves more like structured notes than anything resembling retrieval-ranked memory. If you're using AI coding agents on a real codebase and running into the "it doesn't understand my project" wall, feedback is genuinely welcome:

npm install -g @ayushkumar320/codebuddy

If you give it a try, or you've tackled this problem a different way in your own setup, sharing that experience would be worthwhile.

CodeBuddy is a context-management layer for AI coding agents such as Claude and Codex. It prepares focused, relevant context instead of forcing full-repository reads, and can optionally integrate with Graphify for a more detailed architecture map. It's available on npm as @ayushkumar320/codebuddy.