OpenWiki Road Test: Grounded Claims and a Fully Local Ollama Run

Documentation rots because it's written once. Memory stays useful because it keeps getting overwritten. Most of what passes for docs is really the first kind dressed up as the second, and that's the reason it goes stale.

OpenWiki, a tool from LangChain, is an interesting attempt to change that. Below is an explanation of what it does and how its self-correcting mechanism works, followed by a road test on a completely open-source stack with local models. That run produced four separate failures, an almost comical rescue and a result that was better than expected.

What OpenWiki is

LangChain, the team behind much of the open-source plumbing for AI apps, released OpenWiki as a free command-line utility under the MIT license. It's pointed at a repo, where an agent studies the code and writes interlinked markdown pages into a folder named openwiki/. As commits arrive, the pages are brought up to date.

Humans aren't the main audience here, since the wiki is written for agents. The tool inserts a pointer into AGENTS.md and CLAUDE.md, the files that Claude Code, Codex, Cursor and similar assistants load when they start. Any coding agent opening the repo then finds a ready-made map and skips the usual rediscovery of the architecture. Skimming a summary costs far fewer tokens and minutes than combing through the source again.

Updates come from version control. The tool checks which commits landed since the last run, reads the diffs and rewrites only the affected parts. Hooked up to a scheduled CI job, the wiki refreshes itself daily and opens a pull request just as a human teammate would.

Grounded Claims: facts with evidence attached

Plenty of tools generate documentation. What sets OpenWiki apart is a mechanism called Grounded Claims, and it deserves a careful look.

Each important statement lives in two places. It's prose on a wiki page, and it's also a structured entry in a companion file that points at the precise lines of code backing it up, together with a hash of those lines as they looked at that moment.

Here's one from the test run. A page says events get "persisted to the run_events table before any listeners are invoked". Its companion entry ties that sentence to lines 9 through 26 of a single file. Change those lines and the stored hash no longer fits, so the claim goes stale. Staleness forces the page to be regenerated on the following run, even when the update planner had judged the page fine.

That's the gap between documentation and memory. Documentation waits for someone to notice it's wrong. A claim knows when it's wrong and schedules its own repair. The docs don't merely describe the code, they can be falsified against it.

Where the idea comes from

OpenWiki didn't appear from nowhere. In April 2026, Andrej Karpathy, a founding member of OpenAI, posted a short gist describing an "LLM Wiki" with three layers: immutable raw sources, a markdown wiki that an LLM maintains from them, and a schema describing how the wiki is organized. The argument targeted retrieval-augmented generation, the standard practice of searching documents at question time. Compile knowledge into durable pages once, he said, and it compounds. Fetch fragments every time and it evaporates.

Within days the gist passed 5,000 stars, and LangChain names it as a direct influence. Seen that way, OpenWiki is the sketch made into a product, plus Grounded Claims on top. The output is written in Google's Open Knowledge Format, version 0.2 (the OKF spec lives in Google's knowledge-catalog repo), which is meant to let any agent consume packaged knowledge. Markdown wikis as agent memory are quietly becoming a shared pattern, and OpenWiki is its most complete form so far, having gathered 15,000 GitHub stars in two months while shipping new releases every few days. Its own repository is documented by itself.

The road test: fully open source

Following the official docs, a hosted OpenAI or Anthropic model can produce a wiki in around three minutes at about ten cents, according to community reports. The tougher test is whether everything can run on an open stack with a local model, keeping all code on the machine.

The setup

Four free, open pieces:

• OpenWiki v0.4.0, from a global npm install.
• Ollama for serving models locally. There's no built-in Ollama provider, yet OpenWiki can use any OpenAI-compatible endpoint, which Ollama exposes.
• Two open-weight models: gpt-oss:20b from OpenAI (a 13GB download) at first, then Alibaba's qwen3.5 4B (3.4GB).
• Hardware: a laptop with 24GB of unified memory, mid-range and nowhere near workstation class.

The target was a personal prototype called Hangar, an agent-orchestration server with a web UI. It has 58 TypeScript files spread across a server, a web client and examples. That's real code of modest size, with nothing exotic in it.

The provider configuration is a handful of lines in ~/.openwiki/.env:

OPENWIKI_PROVIDER=openai-compatible
OPENAI_COMPATIBLE_API_KEY=ollama
OPENAI_COMPATIBLE_BASE_URL=http://localhost:11434/v1
OPENWIKI_MODEL_ID=gpt-oss:20b
OPENWIKI_TELEMETRY_DISABLED=1

A .openwikiignore file at the repo root excluded node_modules, build output, lockfiles and images. It acts as a hard boundary on what gets read, and with a local model that matters double, since everything the agent reads has to squeeze through a smaller window.

Then openwiki code --init --print was run, and it failed. Four times in four different ways. Every failure had a lesson the error text never spelled out, since that text barely changed from one to the next.

Failure 1: a window nobody mentioned

Two minutes in, the first run ended with a single line:

Repository planning worker exited without submit_plan.

No trace, no reason. The cause lay well outside OpenWiki. Ollama starts every model with a 4,096-token context window, which is the working memory the model can see at one time. The repository-scan prompt is many times bigger than that. Ollama doesn't raise an error on overflow. It trims the beginning without a word, so the model never read the part explaining that a submit_plan tool existed. ollama ps backed this up by listing the model at CONTEXT 4096.

Fixing it takes a single environment variable set when starting Ollama. Anyone repeating this experiment at home needs that line more than any other:

OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Failure 2: one message, another reason

With the window enlarged, the planner died again, with the same line after the same two minutes. This is where debugging local AI stacks gets truly hard: one error text, several root causes and no telemetry to tell them apart.

The next step was testing the layer underneath. A hand-built request to Ollama's endpoint, carrying a submit_plan tool schema, would show whether the model could make a structured tool call at all:

curl http://localhost:11434/v1/chat/completions \
  -d '{"model":"gpt-oss:20b", "tools":[…submit_plan schema…], …}'

It returned a perfectly formed tool call, and both models did. The plumbing worked in isolation, so the fault lay in the transport between OpenWiki and Ollama. OpenWiki sends non-streaming requests by default, and this pipeline needed streaming. One more line in the .env file:

OPENWIKI_OPENAI_COMPATIBLE_STREAMING=true

The planner passed on the next run.

Failure 3: a strong model that never submitted

Now it got interesting. With planning working, the 20B model started real work. Calls stretched from seconds to over a minute, and after eight minutes a genuine wiki page landed on disk: an accurate description of Hangar's event bus. File paths were correct, the behavior was right (persist first, distribute after), and there was a proper field-reference table. This wasn't boilerplate. The model understood the code.

Then the run died anyway:

/openwiki/architecture/event_bus.md worker exited without submit_page.

The model had finished the difficult writing and then skipped the formal tool call that hands in the page. The pipeline discarded a page that was already on disk. Ability wasn't the problem. Following through at the tail of a long context was.

Failure 4: the small model stumbled too

Next up was qwen3.5 4B, a model with one of the best-tested tool-calling templates in Ollama, now with a 65,536-token window. It progressed further and completed the event bus page together with its claims sidecar, which meant the full pipeline had worked end to end for the first time. Then, 51 minutes in and midway through page two, it died with the familiar message about submit_page.

The cause was now plain. A page worker piles up context while it works: the draft, the evidence it has read and the history of tool calls. At some point the closing instruction drops out of a window that cuts silently. Which model was in use made no difference. The deciding factor was what the serving layer does when memory runs out, and the answer is nothing, with no warning.

The rescue: a Ralph loop

The solution turned out to be silly. Version 0.4.0 saves progress to a durable run file, openwiki/.run.json, so a generation that gets interrupted picks up where it stopped. A tool that crashes every few pages can therefore be fixed by running it again and again until a run exits cleanly.

That approach is called a Ralph loop, after the lovably clueless kid on The Simpsons: an intentionally dumb while-true that repeats the same instruction until the job is finished. Rescuing someone else's product wasn't what anyone expected from it. The entire harness is a few lines of shell:

for i in 1 2 3 4 5 6 7 8; do
  if openwiki code --init --print; then
    echo "COMPLETE after $i iterations"; break
  fi
done

Ollama's window went up once more, to 98,304 tokens, for extra headroom, and then the loop ran.

Six passes and just under three hours later, it was done. Pass one saved four finished pages and pass two saved four more. Passes three to five all got stuck on one difficult page and saved nothing, which is the point where a person watching would likely give up. Pass six got through, and the remaining fifteen pages followed in a single sweep, since index and summary pages are quick to write. The end result was a 23-page wiki with 18 grounded claims, every fact tied to hashed evidence, and an exit code of zero.

Nothing about the model improved. Failures just stopped being expensive, and that only works because the checkpoint keeps finished work safe. Persistence combined with a save file outperforms fragile cleverness. In the finished setup the repo, agent, model server, model, outputs and retry loop all sit inside one machine, and the only line crossing the boundary is the one that never gets used.

An honest comparison

Compare the two routes. The cloud route takes about three minutes and roughly ten cents with no spend cap, needs no special skill and sends the code off the machine. The local route takes about four hours end to end and costs nothing, but it requires diagnosing four layers of failure and building a retry harness, and no byte leaves the computer.

Most of the time, for most people, the local route isn't worth it. Paying forty times the duration, plus the expertise to track down silent failures, to save ten cents is a bad deal. When code isn't allowed to leave the building, though, it's the only choice, and it clearly works.

The more important finding is different. Every failure was shaped by configuration and not by capability. The models understood the code, and the 20B model wrote accurate architecture pages. What broke each time was plumbing: default windows that truncate silently, transports that drop tool calls and context that runs out without warning. "Local models aren't good enough" is the wrong lesson. The right one is that local serving stacks still fail silently and nobody surfaces it. That's an engineering problem and not a problem of model scale, and engineering problems get fixed.

Should you try it?

When code already goes through cloud APIs, go ahead and try it now. The quick version delivers, the pages are good, and nothing else on the market has anything like the claims mechanism. A few warnings from the sharp edges: commit first, because init wipes out an existing wiki; keep an eye on costs with a metered key, since nothing caps spending and reports vary widely; and remember that this is an early 0.x project that changes fast.

For fully private use, it works as of this writing with the fixes above and some patience. Raise Ollama's context window before anything else.

And for anyone who already keeps knowledge in markdown, the wider point is that the industry keeps converging on one answer from every direction: plain text files, maintained by agents and corrected by evidence. The wiki isn't the product. The correction loop is, and OpenWiki is the first tool to ship one with receipts.