Project Cedar: Testing Whether AI Agents Can Maintain a Company Markdown Wiki

Obsidian can carry context between a person's projects and their coding agents. Plans, decisions, requirements and constraints outlive individual sessions, Claude, Codex or any other agent tool can read the same project files, and a human can inspect and correct those files and switch models without leaving project knowledge stuck in a chat history. This works well inside one person's projects because that person makes most of the decisions and knows how everything is organized.

Inside a company the picture changes. Knowledge is scattered over repositories, tickets, meetings, documents and many different people. Sources contradict one another, and decisions get reversed. A set of meeting notes may describe an idea the team later dropped, whereas an approved architecture document holds the decision that actually stands. So the open question is whether a per-project habit can scale to a whole organization, and whether shared Markdown could serve as an agreement between employees and agents.

The LLM Wiki idea

The pattern comes from a recent proposal by Andrej Karpathy, who named it the LLM Wiki. It has three layers:

• Raw sources that are never modified
• A Markdown wiki, interlinked and maintained by an agent
• A schema that tells the agent how to organize and maintain that wiki

Three operations act on those layers. Ingest brings in new sources. Query draws on the maintained knowledge. Lint checks the wiki for gaps, stale material, conflicts and poor organization.

Possible sources named in the proposal include Slack conversations, meetings, project documents and customer calls, which the agent compiles into knowledge that persists from one task to the next. That differs from a retrieval system hunting for a few relevant fragments each time somebody asks a question. Here the agent keeps an artifact that evolves as new information comes in.

A first architecture that turned out wrong

A natural first reading of the LLM Wiki suggests two separate knowledge layers: a wiki generated by the agent, and Obsidian files placed on top of it. That was unnecessary, since it duplicates content and creates a synchronization problem.

The generated Markdown can itself be the wiki, and Obsidian is merely one window onto it, a place to browse, read and fix pages. Markdown is the shared contract, and Obsidian is the human workbench. Swap the window for another one and the knowledge underneath stays as it was, which is how a real wiki behaves. An organization might publish approved pages through SharePoint, Confluence or an internal portal, while engineers and agents keep working from the maintained source.

The proposed flow is as follows. Company sources feed immutable raw evidence. A wiki-maintainer agent uses the schema and governance rules to propose changes, and approved changes enter a Git-backed Markdown wiki after review. A service layer handles authentication, permissions, search and audit history. People reach the knowledge through Obsidian or a company wiki, and agents use the same service layer with their own identities and permissions.

Company knowledge needs more than links between notes

A personal vault works partly because its owner remembers the context around the files: which meeting note was speculative, when a requirement changed even though not every related page was updated, and which two files describe one decision under different names. A company can't rely on one person carrying all of that.

Its knowledge system has to tell apart:

• a mere proposal versus a decision that was approved
• what holds today versus what is only history
• an official policy versus a passing remark in a meeting
• a contradiction already settled versus one still open
• a gap in the record versus a choice nobody has made yet

Beyond that come named owners, review steps, citations back to sources, rules for freshness, access boundaries and a trail showing which changes an agent made. An individual's knowledge base can live with ambiguity because the owner remembers what's missing. A shared company layer must never let an offhand remark quietly harden into an approved decision.

That's why the schema matters. It does more than name folders and files. Through it the agent learns the way this company encodes what it knows. Metadata on a single wiki page could look like this:

status: verified
owner: platform-team
last-reviewed: 2026-08-02
review-after: 2026-11-02
sources:
  - adr-014
  - meeting-2026-07-29
supersedes:
  - authentication-proposal-v1

Getting an agent to emit Markdown is easy. Defining what it may treat as organizational truth is the hard part.

A fictional company as test bed

A public software repository doesn't carry enough organizational complexity for this question. A tidy repository can exercise ingestion, linking and basic questions, but it doesn't naturally contain shifting client requirements, stakeholder disagreements, informal notes, superseded architecture or sources of unequal authority.

To get around that, the experiment invented Project Cedar: an imaginary engagement in which Northstar AI Consulting works for Prairie Health Network. Everything about it is fake. The consultancy is building a system that converts policy documents and recordings of quality reviews into draft summaries linked to their sources, plus internal search. The scenario echoes real concerns such as confidential data, local versus cloud inference, human review, model evaluation, architecture choices, traceability to sources and requirements that keep moving.

The synthetic corpus holds 25 initial Markdown artifacts. In real life these would come in many formats, but the test accepts that compromise. Material includes background on the companies, the client's policies, requirements, architecture decisions, meeting notes, model evaluations, records of implementation work, status reports and risk entries.

A second set of eight later artifacts changes the project. Security requirements tighten, a model evaluation invalidates an earlier assumption, acceptance criteria shift, and a new architecture decision supersedes the original design while an informal meeting note keeps describing parts of the old one. These changes test whether an application can evolve stored knowledge.

Two further files exist for the evaluator only. One is a knowledge map recording the facts, decisions, conflicts and relationships deliberately planted in the corpus, which acts as the source of truth. The other is a set of evaluation questions with expected answers and required source evidence, used to find out whether knowledge was updated correctly. The wiki-maintainer agent must never see them during the build, so the test shows whether it truly reconstructed what the fictional company knew and didn't merely write a convincing-sounding wiki.

An experiment that can fail

The planned protocol has two stages.

In stage A the agent receives the initial corpus and builds the first Markdown wiki. It has to say what the company is building, what constraints bind the system, the reasoning behind the original architecture, which risks remain open, which decisions are still unsettled and which sources support each answer. The wiki and its answers are committed before any later evidence appears.

Stage B releases the change corpus. Now the agent has to revise every page that is affected, keep superseded decisions around as history, make open disagreements visible, and resist the temptation to treat the most recent document as the final word. The Git diff then becomes evidence too: it reveals whether existing knowledge was kept coherent or merely replaced by one more summary.

That was the full protocol. The first real implementation took a different path, because it was built as part of another experiment in which an AI agent developed a software project during a long drive.

The first build

The first Cedar build grew out of that driving experiment, whose details are beside the point. What counts is what the agent delivered. Its inputs were part of the Cedar corpus, instructions for the project and an evaluation set kept out of its sight. The ChatGPT Project in use capped uploads at 25 files, so only eight of the 25 initial artifacts reached the agent, and the change corpus was held back to be supplied later as a test of whether it could carry changes into the docs.

What came back was a local command-line program written in Python. It read Markdown sources and their frontmatter, pulled out evidence tagged with the exact lines it came from, and tracked how authoritative a source is separately from when it was published. Documents with explicit versions were labeled current or historical. The tool could also write a Markdown knowledge report, fetch evidence relevant to a question, and audit the wiki for structural faults. Source documents sat apart from the generated wiki/ folder, and the repository kept maintainer instructions, run records, evaluation material and output in their own places. All 26 tests passed. It was a functioning first version, although the shrunken corpus meant only a slice of the Cedar protocol could be exercised.

Could it maintain what the company knew?

On the held-back evaluation the system scored 50%. It was good at keeping evidence: source references and line-level provenance showed exactly where extracted facts originated. The difficulty lay in finding and reading the right evidence when a realistic question came in.

A real company question can demand merging facts from several files, favoring an approved decision over a casual note, noticing that a newer document may carry less authority, telling an open question apart from a gap in the record, and keeping uncertainty alive when nothing final has been decided. Evidence preservation came out stronger than evidence selection. The structural audit caught mechanical flaws like absent metadata or a broken layout, but semantic conflicts slipped by more easily, since two documents could hold incompatible positions in unrelated wording and the system didn't always see the clash.

In short, citations were kept without any grasp of which ones carry weight. For company knowledge that is a real weakness, because a well-cited answer can still be wrong when it rests on a low-authority source, overlooks a later approved decision, or glues together facts that don't belong side by side.

The incomplete corpus also limits the conclusion. The agent never received the change corpus, so this run didn't test the hardest part of the original design: updating current knowledge after authoritative sources change while keeping the old state as history. The partial answer is that an agent can build a structured, source-linked representation of company knowledge, but there isn't yet enough evidence that it can maintain it reliably as the company changes.

The wiki needs a company interface

The experiment also sharpened a point about architecture. Obsidian is handy for looking over generated Markdown, following links, checking metadata and fixing a page manually. That suffices for a personal setup or prototype, yet no company can hand every employee and agent direct access to one vault. Between the Markdown wiki and its users a real system needs an intermediate layer, which would cover:

• sign-in handled by the company's identity provider
• access rules tied to users, teams, projects and roles
• distinct identities with their own rights for each agent
• a review and sign-off step for every edit an agent proposes
• search results filtered by what the asker may see
• comments, notifications and ownership
• simultaneous edits and version clashes
• an audit trail naming who or what altered each page
• retention and deletion policies
• a front end that nontechnical staff can actually use

Authentication establishes who is asking, while authorization decides what that identity may reach, in terms of sources, pages and operations. Agents fall under the same split. A finance team's agent shouldn't get at clinical records just because both file sets live in one wiki, so agents need their own service identities, limited permissions and the audit obligations humans have. Permissions must also follow information as an agent reshapes it: a project summary drawn from a public architecture document and a restricted security review could expose restricted details even if the page looks harmless.

Obsidian can remain the workbench for maintainers and technical reviewers, while most employees reach the knowledge through a wiki website or internal portal that reads the same maintained knowledge via a controlled service and doesn't make another copy. The architecture then has four parts:

1. Evidence layer: immutable company sources with their original permissions
2. Knowledge layer: the Git-backed Markdown wiki and its source relationships
3. Control layer: identity, permissions, review, audit history and search
4. Interface layer: Obsidian for the people maintaining the wiki, a company wiki site for employees, and restricted APIs for agents

A company interface doesn't duplicate knowledge. It controls how people and agents interact with it.

What to test next

The next run should use the full Cedar corpus: ingest all 25 initial artifacts, score the initial wiki and commit it, then introduce the eight change artifacts and check whether the agent updates every affected page, marks old decisions as superseded, retains the history worth keeping, follows the source-authority rules, brings unresolved disagreements into view, refrains from making up decisions, and produces a Git diff a person can review.

Retrieval should also be scored apart from maintenance. A system might hold the right knowledge yet retrieve wrong evidence for a question, or answer correctly by searching raw sources while its maintained wiki goes stale, and those failures call for different evaluations. A later round should add user and agent identities to the corpus, so questions can check that an authorized reviewer gets the correct answer while another user is denied restricted information. That brings the experiment closer to a real company deployment.

Can agents maintain what a company knows?

The first Cedar build supports only a qualified answer. The agent delivered a working system that took in documents resembling a company's, kept source references, labeled document versions, produced a Markdown wiki and audited its structure. The evaluation then showed those skills fall short, since maintaining company knowledge also calls for picking evidence well, reasoning about authority, spotting conflicts, handling uncertainty, review and access control.

Markdown can serve as the common artifact, but a company still needs a governed interface around it. Both people and agents need identities, permissions have to travel from the original source into generated pages, agent edits need review, and each accepted change needs a history. Producing the wiki was the easy part. What takes real effort is deciding what the company accepts as knowledge, who may see it and how an agent is permitted to change it.