Beyond One Retrieval Path: Canonical Knowledge, RAG and Live Queries for Enterprise Agents

A headline claiming that a new open format is replacing the vector database invites skepticism. The part worth keeping from that claim is not the replacement story but a sharper architectural question: are too many different kinds of enterprise knowledge being pushed through one and the same retrieval mechanism?

Inside a company, "knowledge" covers very different things. A support ticket counts. So does a 60-page design document, a database schema, a metric definition or an incident runbook. The current number of delayed shipments counts too. These things behave differently, so it is odd that an AI agent should reach all of them in the same way.

The useful framing is not OKF versus RAG, and it is not that vector databases are dead. It is a practical triage with three buckets: knowledge to retrieve, knowledge to hold as canonical, and knowledge to query live.

An ordinary enterprise problem

Picture a fictional logistics SaaS company with about 120 engineers. Like many firms experimenting with agents, it has connected internal docs, support workflows, operational systems, metrics and runbooks to an agent platform. For a large share of questions, that works fine.

Then someone asks for the on-time delivery rate, and the system turns up three definitions:

• The data team counts shipments that arrive by the promised date.
• The product dashboard applies a somewhat different calculation.
• Customer-facing documentation leaves out cancelled orders and returns.

None of these is nonsense. Different teams wrote them for different reasons. The trouble begins when an agent has to decide which one is authoritative. A vector database can report what is semantically close. It has no way of knowing which version the company has declared to be the definition. That is a separate problem.

Relevance is not authority

A standard RAG flow returns every document that matches. If three documents describe one metric three ways, all three come back. Better ranking, extra metadata, reranking, a bigger top_k or a larger context window can all help. None of them answers the organizational question of which definition is the official one.

So the real weakness is not simply that RAG hallucinates. Relevance and authority are different properties, and enterprise agents increasingly need both.

When structure is the meaning

Another reason not all knowledge fits one retrieval pipeline concerns shape. Take a table with ten rows and seven columns. Its meaning does not live in the words alone. It comes from how headers, row labels, values, units and surrounding context relate to each other.

Cut that table into 256-token chunks and one fragment might hold only a pair of headers, say region and revenue, while another holds East 2.1M and West 1.8M. The model may rebuild the relationship correctly, or it may not.

For prose, chunking is usually what is wanted. For structured knowledge it becomes a strange detour:

1. begin with structure
2. flatten it into text
3. split the text into fragments
4. embed the fragments
5. retrieve a handful
6. ask the model to rebuild the original structure

At some point the question is whether the thing should have been fragmented at all. The same applies to database schemas, join paths, configuration rules, metric formulas, SLAs and operating procedures. Structured content is not forbidden in RAG. The point is that structure deserves special handling when losing it changes the meaning.

The vector database was never the source of truth

The fictional company had another familiar problem: knowledge was scattered. Schemas were described in database comments, GitHub wikis, Confluence and migration files. Metric definitions sat in analytics dashboards, product specs, data-team docs and old slide decks. Runbooks were spread across Notion, incident tickets, Slack and GitHub. Some join logic lived mostly in the head of whoever had been on the project longest.

Building a vector database on top of that did not produce a source of truth. It produced a searchable index of existing inconsistency. That has value, but it is not governance.

What OKF adds

What stands out about the Open Knowledge Format is not Markdown, which already exists. It is the idea of making enterprise knowledge explicitly machine-readable while keeping it pleasant for humans. In concept, a concept file starts with structured metadata:

type: metric
name: on_time_delivery_rate
owner: data-platform
reviewer: product
trust_tier: verified
stale_after: 1 day
related:
- shipments
- sla_definition

and continues with an explanation written for people:

# On-Time Delivery Rate
Percentage of shipments delivered on or before the current
promised delivery date.
## Excludes
- cancelled shipments
- test orders
- returns

The agent no longer receives merely some text about on-time delivery. It can also learn who owns the definition, whether it is current, whether it has been verified, what it depends on and which other concepts relate to it. The question changes from which chunk looks most similar to which concept is canonical and whether it is safe to use.

OKF does not replace RAG

Here the conclusion departs from the headline. A more accurate picture is a progression: canonical knowledge lives in OKF, exploratory knowledge in RAG, and current state in live systems.

Canonical knowledge covers database schemas, metric definitions, SLAs, business rules, runbooks and ownership information. The question asked of it is usually what the official, current definition is.

Exploratory knowledge covers historical reports, incident history, support conversations, design documents, case studies and customer feedback. The question here is what information is relevant to what is being looked for. This is exactly where RAG is strong.

Live state covers today's on-time rate, a current customer balance, active incidents, open orders and inventory levels. The question is what is true right now, and a vector index is probably the wrong place to answer it. The operational system that owns the number should supply it.

Some data should not be retrieved

This is where the architecture gets more interesting. Suppose a user asks for the top five customers by revenue this quarter. One design pulls documents and data into the index and lets the model sort it out. That is possible, but a better flow exists: the agent looks up the canonical meaning of revenue, builds a query, and runs it against the system of record. For example:

SELECT
customer_name,
SUM(revenue) AS total_revenue
FROM sales_data
WHERE quarter = 'Q3_2026'
GROUP BY customer_name
ORDER BY total_revenue DESC
LIMIT 5;

The agent now gets two things from two sources. Meaning comes from the canonical knowledge layer. Current values come from the system that actually owns the data. The agent is no longer retrieving data. It retrieves structure, then queries the live system.

One agent, three knowledge paths

With that distinction in view, a few sample questions route differently:

• What does on-time delivery mean? Consult the canonical metric definition.
• What is today's on-time rate? Take the calculation logic from the canonical definition, then query the live warehouse.
• Why did the rate fall last quarter? Pair the canonical definition with historical warehouse data, and possibly RAG across incident reports and carrier notes.
• What happened with Customer X in Q1? Run a search over past tickets, notes and reports.
• A database outage is under way, so what now? Use the canonical runbook.

The agent is the same and the paths differ. That is a more productive thing to discuss than whether vector databases are being replaced.

The router moves to the center

With several knowledge paths, something must choose among them. A first router can be deliberately simple.

class HybridKnowledgeRouter:
    def __init__(self, okf, rag):
        self.okf = okf
        self.rag = rag

    def classify(self, query: str) -> str:
        q = query.lower()

        if any(word in q for word in [
            "schema", "column", "definition",
            "metric", "sla", "runbook"
        ]):
            return "canonical"

        if any(word in q for word in [
            "history", "report", "example",
            "case study", "what happened"
        ]):
            return "exploratory"

        if any(word in q for word in [
            "today", "current", "now",
            "latest value", "how many"
        ]):
            return "live"

        return "hybrid"

Its context method follows:

def get_context(self, query: str):
    query_type = self.classify(query)

    if query_type == "canonical":
        return self.okf.lookup(query)

    if query_type == "exploratory":
        return self.rag.search(query)

    if query_type == "live":
        concept = self.okf.lookup_relevant_schema(query)
        return {
            "concept": concept,
            "requires_live_query": True
        }

    return {
        "okf": self.okf.search(query),
        "rag": self.rag.search(query)
    }

This is not production-grade classification, which is acceptable. The first goal is only to find out whether the separation is useful.

Why not just BM25 or better metadata?

Before adding another mechanism, the idea deserves a proper challenge.

BM25. For structured text it can be excellent. Exact field names matter, keywords matter and deterministic ranking helps. For schema-like queries it may beat embeddings. But it leaves the underlying architecture unchanged. A table placed in a search index must still be represented and retrieved somehow, and the result is better fragments, which are still fragments. BM25 improves retrieval. It does not create authority, ownership, freshness or canonical semantics.

More RAG metadata. This is also reasonable. Fields such as the following can be attached, with aggressive filtering on top:

document_type
owner
version
effective_date
trust_tier
source

That may be enough for many systems. But once metadata is the mechanism telling the agent what is authoritative, current, related and safe, a structured knowledge layer is already being built. By then, stating the concept outright is tidier than keeping it as an oddball document buried in a general-purpose index.

Live queries bring a safety problem

The moment an LLM is allowed to construct queries against operational systems, the stakes rise sharply. Nobody wants this:

DROP TABLE shipments;

Nor this:

SELECT * FROM shipments;

run against a table of 50 million rows because the model got curious. The better stance is not to give the agent database access. It is to give the agent a constrained query capability. A schema concept can describe the permitted shape of a query:

data_source: analytics.shipments
allowed_columns:
  - shipment_id
  - customer_id
  - carrier_id
  - status
  - promised_delivery_date
  - actual_delivery_date

required_filters:
  - tenant_id

max_result_rows: 1000
access_mode: read_only

The generated query is then validated against that policy:

def validate_query(query, policy):
    checks = [
        uses_allowed_table(query, policy),
        selects_allowed_columns(query, policy),
        contains_required_filters(query, policy),
        respects_row_limit(query, policy),
        is_read_only(query),
    ]

    if not all(checks):
        raise UnsafeQueryError()

    return True

In production, further steps belong here: separate execution credentials, tenant isolation enforced outside the model, database-level permissions, row and cost limits, and logging of every query that runs. The governing principle is that the model proposes and infrastructure decides whether a proposal may run.

Canonical knowledge needs an owner

A single source of truth sounds wonderful until nobody updates it. Ownership and freshness matter more than the file format. A concept might carry fields like these:

owner: data-platform@company.io
reviewer: product-analytics@company.io
stale_after: 7 days

Whenever a metric definition is edited, the matching knowledge update belongs in that same piece of work. When a schema changes, its canonical form should be regenerated or reviewed. People should not retype what the database already knows, so machine-known facts are generated:

for table in database.tables():
  generated = generate_okf_schema(table)
  save(generated)

Humans then review what needs judgment: business meaning, ownership, sensitive columns, allowed usage, relationships and caveats. Skip this, and the single source of truth turns into yet another repository that has gone stale.

Versioning matters more than it seems

Canonical does not mean timeless. Imagine the company changes on-time delivery from on-time shipments divided by total ordered to on-time shipments divided by total delivered. If the old definition is simply overwritten, an agent answering a historical question may apply today's logic to last year's report. Concepts therefore need versions or effective dates:

versions:
  - version: "2.0"
    active: true
    effective_from: 2026-06-10
    change: "Denominator changed to delivered shipments"

  - version: "1.0"
    active: false
    effective_from: 2025-01-01
    effective_to: 2026-06-09

Two questions now get different answers. A question about the definition of OTD uses the current version, while a question about why the Q1 2026 report calculated it differently uses the version active at that time. Canonical does not mean immutable. It means controlled.

When RAG and canonical knowledge disagree

Sooner or later the agent will retrieve an old document stating that OTD is total on-time shipments over total ordered shipments, while the canonical concept says on-time delivered over total delivered. The system needs an explicit precedence rule, and for a contradiction a response such as:

if contradiction(okf_context, rag_context):
  log_conflict(…)
  prefer(okf_context)

It can even tell the user: an older document conflicts with the current metric definition, and the current verified definition was used. The agent is then doing more than answering. It is uncovering knowledge debt, which may be among the more valuable side effects of the design.

Measuring more than the final answer

Tracking only a hallucination rate is not enough in this architecture. How the system reached an answer should be measured as well, for example:

% queries routed to correct source
% canonical lookups with stale concepts
% live queries rejected by policy
% RAG/OKF conflicts detected
% concepts past stale_after
% concepts with no active owne

A wrong final answer can come from very different failures. The model may have reasoned badly. The router may have chosen RAG where canonical knowledge was needed. The canonical concept may have been stale. The live query may have been right while the metric definition was wrong. These are different engineering problems, and observability should keep them apart.

When this is overengineering

A five-person engineering team with only a few internal documents likely has no use for a knowledge-governance platform. If the use case is mostly finding relevant information across thousands of documents, RAG may suffice. If knowledge rarely changes, freshness machinery may not pay off. If nobody owns documentation today, adding YAML frontmatter will not conjure an ownership culture.

The pattern becomes attractive when several of these hold:

• several teams each define the same concepts
• agents carry out operational actions
• schemas and metrics are modified frequently
• incorrect knowledge costs the business something real
• definitions require owners and an audit trail
• live data must be paired with canonical semantics

The more that apply, the more the split pays off.

Where MCP fits

When agents draw on several knowledge and data sources, MCP becomes a natural integration point. An agent might see capabilities such as:

get_canonical_metric_definition
get_schema
search_historical_documents
query_warehouse
get_runbook

The router picks the appropriate one. OKF describes concepts and constraints, MCP exposes tools and resources, live systems stay the owners of current data, and RAG remains the discovery layer for unstructured history. These responsibilities are best left separate unless there is a strong reason to merge them.

A sensible first build

Starting with 300 concepts would be a mistake. Ten is better, specifically ten concepts that people already argue about. For instance:

3 metrics
2 schemas
2 SLAs
2 runbooks
1 business rule

Then measure a few things. Can the agent find the canonical concept? Does it know when that concept is stale? Does routing pick the right path? Can live queries run safely? Do people actually maintain the concepts? Good results justify widening the scope. Poor ones justify stopping. An architecture experiment ought to be able to fail without costing much.

The larger lesson

The most useful takeaway is not a new file format. It is that enterprise knowledge has different modes of truth. Some is authoritative, some is exploratory and some is current state, and each mode deserves its own access pattern. A handy mapping pairs each question with a mode:

• Official definition: canonical knowledge.
• Relevant information: retrieval.
• Current truth: a live query.
• What the agent should do: combine all of the above under policy.

RAG stays valuable, vector databases stay valuable, and OKF can be valuable too. The architectural mistake is assuming any one of them should own every question. The future is not about replacing RAG. It is about knowing when RAG is the right tool and when it is not. Once agents take real actions inside companies, choosing the source of truth stops being a knowledge-management matter and becomes an architectural decision.