What a Schema Can't Tell Your Agent: Open Knowledge Format on BigQuery

The prompt file nobody owns

Connect an AI agent to a data warehouse and a familiar artifact shows up sooner or later. At first it's three or four tidy lines in the system prompt: leave refunds out of revenue, the fiscal year opens in April, ignore internal test accounts. Months go by and the file balloons to some four hundred lines. A single engineer is brave enough to edit it, and the analyst who actually owns the metric definitions has never even seen it.

The sheer size is the lesser problem. What hurts is that nobody can maintain the thing. A prompt string has no named owner, no review process and no expiry date. If Finance publishes a new revenue policy in January, the engineer never hears of it, the analyst has no way to change the code, and the prompt never lapses by itself.

Under that kind of friction, knowledge goes stale or never gets finished, so the agent runs on partial definitions and unspoken assumptions. And gaps don't make agents fail visibly. No crash, no SQL syntax error, no empty result comes back. The agent inspects the raw schema, guesses whatever business rules are absent, and hands over a neat, believable figure that ends up on an executive dashboard. A maintenance annoyance quietly becomes an accuracy problem.

Open Knowledge Format (OKF) targets exactly that gap. Business rules stop living in prompt strings or opaque vector stores and become plain Markdown files with YAML frontmatter stored in Git. Ownership becomes explicit, changes go through pull requests, and expiration dates are visible to analysts and engineers alike.

Skeptics argue that stronger models and bigger context windows make this pointless, since the agent can be handed the raw schema and docs and left to infer definitions itself. A compact experiment tested that, putting numbers on the price of incomplete knowledge and on what OKF grounding changes. It used two BigQuery tables, a single business question, and an agent built with Google's Agent Development Kit (ADK) under several knowledge setups.

The test scenario

The setting is an online marketplace with an orders table and a payments table, and a question a product manager could ask on any Monday morning.

How much GMV did QRIS payments that settled successfully bring in across Jabodetabek during July 2026?

Three terms need a gloss. GMV stands for Gross Merchandise Value, the sum of goods sold in a period. QRIS is the national QR payment standard in Indonesia. Jabodetabek names the Jakarta metro region: Jakarta, Bogor, Depok, Tangerang and Bekasi.

Every column the question needs exists in the warehouse, and yet the schema can't answer it. Four decisions are hidden inside the question, and a person made each one:

• The status that proves money arrived. Payments carry both status_code and transaction_status. The appealing status_code = 200 only says the gateway call succeeded, cancellations included. The funds have reached the merchant account solely when transaction_status = 'settlement'. The seeded values copy a real payment gateway's public API.
• The orders that count. A settled payment attached to a returned order is cash going back out, yet it keeps reading settlement until the refund is processed. The filter therefore needs orders.order_status = 'completed', and the schema never hints at it.
• The timestamp that sets the period. Settlement lags orders by hours or even days. Slicing July by orders.order_ts yields one total and slicing by payments.settlement_ts yields another, so an order from 31 July may land in August.
• The cities that make up the region. There's no region column. Jabodetabek has to be spelled out as thirteen exact destination_city values, and Kota versus Kab. marks two distinct places and not variant spellings.

A mistake on any of these still produces a query that runs. There's no error, no empty result and no stack trace, only a well-formed number headed for a board deck.

Context versus knowledge

The obvious fix is to write down the four decisions and give them to the model with the schema. A capable model can often reason to the right answer from that. But it does so again in the next session, and nothing forces two sessions to agree. The definition of GMV isn't stored anywhere. It gets re-derived on demand by a process that doesn't remember yesterday's conclusion.

This difference carries the rest of the argument. Context is material the model reasons over. Knowledge is a conclusion someone has already reached and approved. Reasoning is paid for again in every session, whereas a written conclusion is paid for just once.

Written where, though? Not in the prompt string, for the reasons above. A database or vector store gives you storage and takes away review, since an analyst won't open a pull request against an embedding store.

What OKF is

Google's proposal is OKF, and it's deliberately plain. A bundle is simply a folder of Markdown files with YAML frontmatter, shared via Git, with no database, vector store or SDK involved. One file describes one concept, type is the single mandatory frontmatter key, and ordinary Markdown links join the files, which turns the bundle into a graph instead of a pile.

What goes inside is the more useful question. Types aren't registered centrally, and consumers must tolerate ones they don't recognize, so what follows is a starting vocabulary and not a schema. Across Google's two published sample bundles, the directories look like this:

• tables/ and datasets/ say what the data is: what a row stands for, how columns join, which enum values exist. This part can be derived from the warehouse, as the next step demonstrates.
• metrics/ explains what a number means. GMV and its four conditions go here.
• references/ holds domain knowledge that has no column anywhere, such as the thirteen Jabodetabek cities, a product taxonomy or the list of internal test accounts.
• policies/ stores the source-of-truth document a definition points back to, for example the revenue recognition rule Finance reissues each January.
• computations/ and attesters/ keep approved SQL for a metric, along with code that checks a result really came out of it.
• skills/ captures procedures: when to use something, its preconditions and its steps.

A fair test for whether something belongs in a bundle: would it be explained to a new analyst during their first week? If so, it goes in a file.

Version v0.2 of the spec adds three frontmatter fields that address the opening problem head-on. sources lists what a concept was derived from. verified collects sign-off events shaped like {by, at}, and a concept without any is literally marked unverified. stale_after gives an absolute moment after which the concept counts as out of date. Together they supply an owner, a review gate and an expiry, all inside a file that an analyst can edit and a reviewer can read as a pull request.

Could a team simply invent its own folder layout? It could, but a common vocabulary enables things a private layout can't. The reference agent described next can draft half a bundle because it already understands what a BigQuery Table concept is. Reviewers know what signing off verified commits them to. The format is vendor-neutral too, bound to no agent, framework or model provider, so knowledge written today survives a change of stack.

Many readers will think this is an agent skill with extra steps, and the overlap is real enough to address. OKF includes type: Skill as just one concept type. Google's sample bundle has skills/run-on-bq.md laid out as When to use, Preconditions and Steps, which resembles any ordinary agent skill. Skills are a subset of OKF rather than a synonym. A skill instructs an agent how to act, while OKF explains what terms mean, who checked them last and when the answer lapses. Using it breaks into three steps: generate what the warehouse knows, write what only people know, and let the agent read both.

Act one: generate what the warehouse already knows

When a BigQuery table is already documented, the dull part doesn't need hand-writing. The OKF repository includes an authoring agent, and one command converts a BigQuery dataset into a bundle.

uv tool install git+https://github.com/GoogleCloudPlatform/open-knowledge-format

reference-agent enrich \
  --source bq \
  --dataset "$PROJECT_ID.marketplace" \
  --out bundles/generated \
  --no-web

It emits a datasets/ document, a tables/ document for each table, and refreshed index.md files throughout. Adding --no-web skips the optional step that merges in existing documentation, which makes the first run repeatable.

The result is better than expected. It described each table's grain, found the right join key and documented the status enum. It will rarely produce a Metric or Reference document, since a BigQuery source yields dataset and table concepts and nothing more. The boundary is whatever the warehouse already records about itself.

Verify this before assuming another warehouse behaves the same. The status values appear in the generated documents only because someone had put them in the column descriptions of the DDL. Aim the generator at bare, undocumented columns and it has much less to go on.

Generated documents can contain errors, and the experiment holds an example. One run produced a query pattern with sum(t2.total_price), but there's no such column. The actual name is gross_amount, which the same file gets right in its own schema table two sections earlier. Running that snippet fails.

That's where verified pays off. Each generated document records generated: {by: reference_agent/...} but no verified event, so the format itself flags it as unreviewed. A bundle documents its own provenance, and separating machine output from human decisions takes no diff.

Act two: write down what only people know

Two hand-written files in the same bundle and format are enough to capture the four hidden decisions. metrics/gmv.md is the more important of them, and its frontmatter shows where the trust fields earn their place.

---
type: Metric
title: Gross Merchandise Value (GMV)
description: Total settled transaction value for a period in IDR, excluding cancellations, reversals and returns.
tags: [metrics, finance, headline-metric]
generated: { by: human:analytics-lead@marketplace.example, at: 2026-09-01T09:00:00+07:00 }
verified:
  - { by: human:analytics-lead@marketplace.example, at: 2026-09-01T11:00:00+07:00 }
stale_after: 2027-01-31T00:00:00+07:00
---

The body spells out all four conditions and states plainly that the schema supplies none. Settlement means transaction_status = 'settlement' and expressly not status_code = 200. Orders must be order_status = 'completed' with reversals removed. For the period, settlement_ts is the cutoff, tested with a half-open range that carries an explicit +07 offset: from 2026-07-01 00:00:00+07 inclusive up to 2026-08-01 00:00:00+07 exclusive. A BETWEEN over plain date strings loses the last day and treats every timestamp as UTC, so each boundary slides by seven hours.

The stale_after value does real work. It solves the January problem in a single line: Finance reissues revenue recognition rules each January, so the definition lapses on 31 January 2027, and the agent is told to report that instead of presenting the figure as current.

The second file, references/jabodetabek.md, enumerates the thirteen exact destination_city strings and cautions against LIKE '%Jakarta%', which would quietly miss eight of them.

Here collaboration turns tangible. Redefining GMV is now a pull request against a Markdown file that the owning analyst can review, with no prompt editing and no backend redeploy. Google's sample bundle even retains a deprecated gross-margin-legacy.md so historical results stay reproducible, which files make simple and prompt strings can't.

Act three: the agent that reads them

For data access, the agent uses Google Cloud's managed BigQuery MCP server at https://bigquery.googleapis.com/mcp, which switches on once the BigQuery API is enabled for the project. Nothing has to be deployed.

The only custom code in the project is a tool that reads a file.

def read_okf_concept(path: str) -> dict:
    """Read one OKF concept document from the local bundle.

    Args:
        path: Path relative to the bundle root, for example "metrics/gmv.md".
            Call this with "index.md" first to find which concepts exist.

    Returns:
        'status', and on success 'content' with the raw markdown and frontmatter.
    """
    doc = (OKF_BUNDLE / path.lstrip("/")).resolve()
    # `path` is model-controlled. Block anything that escapes the bundle.
    if not doc.is_relative_to(OKF_BUNDLE) or not doc.is_file():
        return {"status": "error", "error": f"no concept at {path}"}
    return {"status": "success", "content": doc.read_text(encoding="utf-8")}

And then the agent itself is defined.

bigquery_mcp = McpToolset(
    connection_params=StreamableHTTPConnectionParams(
        url="https://bigquery.googleapis.com/mcp",
        headers={
            "Content-Type": "application/json",
            "Accept": "application/json, text/event-stream",
        },
        sse_read_timeout=300.0,
    ),
    header_provider=_bigquery_auth_header,
    tool_filter=[
        "list_dataset_ids",
        "list_table_ids",
        "get_dataset_info",
        "get_table_info",
        "execute_sql_readonly",
    ],
)

root_agent = Agent(
    name="marketplace_analytics_agent",
    model="gemini-3.7-flash",
    instruction=(
        "Before writing any SQL: call read_okf_concept('index.md'), then read every "
        "concept the question touches: the metric, the payment status, the geography.\n"
        "Use only the filters, joins and period cuts those concepts define. "
        "Never invent a status value or a metric formula.\n"
        f"Run queries with execute_sql_readonly, passing projectId='{PROJECT_ID}'.\n"
        "Show the SQL you ran, and name the concepts you relied on."
    ),
    tools=[read_okf_concept, bigquery_mcp],
)

The experiment then ran this conversation.

What actually happened

Agent, question and seeded data stayed the same across configurations. Only the directory that OKF_BUNDLE pointed at changed, and each configuration ran four times.

With no knowledge layer, the reader tool pointing at an empty folder and only the MCP server's schema introspection to rely on, four runs of the same question gave three different headline numbers: IDR 371,251,000, 333,020,000 and 371,886,000. The top one overshot the truth by 29%.

With the generated bundle, one reference-agent enrich run, the figure stopped wandering. IDR 329,590,000 came back every time, stable and repeatable but 15% too high.

Adding the two hand-written concepts produced IDR 286,740,000 every time. That's correct, and the answer names the concepts it relied on.

The progression is plain: answer accuracy is bounded by how precise the supplied knowledge is. With no layer, the agent guesses the missing logic on every turn and lands up to 29% off, erratically. Machine-derivable schema context removes that instability and cuts the error to 15%, but it can't finish the job alone, because the remaining gap is business policy and not a technical one. Only explicit, human-reviewed definitions for the unwritten rules, such as the exact timestamp boundaries and order statuses, let the agent hit the exact figure reliably.

The biggest lever on accuracy turned out to be neither a longer prompt nor a larger model. It's structured, version-controlled domain knowledge placed where the database schema stops.

What this approach doesn't fix

The grounding depends on instructions, so a model is free to ignore it, and nothing prevents SQL from being written before a concept is read. Anyone wanting a hard guarantee can add an ADK before_tool_callback that rejects a query call if no concept was read during that turn.

Markdown carries no schema validation either. An analyst could commit a malformed concept, and OKF stays permissive on purpose, requiring consumers to tolerate unknown types and dead links. Real protection comes from code review on the pull request, plus verified and stale_after, which expose unreviewed or expired definitions to the agent when it reads them. The spec labels the trust tiers as advisory signals, so using them is up to whoever builds the agent. The agent here reads them, and others need to as well.

These figures aren't benchmarks, either: a single question, one model and four runs per setup on seeded data designed to hold the traps. They show a mechanism and say nothing about OKF's average effect on accuracy.

The lesson that holds up

The mechanism still has value. When a data agent is wrong, the answer doesn't look wrong. No exception fires, no result comes back empty, and nothing turns red. The SQL is valid and the figure looks reasonable, so it lands on a slide long before anyone doubts it.

The error to fear most isn't the erratic kind. With no guidance the agent returned a new number each run, easy to spot in an afternoon. With half the knowledge it returned the same number every run while casually mentioning an alternative it couldn't justify choosing. A stable result feels like proof of correctness but proves nothing.

A better model isn't necessarily what closes the gap. One command captured everything the warehouse can say about itself and stopped precisely where human decisions start: the timestamp that bounds a period, the order states that count, the status that means cash arrived. Those are choices and not facts, and they should be recorded once by whoever made them, in a place where that person can be asked later.

A good way to start is smaller than a project. Pick the metric the team keeps arguing over, the one where two dashboards disagree and everyone knows which analyst to ask. Run the generator on the dataset behind it, then write the definition with its owner in one file and one pull request. That conversation is the actual work. The format merely gives it a home and a record of who agreed and when it expires. Tooling is the dullest piece. What's interesting is what happens when an analyst and an engineer must settle on one sentence and commit it, and which hidden decisions prove to be theirs.