Guidance tools make agentic RAG take three steps instead of one flashy call

Salience summaries, clause diffs, and counting verification stay inert until a callable guidance tool, tool search, and incomplete returns teach the model the sequence.

Publisher: REACTAPP.TOPAdded to this site Sep 26, 202628 min read6538 words

This article is published in English.

Sources: Reflections on Tools: Getting Agentic RAG to Take Three Steps-Part 7 of my EDGAR Intelligence MCP journey

Earlier parts of this series built a summary tree, kept raw numbers away from the language model, pulled text from figures, asked what changed between filings, asked how anything is found at all, and asked which company a query meant. This installment is about the narrow interface those pieces actually reach: a short block of tool description text the model scans before choosing a call.

The three instruments

Part 4 dissected year-over-year risk-factor change. Handing both 10-Ks to a model and asking what moved failed not only on cost: Chroma’s Context Rot work holds the task fixed and varies only input length across many models—performance degrades as context grows, unevenly, in ways you cannot predict from the task alone. Failures are silent: a model that reports six changes never tells you it missed a seventh. Character-level redlines alone stumble on reflow; dual summaries are worse. What survived had three instruments, not two:

L1 for salience — RAPTOR-style section summaries (one AI précis per filing section from Part 1). Which topics moved. One call per side. Sees meaning that character pairing cannot.
Diff for evidence — a clause-level redline that yields quotable wording for every flagged theme. Fully mechanical, exhaustive, and reviewable.
Counting for verification — check whether the distinctive wording truly differs across the two documents. Shared phrasing on both sides usually means reshuffling, not a real change.

That triad is still correct. Written as an article paragraph, it is also completely inert. Nothing in it makes a tool fire. A model asked what changed in Apple’s risk factors has never read the essay. It has a question, a tool list, and whatever those tools say about themselves. Seeing diff_filings, its reasonable instinct is to call it and narrate the return. Agentic RAG often skates past this: retrieval is the easy half; the hard half is teaching the model which three steps to take and in what order using only the text attached to tools.

_SECTION_SLUGS = (
    "overview", "routing", "setup", "deal_walkthrough", "diff_workflow",
    "tools", "warmth", "time_axes", "sql_track", "metrics", "alpha", "fields",
    "idempotency", "presentation", "response_bounds",
)


def _split_sections() -> dict[str, str]:
    parts = _GUIDANCE_MD.split("\n## ")[1:]
    assert len(parts) == len(_SECTION_SLUGS), (
        f"guidance has {len(parts)} sections, {len(_SECTION_SLUGS)} slugs"
    )

If the guidance is wrong, the instruments never compose. If it is right but unread, they still never compose. The rest of this piece is about making that guidance load, stay short, and survive tool search.

Selection, invocation, and the thing that is neither

Tool use splits into two moments people already know how to name.

Selection is deciding which tool belongs in the active set for this turn—routing, tool search, catalog filtering. Invocation is emitting a structured call with arguments once a tool is available. Frameworks and model APIs obsess over both.

There is a third moment that is neither: orientation—the prose that tells the model why a tool exists in a workflow, what must happen before it, and what “done” looks like afterward. Orientation is not a retrieve-or-generate choice. It is the sentence that turns three inert instruments into a sequence.

Most stacks bury orientation in a system prompt that grows forever, or omit it and hope the tool name is enough. Neither scales when you have dozens of MCP tools and a user who asks a one-line question about two filings.

The guidance tool

A guidance tool is a tool whose only job is to return procedure: the short, authoritative description of the multi-step method. It does not fetch filings. It does not compute diffs. It returns text the model is meant to follow for the next several turns.

For the full repeatable diff flow see `edgar_guidance(section="diff_workflow")`.

Calling it is cheap compared with stuffing the whole method into every system prompt. It is also discoverable: tool search can surface it when the query smells like “what changed,” and leave it out when the user only wants a single numeric fact.

Guidance tools fail when they become second copies of documentation that drift, or when they try to do retrieval themselves. Keep them procedural and short.

The skills point at it. They do not copy it.

Skills (or skill packs, or agent playbooks) should reference the guidance tool by name and purpose, not paste its body. Copying creates two sources of truth. When the method changes—new verification step, new counting rule—you update the guidance tool once. Skills keep saying “call workflow_guidance first for YoY risk-factor change.”

For document selection and extraction strategies, use PFW_get_guidance
(see quick reference chart for section selection).

For document selection strategies and multi-document workflows, use
PFW_get_guidance (see quick reference chart for section selection).

For cross-MCP workflows, use PFW_get_guidance (see quick reference chart).

For field selection guidance and token estimates, use
PFW_get_guidance(section='tools').

That indirection feels bureaucratic until the day you fix a silent failure mode and realize twelve skill files would have been wrong copies.

What USPTO MCP servers taught about guidance tools

Earlier USPTO MCP work made the same mistake in a different domain: too many tools, each with a friendly name, none with an authoritative sequence. Agents called the shiny retrieval tool first and skipped verification. The fix was not more tools—it was a guidance surface that said, in tool-return text, which order was mandatory and which returns were incomplete without a follow-up call.

WORKFLOW: even the substantive-only headline is clause-level and can
still be noisy on a heavily rewritten section. For "what materially
changed" questions, run RAPTOR L1 FIRST — search_filings(level=1,
section_path=<the section>) on each filing to read the two section
summaries and build a short topic list (typically 5-10 items) of what
actually moved. Then call this diff to pin the EXACT clause language
for each flagged topic, and skip every pair that doesn't touch one.
The L1 contrast also surfaces meaning that a cold diff buries in
reflow (e.g. a new tax-law valuation allowance). See
edgar_guidance(section="diff_workflow") for the full repeatable flow.

Lessons that carried into EDGAR work:

Guidance must be callable, not only documented in a README the model never sees.
Guidance must be short enough to survive context pressure after tool search injects other descriptions.
Guidance must name failure modes (“do not stop after the redline”) the same way it names happy paths.
Guidance must stay domain-owned—USPTO sequences do not belong in EDGAR tool text and vice versa.

So the workflow moved into the docstrings

Putting the method in a markdown file the human likes does nothing for the model. Putting it in the tool docstring / description of a guidance tool does, because that is the text the runtime injects when the tool is selected.

- `alpha` (default = EDGAR_RAPTOR_HYBRID_ALPHA, falling back to
  EDGAR_HYBRID_ALPHA): only affects the L0 clause collection, which
  carries BM25 sparse vectors. 1.0 = pure dense, 0.0 = pure BM25. Do
  NOT lower it for an exact citation or identifier (a literal
  "$750,000,000", a defined term, "Section 9.1(d)") — keep the default
  and pass `rerank=True`, which supplies most of the precision on a
  short identifier-only query.

The migration looked like this in practice: take the article’s three-instrument paragraph, compress it into imperative steps, attach success criteria, and expose it as the return body of get_yoy_risk_factor_method (name varies). Skills and system prompts only point at that tool. When Part 4’s method gained a counting verification step, only the guidance return changed.

Docstrings are not a dumping ground for novels. Target a few hundred words of procedure, not the full essay. Link out for humans; keep the callable surface tight.

Return paths: the second half of the sentence

A tool that returns a huge redline without saying what to do next trains the model to summarize and stop. Return paths are the second half of the sentence: after evidence arrives, which tool confirms, which tool counts, and when the answer is allowed.

- For a MULTI-YEAR or HISTORICAL question about one company, prefer
  `group=` over a single `cik`: a kind='lineage' group covers the
  whole predecessor/successor chain (so a question that crosses a
  name/CIK change — Avago→Broadcom, Apollo's predecessor — doesn't
  silently miss the older filings) and its members COLLAPSE into one
  issuer block, with each hit keeping its own cik and the block
  carrying `lineage_ciks` for provenance. `resolve_parties_to_cik`
  reports which group to pass.

Design returns as partial results plus next-step hints when the workflow is multi-hop. That does not mean the tool should become an agent itself—it means the payload should not look like a finished answer when verification is still pending. Separately, error returns should distinguish “no drift found” from “could not run the diff,” or the model will invent confidence.

The same mechanism, a different workflow

Guidance tools are not unique to risk-factor change. Company resolution (Part 6), figure extraction (Part 3), and numeric-safe summarization (Part 2) each need a short callable method when the user question is underspecified. The pattern repeats: one guidance tool per workflow family, skills that point, returns that refuse to look finished early.

AI-RANKED RESULTS — ordered by semantic similarity, not authority. L0 hits
are extracted clause text. L1/L2 hits are AI-generated summaries — take
especial caution with figures ... A claim about WHAT CHANGED, read off two
L1/L2 summaries, is likewise unverified until the distinguishing language
is confirmed against the underlying clause text on both sides.

Sharing one mega-guidance blob across workflows recreates the system-prompt blob problem. Prefer several small guidance tools with crisp trigger language in their descriptions so tool search can pick the right one.

And one that starts the whole thing

Some sessions need a bootstrap guidance call before any retrieval: which company, which forms, which years, which instrument order. Without it, agents jump to diff_filings on the wrong pair of documents.

INTERPRETING THE REDLINE — you can and should tell real edits from chunking
artifacts; do that filtering rather than treating every delta as substantive.
... The word-level highlights are the tell: differences only at the START
and/or END of the clause (or a small leading/trailing add/delete) mean a
realignment, while a term changing in the MIDDLE (amounts, dates, defined
terms, added/removed obligations or conditions) is a real edit.

Bootstrap guidance is allowed to ask clarifying questions via the chat channel while still forbidding premature tool fireworks. Keep it idempotent—calling it twice should not mutate store state.

Thirty-five tools

The EDGAR MCP surface grew to on the order of thirty-five tools. At that size, dumping every description into the prompt is impossible, and letting the model see a flat list invites random first calls. Tool search becomes mandatory: embed query against tool metadata, inject a shortlist, then let the model choose.

if counts.get("substantive_modified"):
    response["chunking_note"] = AI_DIFF_REALIGNMENT_NOTE

Thirty-five is not the point—the point is that guidance tools must survive search. If the guidance tool’s description does not match “what changed / year over year / risk factors,” it will never enter the shortlist, and the three instruments stay inert again.

Tool search, and the three tools

For the YoY risk-factor workflow, the search hit list should preferentially include: the guidance tool, the L1 salience tool(s), the diff tool, and the counting/verification tool. Everything else is noise for that question class.

"revenue is a derived metric — one issuer's consolidated number.
 A multi-CIK `ciks=` list would blend inputs across issuers into a
 figure belonging to none of them. Call once per `cik=` (or pass a
 lineage `group=` for a predecessor chain)."

Tuning search is product work: synonyms in descriptions, negative examples (“not for single-company lookup”), and evaluation sets of user questions with expected first tools. Without that eval loop, guidance remains a philosophical preference.

What the person sees

Users should not see the scaffolding. They ask a plain question and get a cited answer. Underneath, the transcript may show guidance → L1 → diff → count. Product UX can collapse those into progress labels (“checking what moved,” “pulling exact clauses,” “verifying novelty”) without exposing tool names.

Input(s) [...] were not tagged by this issuer under any recognized XBRL
concept in the requested periods, so the derived value is 'unknown'
wherever they are needed. Not every issuer reports every line item
(e.g. asset-light filers omit capex), so the same metric may resolve
for an issuer that does tag them.

When something fails, show the stage that failed. “Diff unavailable” is actionable; “the agent erred” is not.

The proxy pages

Human-facing proxy pages document the same workflows for operators and auditors. They are allowed to be long. They must not become the only copy of the method the model needs—because the model does not browse your docs site mid-flight unless you tool that browse deliberately.

Per-figure descriptions are NOT produced on this path: charts and images
on these pages carry no annotation, and their absence is not evidence the
page had no figures ... check the linked source image (source_url) before
concluding something is absent from the filing.

Keep proxy pages and guidance tools linked: page says “canonical machine instructions live in tool X,” tool description says “human narrative at URL Y.” Drift between them is a release bug.

Reproduce it yourself

To reproduce the pattern outside this series:

Write the method as imperative steps with named tools and stop conditions.
Expose it as a guidance tool return body.
Point skills at the tool; do not paste the body.
Make returns of evidence tools declare incompleteness until verification runs.
Put tool search in front of a large catalog; evaluate first-tool accuracy on a golden question set.
Keep human docs as proxies, not as the runtime source of truth.

armth: hits from filings whose summary tree is still cold (L0-only)
trigger background tree builds — `tree_builds_enqueued` in the result
says how many; re-run the search shortly after to get L1/L2 summaries.

Evaluate a math expression EXACTLY (deterministic; LLMs are error-prone
at multi-digit arithmetic). Compute, calculate, arithmetic, math.

EXPECTED_NON_DEFERRED = {
    "guidance.py",
    "resolve_parties.py",
    "edgar_strategic_search.py",
}

# Anthropic's stated range for non-deferred tools.
MAX_NON_DEFERRED = 5

The method is going to change

Part 4’s triad will not be the last word. New verification tricks, cheaper salience models, better reflow handling—all will land. If the method lived only in essays and skill paste-ups, every improvement would be a multi-file archaeology project. Living in one guidance tool makes change a single edit plus a regression pass on the golden questions.

Use your tool-search or tool-discovery step, but do not CALL any of the
tools it finds. I want to read them, not run them.

1. Search for tools now, by server name and by keyword, and list the MCP
   servers and tools you can reach. If your first look says you have no
   tools, search again before concluding that: with tool search enabled,
   most tools stay hidden until you go looking for them.

2. Pick one non-trivial tool. Show me, verbatim and without summarising
   or reformatting:
   - its exact tool name
   - its complete description text, every line of it
   - its full input schema: each parameter, its type, its default, and
     the description attached to that parameter

   Print the description IN FULL even if it runs to a thousand words or
   more. Do not abbreviate it, do not list its section headings instead,
   and do not tell me to look at the tool result: I cannot see what you
   can see, which is the whole point of this.

3. Then tell me, in your own words:
   - does that text instruct you to call anything BEFORE this tool, or
     AFTER it?
   - does it tell you how to interpret or filter the result?
   - does it tell you to relay anything to me?
   - is there anything in it you would not have done on your own?

The takeaway

Agentic RAG is not “retrieve then generate.” For change detection across SEC filings it is a composed instrument chain that only fires correctly when the model is told—in a place it actually reads—which order to use. The three instruments (salience summaries, clause diffs, counting verification) solve the scientific problem from Part 4. They do nothing until orientation exists as a callable guidance tool, skills point instead of copy, returns refuse to look finished early, and tool search can find the guidance among dozens of siblings.

Treat selection and invocation as solved primitives if you like; the product differentiator is the third moment: orientation. Orientation that lives in READMEs is decoration. Orientation that lives in system prompts becomes an unmaintainable blob. Orientation that lives in a short, searchable, domain-owned guidance tool is operable.

USPTO MCP work taught the same lesson in trademarks and patents: friendly tool names without sequence produce confident wrong first calls. EDGAR work inherited that scar and moved the workflow into docstrings on purpose. Thirty-five tools made tool search non-optional; search made description quality part of correctness, not marketing copy.

What the person sees should stay calm. What the proxy pages say should stay complete. What the model reads before the first real retrieval should stay short, imperative, and versioned with the method. When the method changes tomorrow, change the guidance tool and re-run the golden questions. Do not chase twelve pasted copies.

If you take one engineering rule from this installment: never let the only copy of a multi-step method live in a place the model cannot call. Essays teach humans. Guidance tools teach agents. Skills are pointers. Returns are half-sentences that demand the next verb. Tool search is the gate that decides whether any of that text enters the room.

The series continues: the instruments evolve, the catalog grows, and the guidance surface has to stay the smallest truthful description of how those instruments compose. That is how agentic RAG takes three steps instead of one flashy call.

Concrete checklist for a new workflow family:

Name the instruments and the verification that makes silent failure harder.
Write a guidance tool whose description matches how users ask.
Keep the return body under a few hundred words of procedure.
Point skills at it; ban paste.
Mark evidence returns as incomplete without verification.
Add golden questions where the first tool must be guidance.
Link human proxy pages without making them runtime-critical.
Rehearse a method change as a single-tool edit.

Teams that skip the checklist rediscover Context Rot the hard way: the model “read” both filings, spoke fluently, and missed the change that mattered. Teams that keep the checklist find that agentic RAG becomes boring in the best sense—predictable steps, citable clauses, and a verification count that can fail closed.

Boring is the goal. Flashy single-tool answers are how silent misses ship to users.

Evaluate a math expression EXACTLY (deterministic; LLMs are error-prone
at multi-digit arithmetic). Compute, calculate, arithmetic, math.

Use it over numbers you ALREADY retrieved (e.g. from
query_financial_facts) — it does NOT fetch data. Supports + - * / // % **,
unary minus, and abs, round, min, max, pow, sqrt, log (natural), log10,
log2, exp, floor, ceil, plus constants pi and e. No variables, attribute
access, or other calls (safe AST evaluator).

Examples:
- 5-yr revenue CAGR: "(60922/26974)**(1/5) - 1"
- net debt / EBITDA: "(96700 - 29943) / 34000"
- YoY % change: "(383285 - 394328) / 394328 * 100"
- segment as % of total: "23056 / 81462 * 100"

Returns {expression, result, formatted} on success, or {error, expression}
if the expression is rejected.

Sources

Primary references for this installment include the ReAct and tool-use literature as background, Chroma’s Context Rot study for length-degradation evidence, RAPTOR-style summarization as used in Part 1, prior parts of this EDGAR Intelligence MCP series for L1/diff/count instruments, and the project’s own MCP tool catalog and proxy documentation. Package and API names drift—treat guidance tool text as the versioned contract for agent behavior, and keep human essays as commentary rather than runtime truth.

Why inert methods feel finished on the page

Technical writing rewards complete explanations. A paragraph that names L1, diff, and counting reads like a design. Agents do not read designs; they read tool text under a token budget. The gap between those audiences is where most “agentic” demos quietly become single-tool wrappers. Closing the gap means accepting that some of your best prose must be rewritten as callable procedure—and that the callable procedure is allowed to be less elegant than the essay.

Orientation versus retrieval quality

Improving embeddings, chunking, and re-ranking helps the evidence tools. It does not tell the model to call three of them in order. Teams often spend a quarter polishing retrieval metrics and a week on prompts, then wonder why change-detection still collapses to one diff_filings call. Budget orientation as a first-class workstream: description text, search synonyms, golden first-tool tests, and return-shape reviews.

Failure stories worth encoding

Encode the failures you already paid for: stopping after a redline that only shows reflow; trusting dual summaries that flatten contradictions; treating reorganization as novelty; answering before company resolution finishes; mixing USPTO sequences into EDGAR sessions. Each failure becomes one imperative line in guidance and one golden question that fails if that line is ignored.

Token economics of guidance

A three-hundred-word guidance return is cheaper than stuffing two 10-Ks into context “just in case,” and cheaper than a system prompt that lists every workflow. It is also cheaper than the support cost of fluent wrong answers. Measure guidance like you measure retrieval: latency, hit rate under tool search, and downstream verification pass rate—not only whether the prose sounds clever.

Governance

Who may edit guidance tools? Treat them like production config. Require review, keep history, and block merges that change guidance without updating golden questions. Operators reading proxy pages should see the same version identifier the tool returns in a header line. When audit asks why the agent called three tools, you point at a versioned guidance body—not a Slack argument from last month.

What not to put in guidance

Do not put secrets, environment URLs with credentials, or user-specific state. Do not put entire filing excerpts. Do not put branching novel-length policy. Do put order, stop conditions, and the names of the next tools. If a detail is only for humans, leave it on the proxy page.

Relationship to LangGraph-style graphs

Some teams encode the three steps as a hard graph with no model choice. That is valid and often safer. Guidance tools shine when you still want a model in the loop for underspecified questions, but need the method to remain discoverable among many tools. Graphs and guidance are complementary: a graph can implement the method once selected; guidance helps selection and keeps hybrid agents honest.

Evaluation harness sketch

Maintain a table of user questions, expected first tool, expected tool sequence class, and forbidden early-stop tools. Run the harness on every guidance edit. Fail the build if first-tool accuracy drops. Spot-check transcripts weekly for fluent answers that skipped counting. Publish a simple dashboard so product and eng share the same definition of “agent took three steps.”

On naming

Call the tool what search needs: verbs users type (explain_yoy_risk_factor_method) beat clever internal codenames. Descriptions should include synonyms: change, delta, year over year, risk factors, Item 1A, compared to last year. Negative cues help too: “not for live stock quotes,” “not for single-paragraph summarization.”

Scaling beyond one series

Once several domains each have guidance tools, build a tiny meta-index: domain → guidance tool name. Do not merge domains into one essay-tool. Cross-domain contamination is how USPTO advice leaks into EDGAR sessions. Keep the index dull and explicit.

Closing the loop with Part 4

Part 4 proved why one instrument is scientifically insufficient. This part proves why a correct scientific method still fails operationally without a callable orientation surface. Read them as a pair: instruments without guidance are inert; guidance without instruments is theater. Together they are how agentic RAG earns the “agentic” label on a task that punishes silent misses.

Operational appendix: rehearsal script

Before shipping a guidance change, run a rehearsal: cold session, golden question, confirm guidance is first tool, confirm L1 runs on both sides, confirm diff is requested with the pair guidance named, confirm counting runs, confirm the user-visible answer cites clauses and states verification. Record the transcript ID next to the guidance version. If any step skipped, fix description/search before celebrating retrieval metrics.

Operational appendix: rehearsal script

← All articles Send the brief