PageIndex: Vectorless, Reasoning-Based RAG With a Tree Index

Imagine a RAG framework that drops vector databases altogether, and then posts a benchmark result that no embedding-based system comes close to. That's PageIndex, an open-source project from VectifyAI. On FinanceBench, the standard benchmark for question answering over financial documents, ordinary vector-based RAG lands somewhere around 30 to 50 percent. PageIndex reports 98.7 percent. It didn't get there with a better embedding model. It got there by discarding embeddings.

That result deserves a second look, because financial QA over SEC filings is one of the harshest retrieval tests in production AI. It demands multi-step reasoning, jumps between sections and exact numbers. The best system on the table used no vectors, no chunking and no vector database.

Here's what PageIndex does, how it's built, working Python code, where it shines and where it doesn't.

The wall that long documents hit

Anyone who has built RAG over something longer than a blog post knows the failure. A user asks a system grounded in a 200-page annual report how total net revenue in FY2024 compares with FY2023. The standard pipeline runs like this:

1. Split the document into fixed-size chunks, usually 300 to 500 tokens.
2. Embed every chunk with a model such as text-embedding-3-large.
3. Store the vectors in Pinecone, Weaviate, Milvus, Chroma, pgvector or similar.
4. At query time, embed the question and fetch the top-k nearest vectors by cosine similarity.
5. Pass those chunks to the LLM to write the answer.

For short, unstructured material such as blog posts, support tickets and product docs, that's great. A 10-K filing breaks it in predictable ways.

The query doesn't look like the answer. Vector retrieval assumes the text most similar to the query is the most relevant. But a query expresses intent, not content. A question about revenue trends doesn't surface the one table row on page 47. It surfaces every paragraph that talks about revenue.

One term, sixty places. In a long annual report, "operating income" appears dozens of times in different contexts. Similarity ranks them about equally, and the passage that really answers the question may never reach the top three.

Chunking mangles tables. A table header lands in chunk 14 and the row you need lands in chunk 15. Neither makes sense alone. Overlapping windows, semantic chunking and table-aware parsers all help, but none removes the underlying problem.

No memory across turns. If the previous message asked about assets and the next asks "what about liabilities?", the retriever sees an entirely new query. Nothing tells it the user most likely means the same section of the same report.

Cross-references stay hidden. Suppose page 12 points readers to Appendix G, and that appendix sits on page 87. The appendix text has no semantic overlap with the original question, so vector RAG will never follow the pointer. Injecting cross-reference metadata into chunks is a workaround, but it's brittle and doesn't scale.

These aren't rare edge cases. They come from the architecture. As VectifyAI's co-founder Mingtian Zhang put it in the project's announcement, what retrieval really needs is relevance, and relevance requires reasoning.

What PageIndex is

PageIndex is a vectorless, reasoning-based RAG framework released by VectifyAI in September 2025, created by Zhang and Yu Tang. It's MIT-licensed, and the repository grew quickly, passing 23,000 stars and nearly 2,000 forks by late March 2026.

Its central idea comes from AlphaGo. Rather than exhaustively scanning a space, the way vector similarity compares every chunk, use a learned strategy to navigate it intelligently. PageIndex applies that thinking to documents, with three notable properties:

• No vector database. Document structure and LLM reasoning replace similarity search, so there's no Pinecone, no Chroma and no embeddings.
• No chunking. Documents stay organized as natural sections, chapters, paragraphs and tables, not arbitrary 500-token windows.
• Human-style navigation. It behaves like a person searching a document by hand: look at the table of contents, find the section that probably holds the answer, and read it.

The method has two steps.

Step 1: a hierarchical tree index

Ingesting a document doesn't create any embeddings. PageIndex analyzes the document's structure and produces a hierarchical tree, essentially an LLM-friendly table of contents. Each node holds a title, a short summary of the section, a page range and child nodes for subsections, nested as deep as needed.

A 50-page SEC filing might yield a tree of 30 to 50 nodes, stored as one JSON object. Nothing lives in a vector database, it's only JSON. The whole object fits in one LLM context window, and you can open any node and read it. A node looks like this:

{
  "node_id": "0006",
  "title": "Financial Stability",
  "start_index": 21,
  "end_index": 22,
  "summary": "Covers the Federal Reserve's financial stability oversight...",
  "sub_nodes": [
    {
      "node_id": "0007",
      "title": "Monitoring Financial Vulnerabilities",
      "start_index": 22,
      "end_index": 28,
      "summary": "Describes the Fed's vulnerability monitoring framework..."
    },
    {
      "node_id": "0008",
      "title": "Domestic and International Cooperation",
      "start_index": 28,
      "end_index": 31,
      "summary": "Federal Reserve collaboration with international bodies..."
    }
  ]
}

Each node_id points to the raw content of that section: text, tables and images. The LLM can request the content of any node when it needs it.

The architectural insight is where this index lives. A vector database keeps a separate, static index of embeddings outside the model. Here the JSON tree sits inside the LLM's active context during inference. VectifyAI calls it an in-context index, and the model doesn't merely query it, it reasons over it live.

Step 2: tree search driven by reasoning

When a question arrives, PageIndex gives the LLM the tree, with titles and summaries only and no raw text, and effectively asks where to look given this structure and this question. The model reads the node titles and summaries, judges which sections probably contain the answer and returns a list of node IDs. The system then fetches the real content for those nodes.

Retrieval is a loop:

1. Scan the table of contents to understand the layout and spot candidate sections.
2. Pick a section most likely to contain what's needed.
3. Read it, pulling the raw text from that node.
4. Check if it's enough. If so, write the answer. If not, return to the first step and try another section.

This is a different paradigm from vector search. A vector database computes cosine similarity over every chunk in parallel, which is fast but shallow. PageIndex instead has the LLM reason about where the answer is likely to sit. Along the way the model can chase cross-references, recognize that a question about appendix data belongs with the appendix node, and realize that a multi-part question needs content from two sections. Each step also leaves a reasoning trace, so you can see which nodes were visited and why. Retrieval stops being a black box.

The Appendix G example

A demo built on VectifyAI's MCP integration shows why this matters. Someone asks about the total of deferred assets in a Federal Reserve annual report. The main body, on pages 75 to 82, discusses the change in deferred assets but never states the total. Page 77, though, contains a sentence saying that Table 5.3 summarizes income, expenses and distributions and that Appendix G gives more detailed information.

Vector RAG is stuck here. Appendix G is a table of numbers with no semantic similarity to a question about deferred assets, so the database ignores it. PageIndex does it differently. The LLM reads the tree, goes to the financial section, spots the reference to Appendix G, follows it through the tree, retrieves the right table and returns the exact figure, with the reasoning trace showing each hop.

Python, end to end

The code below is adapted from the official PageIndex cookbook, with comments to show the flow. Start with the install:

pip install pageindex openai

Imports:

import os
import json
import asyncio
from pageindex import PageIndexClient
from openai import AsyncOpenAI

Then credentials and clients. An API key comes from the PageIndex dashboard:

# Grab an API key from https://dash.pageindex.ai/api-keys
PAGEINDEX_API_KEY = os.environ["PAGEINDEX_API_KEY"]
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]pi_client = PageIndexClient(api_key=PAGEINDEX_API_KEY)
openai_client = AsyncOpenAI(api_key=OPENAI_API_KEY)

Build the tree

Bring in the utilities:

import pageindex.utils as utils

Upload a PDF and let PageIndex generate the tree. Indexing takes a little while, so the script polls until it's ready and prints the result:

# Upload a PDF; PageIndex handles the tree generation
doc = pi_client.upload("annual_report_2024.pdf")
doc_id = doc["doc_id"]# Tree generation takes a bit, so we poll
while not pi_client.is_retrieval_ready(doc_id):
    print("Still indexing...")
    import time; time.sleep(5)# Grab the tree and take a look
tree = pi_client.get_tree(doc_id, node_summary=True)["result"]
print("Document Tree:")
utils.print_tree(tree)

The printout shows every section, subsection and page range, which makes it obvious right away whether the tree captured the layout correctly.

Let the LLM search the tree

This is the core step. The function receives the tree and a query:

async def find_relevant_nodes(tree: dict, query: str) -> list:
    """LLM reasons over tree structure to identify relevant nodes."""

Inside, the raw text is stripped out to save tokens, since the model only needs titles and summaries. The prompt casts the model as a document retrieval expert, tells it to follow cross-references and asks for JSON containing its reasoning plus a list of node IDs:

    # Strip raw text to save tokens; the LLM only needs titles and summaries
    tree_without_text = utils.remove_fields(
        tree.copy(), fields=["text"]
    )    search_prompt = f"""
    You are a document retrieval expert. Given a question and
    a hierarchical tree structure of a document, identify all
    nodes likely to contain the answer.    Each node has a node_id, title, and summary.
    Follow cross-references if a section mentions another.    Question: {query}    Document tree structure:
    {json.dumps(tree_without_text, indent=2)}    Reply in this JSON format only:
    {{
        "thinking": "<reasoning about which nodes are relevant>",
        "node_list": ["node_id_1", "node_id_2"]
    }}
    """    response = await openai_client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": search_prompt}],
        temperature=0,
        response_format={"type": "json_object"},
    )    result = json.loads(response.choices[0].message.content)
    print(f"LLM reasoning: {result['thinking']}")
    return result["node_list"]

Fetch content and answer

The next function gathers the raw text for the chosen nodes:

def collect_node_content(tree: dict, node_ids: list) -> str:
    """Pull raw text from the nodes the LLM selected."""
    all_nodes = utils.flatten_tree(tree)
    context_parts = []

Then the loop body assembles context, with each section labeled by title and page range, and the full pipeline wraps it: pick nodes, fetch content, generate an answer with citations, and return the answer together with the nodes used.

    for node in all_nodes:
        if node["node_id"] in node_ids:
            title = node.get("title", "Untitled")
            pages = f"pages {node.get('start_index', '?')}-{node.get('end_index', '?')}"
            text = node.get("text", "")
            context_parts.append(
                f"[{title} | {pages}]\n{text}"
            )
    return "\n\n---\n\n".join(context_parts)
async def answer_query(tree: dict, query: str) -> dict:
    """Full vectorless RAG pipeline: tree search + answer generation."""    # Step 1: LLM picks the nodes
    node_ids = await find_relevant_nodes(tree, query)    # Step 2: Fetch content from those nodes
    context = collect_node_content(tree, node_ids)    # Step 3: Generate answer with citations
    answer_prompt = f"""
    Answer the question using only the provided context.
    Cite specific pages and sections in your answer.    Context:
    {context}    Question: {query}
    """    response = await openai_client.chat.completions.create(
        model="gpt-4.1",
        messages=[{"role": "user", "content": answer_prompt}],
        temperature=0,
    )    return {
        "answer": response.choices[0].message.content,
        "retrieved_nodes": node_ids,
        "context_length": len(context),
    }
# Run it
query = "What was the total value of deferred assets in 2023?"
result = asyncio.run(answer_query(tree, query))
print(result["answer"])
print(f"Nodes used: {result['retrieved_nodes']}")

Plugging it into MCP

For Claude, Cursor or any MCP-compatible agent, PageIndex connects directly through a hosted server:

{
  "mcpServers": {
    "pageindex": {
      "type": "http",
      "url": "https://api.pageindex.ai/mcp",
      "headers": {
        "Authorization": "Bearer your_api_key"
      }
    }
  }
}

A local MCP server also exists for uploading files from disk, and it needs Node.js 18 or newer:

{
  "mcpServers": {
    "pageindex": {
      "command": "npx",
      "args": ["-y", "@pageindex/mcp"]
    }
  }
}

The numbers, in context

The headline figure is from FinanceBench, which draws on actual SEC filings, 10-Ks, 10-Qs and 8-Ks included, and accepts only exact answers. Mafin 2.5, VectifyAI's financial analysis agent built on PageIndex, was run against the full dataset. Reported accuracy:

A gap of roughly 49 percentage points over conventional vector RAG isn't a small improvement. It's a different class of result. Three factors keep coming up:

• Following cross-references. In-document pointers like "see Appendix G" are navigated through the tree. Similarity search has no notion of them.
• Keeping structure. Tables have headers, footnotes and cell relationships. PageIndex retains them as tree nodes, while chunking destroys them.
• Multi-step reasoning. A question comparing FY2023 with FY2024, reported in different parts of the filing, is handled naturally by the iterative loop.

Where it falls short

PageIndex is impressive for its niche, but it doesn't replace vector RAG everywhere.

• It's slower. Every retrieval involves several LLM calls: read the index, reason about nodes, fetch content, check sufficiency, maybe loop. Vector RAG answers in milliseconds, PageIndex in seconds. The co-founder has noted that because retrieval happens inline during generation, time to first token can be comparable to an ordinary LLM call, but total latency is still higher.
• It costs more. Multiple inference calls per query versus one embedding lookup. At high volume the math gets uncomfortable.
• It's built for depth, not breadth. It excels at deep extraction from one long document. Searching 10,000 short documents at once is where vector databases earn their place, and building and reasoning over a tree for each one isn't practical.
• Structure matters. A cleanly headed SEC filing produces an excellent tree. A badly formatted scan with no logical sections yields a degraded table of contents, and everything downstream suffers. A vision-based mode exists for such cases, but structured documents remain the sweet spot.
• LLM reasoning is imperfect too. Vector RAG goes wrong because embeddings only approximate meaning. PageIndex goes wrong when reasoning is only approximate: the model may go to the wrong section, misread a summary or skip a relevant sub-node. The failures are more interpretable and arguably easier to debug, but they still happen.
• One benchmark. FinanceBench measures QA on well-structured SEC filings. Results on messy, multi-domain, ambiguous queries across very different document types haven't been independently validated to the same degree. The 98.7 percent is real, but it's important to know what it measures.

When to use it

Reach for PageIndex when:

• the documents are long, structured and professional: annual reports, contracts, regulatory filings, technical manuals
• accuracy outweighs speed, such as when a wrong answer costs money or creates compliance risk
• queries need multi-step reasoning or hopping between sections
• you need audit trails, since each answer arrives with a reasoning trace and node references, which regulated industries value

Stay with vector RAG when:

• you need sub-second answers at high query volume
• you're searching a large corpus of short documents
• the documents are unstructured or conversational
• 90 percent accuracy is good enough
• cost per query is the main constraint

A hybrid is probably underexplored. Use vector search to find which document in a large corpus matters, then pass it to PageIndex for deep extraction.

What's new since launch

The project has moved quickly since September 2025.

Agentic vectorless RAG. The latest addition wires a self-hosted PageIndex into the OpenAI Agents SDK. The agent is handed three tools (get_document(), get_document_structure() and get_page_content()) and reasons over the tree by itself, calling the right tools and retrieving content without manual node selection. It's the example to start from for agentic workflows:

pip install openai-agents
python3 examples/agentic_vectorless_rag_demo.py

Vision-based, OCR-free mode. A vision-native pipeline now skips OCR. Page images go straight to a model with vision, and the tree is derived from what that model sees on the page. For financial documents where the layout of a balance sheet carries as much meaning as the text, that matters. Charts, merged-cell tables and footnotes all stay intact because nothing is ever converted to text first.

A TypeScript SDK, pageindex-js-sdk, arrived in early 2026 for Node.js integrations and web-based document tools. A one-click .mcpb desktop extension installs PageIndex into Claude Desktop with OAuth handled automatically. ChatIndex, a separate repository from January 2026, applies the same tree-indexing idea to long conversation histories.

The wider shift

PageIndex fits a broader move from passive retrieval, which fetches whatever looks similar, to active, agentic retrieval that reasons about where to look. It's already visible in coding tools. Claude Code and Cursor have drifted away from vector-based RAG for code in favor of actively exploring the codebase, and the PageIndex team expects documents to follow. For structured documents, that prediction looks plausible.

Vector databases aren't going away. They're fast, cheap and good enough for a huge range of problems. But where correctness, transparency and deep document understanding are non-negotiable, as in financial audits, legal discovery, medical records and regulatory compliance, reasoning-based retrieval is a fundamentally better fit.

The code and docs live at https://github.com/VectifyAI/PageIndex, with documentation at https://docs.pageindex.ai/.