Vectorless RAG with PageIndex: Retrieval by Reasoning Instead of Vectors

PageIndex swaps embeddings and chunking for an LLM that walks a document tree. A short Python sketch, and the cases where vector RAG is still the better pick.

Publisher: REACTAPP.TOPAdded to this site Oct 23, 20269 min read2090 words

This article is published in English.

Sources: **I Stopped Using Vector Databases for RAG : PageIndex Vectorless RAG**

Anyone who has built RAG systems for a while knows the irritation. You spend hours on embeddings, tune chunk sizes, choose a vector database, and the system still gives the wrong answer. It found something similar, yet "similar" and "relevant" aren't the same.

Financial reports show it vividly. Take a 120-page 10-K filing. A vector RAG setup keeps pulling chunks from the executive summary while the real answer sits in a footnote on page 87. Both passages share keywords and meaning, but only one contains the number you need.

PageIndex was built to attack exactly that.

What PageIndex is

PageIndex is an open-source RAG framework from the VectifyAI team. Its central idea is simple and different from most of what's out there. Rather than turning a document into vectors and running a nearest-neighbour search, it builds a hierarchical tree from the document and has an LLM reason its way to the answer.

It skips embeddings, vector stores and chunking altogether.

The project calls this "vectorless RAG", and the label is fairly accurate. It appeared in September 2025 and drew quick attention, mostly for one reported figure: 98.7% accuracy on FinanceBench, against roughly 50% for traditional vector RAG on the same benchmark. Treat vendor-reported benchmark numbers with care and test on your own documents, but if the gap holds up even partly, it's a different league.

Where vector RAG breaks down

The core assumption of vector RAG is that the text most semantically similar to the query is also the most relevant. In practice that assumption fails all the time.

Ask "What was the company's total debt in 2023?" and the highest-scoring chunks may be:

the CEO letter discussing a debt management strategy,
a risk section on debt covenants,
a glossary entry defining long-term debt.

All three are close to the query in meaning, and none is the answer, which is a specific figure on the balance sheet on page 64.

Retrieval by reasoning

The shift in thinking is this. Instead of asking which chunks look like the query, PageIndex asks where a human expert would look in this document to answer it. That's a different retrieval strategy, and it suits structured, complex documents far better.

How it works

There are two phases.

Phase 1: build the tree index. Given a document, PageIndex doesn't chunk it. It reads it and constructs a hierarchy that mirrors the document's natural structure, like an intelligent, deeply nested table of contents. Each node stands for a section or subsection, with a title and a summary of its contents. The LLM writes those summaries during indexing, and they become the retrieval signal later.

For a 10-K filing, the tree might look like this:

Annual Report 2023
├── Business Overview
│   ├── Products and Services
│   └── Market Position
├── Risk Factors
│   ├── Financial Risks
│   └── Operational Risks
├── Financial Statements
│   ├── Balance Sheet
│   │   ├── Assets
│   │   └── Liabilities
│   └── Income Statement
└── Notes to Financial Statements
    ├── Note 1: Accounting Policies
    └── Note 12: Long-term Debt

The document is no longer flat. It has structure, and the LLM can move through it the way a person flips through a book.

Phase 2: tree search by reasoning. When a query comes in, nothing hits a vector database. The LLM receives the query and the top level of the tree and is asked to reason about where the answer may live. It might conclude that a question about long-term debt belongs in the Financial Statements section, probably the Notes, and return a node ID.

PageIndex then expands that node, showing the next level of detail, and the LLM keeps drilling until it reaches the content that answers the question.

A human analyst works the same way. Nobody reads a 200-page report from cover to cover. You check the contents, find the right section and go straight there. PageIndex does that at machine speed.

Apart from the last content fetch, each stage is the model reasoning. Nothing is matched by vector; the LLM works out where to look.

A small implementation

The following walk-through builds a simplified version of the idea in Python, with Gemini as the LLM.

Step 1: parse the document

Start by extracting raw text page by page. PyMuPDF handles PDFs cleanly.

import fitz  # pip install pymupdf

def parse_pdf(pdf_path: str) -> list[dict]:
    doc = fitz.open(pdf_path)
    pages = []
    for i, page in enumerate(doc):
        text = page.get_text().strip()
        if text:
            pages.append({"page_num": i + 1, "text": text})
    doc.close()
    return pages

Then group consecutive pages into sections. Unlike arbitrary chunking, this respects page boundaries and keeps related content together.

def group_pages_into_sections(pages, per_section=3):
    sections = []
    for i in range(0, len(pages), per_section):
        batch = pages[i : i + per_section]
        section_id = f"S{str(i // per_section + 1).zfill(3)}"
        combined_text = "\n\n".join(p["text"] for p in batch)
        sections.append({
            "section_id": section_id,
            "start_page": batch[0]["page_num"],
            "end_page": batch[-1]["page_num"],
            "text": combined_text,
        })
    return sections

Step 2: build the tree index with an LLM

Here lies the heart of the method. Embeddings are skipped, and the model reads a preview of each section and writes a structured summary. What comes out is a nested JSON tree, each node carrying a title, a short summary and a list of key topics.

from google import genai
from google.genai import types

client = genai.Client(api_key="YOUR_GEMINI_API_KEY")
def index_section(section: dict) -> dict:
    preview = section["text"][:1500]
    prompt = f"""Read this section from a document and summarize it.
Section pages: {section['start_page']} to {section['end_page']}
Text:
{preview}
Respond with ONLY valid JSON:
{{
  "title": "short descriptive title (5-8 words)",
  "summary": "2-3 sentence summary of what this section covers",
  "key_topics": ["topic1", "topic2", "topic3"]
}}"""
    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=prompt,
        config=types.GenerateContentConfig(temperature=0.0),
    )
    parsed = json.loads(response.text.strip())
    return {
        "node_id": section["section_id"],
        "title": parsed["title"],
        "pages": f"{section['start_page']}-{section['end_page']}",
        "summary": parsed["summary"],
        "key_topics": parsed["key_topics"],
    }

Run it over all sections and the complete tree is saved as a JSON file.

def build_tree_index(sections):
    nodes = [index_section(s) for s in sections]
    return {
        "title": "Your Document Title",
        "total_sections": len(nodes),
        "nodes": nodes,
    }
# Save for reuse
with open("tree.json", "w") as f:
    json.dump(tree, f, indent=2)

Step 3: tree search driven by reasoning

At this point the approach splits from vector RAG entirely. A question arrives, and instead of searching a database you give the LLM the whole tree and ask which sections most likely hold the answer. It behaves like an analyst scanning a table of contents.

def retrieve_sections(tree: dict, query: str) -> dict:
    # Build a compact text representation of the tree
    tree_text = f"Document: {tree['title']}\n\n"
    for node in tree["nodes"]:
        tree_text += f"[{node['node_id']}] Pages {node['pages']} | {node['title']}\n"
        tree_text += f"  Summary: {node['summary']}\n"
        tree_text += f"  Topics: {', '.join(node['key_topics'])}\n\n"
        prompt = f"""You are a document retrieval expert.
        Given this document tree, identify which sections most likely answer the question.
        Think step by step about where a human expert would look.
        {tree_text}
        QUESTION: {query}
        Respond with ONLY valid JSON:
        {{
          "reasoning": "your step-by-step reasoning about where to look",
          "selected_ids": ["S001", "S004"],
          "confidence": "high/medium/low"
        }}"""
            response = client.models.generate_content(
                model="gemini-2.0-flash",
                contents=prompt,
                config=types.GenerateContentConfig(temperature=0.0),
            )
            return json.loads(response.text.strip())

The reasoning field in the response is more than decoration. It records why the model picked those sections, which is what explainability means here. You can inspect the retrieval decision, something vector search can't offer.

Step 4: fetch the content

With the chosen section IDs in hand, pull the raw text of those sections. No vector lookup and no embedding computation, only a dictionary lookup on sections you've already parsed.

def retrieve_content(selected_ids: list, sections: list) -> str:
    section_map = {s["section_id"]: s for s in sections}
    context_parts = []
    for sid in selected_ids:
            if sid in section_map:
                sec = section_map[sid]
                context_parts.append(
                    f"--- Pages {sec['start_page']}-{sec['end_page']} ---\n"
                    + sec["text"][:3000]
                )
        return "\n\n".join(context_parts)

Step 5: generate the answer

The retrieved text goes to Gemini together with the original question, and the model composes a precise answer with page-level citations.

def generate_answer(query: str, context: str) -> str:
    prompt = f"""Answer the question using only the provided context.
Be specific. Include exact numbers, technical terms, and cite page numbers.CONTEXT:
{context}
QUESTION: {query}
ANSWER:"""
    response = client.models.generate_content(
        model="gemini-2.0-flash",
        contents=prompt,
        config=types.GenerateContentConfig(temperature=0.1),
    )
    return response.text.strip()

PageIndex against vector RAG

The two approaches differ along several lines. Vector RAG is fast, scales across large collections and needs embeddings and an index. PageIndex needs none of that but makes several sequential LLM calls per query, and in return it gives deeper accuracy on structured documents and a full reasoning trail.

When to choose PageIndex

It isn't a universal replacement for vector RAG. It's a different tool for a different job.

Reach for PageIndex when:

You work with a single long, structured document where accuracy outweighs speed: financial filings, legal contracts, regulatory documents, technical manuals, academic papers. Anything with meaningful hierarchy where "close enough" isn't enough.
You need explainability, because users or stakeholders must see exactly where an answer came from. You get a reasoning trace with section references and page numbers.
You build for regulated sectors such as finance, legal and healthcare, where a wrong answer has real consequences and the reported accuracy matters.

Stay with vector RAG when:

You search across a large collection and need to work out which documents are relevant. Vector RAG excels at that, whereas PageIndex goes deep into one document and isn't meant to pick the right one out of a thousand.
Throughput and latency matter to you. Several LLM calls in a row for each query get expensive and slow once traffic grows.
The material is brief, loosely organised or chatty. The strength is navigating hierarchy, and without hierarchy the advantage disappears.

Combine them. Use vector search to find the right document in your collection, then use PageIndex to pull the precise answer from inside it. A hybrid like that takes the best of both.

The bigger idea

What PageIndex really does is treat retrieval as a reasoning problem instead of a similarity problem. The shift matters beyond documents. Reports suggest Claude Code stopped relying on embedding search for finding code and now leans on reasoning-driven lookup. The pattern is getting clearer: when a task requires understanding structure and following logic, reasoning beats similarity.

Vector databases aren't going away. They remain fast, scalable and ideal for semantic search over large collections. But for the hard problem of finding the exact right information inside a complex document, the reasoning-first approach looks like the stronger option.

PageIndex is among the better implementations of that idea today. It's open source, has a clean API, and the project publishes a cloud service and an MCP server for plugging it into Claude or other agentic workflows without writing retrieval code from scratch. The official cookbook examples are a sensible start and can get a working system running in around half an hour. If your users ask precise questions about 200-page reports, it's worth a try.

← All articles Send the brief