ROE Versus TechRAG: GraphRAG on Docker and Qdrant or on SQLite and LanceDB

Seeing an architecture reimplemented independently by someone with different tools is a useful test of whether the design was ever about the tools. That's the case with the RAGraph Ontological Engine (ROE), a local GraphRAG design, and TechRAG, an open-source MIT-licensed project that adopts its thesis but swaps nearly every infrastructure choice. Comparing the two shows which ideas carried over and which got reinterpreted.

The shared thesis

A conventional RAG system pulls chunks that resemble the question, then leaves the hardest part to the model: rebuilding relationships, causes, dependencies and constraints between concepts by itself. ROE answers with a three-layer hybrid:

1. Vector search through Qdrant, with bge-m3 embeddings produced locally by Ollama
2. A concept dictionary (the domain ontology), seeded from a sample spread across the document
3. A knowledge graph of nodes, edges and atomic facts that point back to their original sources

The goal is modest. Skip stuffing a full manual into the prompt. The model instead receives a small working set: a handful of chunks, some atomic facts and a few graph paths. That means less clutter, tighter context and answers a reader can verify. The original ROE ran on Docker (Qdrant plus MongoDB) and offered a web interface with ontological chat and graph visualization, along with an audit system reporting coverage metrics and reasoning scores.

TechRAG keeps that thesis and makes choices that nearly mirror those.

Storage: from two services to one file

ROE stores vectors in Qdrant and everything else (documents, ontology, audit) in MongoDB. That means two external services, Docker and exposed ports. TechRAG needs no external infrastructure. LanceDB, an embedded serverless vector database, writes straight to the filesystem, and SQLite via SQLAlchemy holds the rest: documents, chunks, entities, atomic facts, semantic paths and the serialized ontology, all inside one roe.db file.

The first layout looks like this:

# ROE (original)
Qdrant (vectors) + MongoDB (docs/ontology/audit)
→ Docker, ports 6333 and 27017

And TechRAG's looks like this:

# TechRAG
LanceDB (vectors) + SQLite (everything else)
→ Zero external services, one .db file

Setup changes completely. ROE needs Docker with both containers running, while TechRAG begins with uv run roe ingest document.pdf and just works. One design suits a controlled environment, the other works nearly anywhere, even a developer laptop.

The graph as relational tables

ROE planned Neo4j for its graph, with MongoDB as a lighter choice for storing documents. TechRAG brings something else, SQLGraph. Nodes and edges become relational tables in SQLAlchemy and no graph database is involved. The entities table stores nodes with a name, an ontological type and a description. The facts table stores edges with subject_id, predicate, object_id, source_text and confidence.

It isn't a native graph, though it functions well, answers to plain SQL JOINs, and carries a benefit Neo4j can't match: no infrastructure dependency. A breadth-first search written in pure Python walks the relationships to assemble the context passed to the model. For prototypes and moderately sized documents, a graph on SQLite is more than enough.

A far smaller model

The gap in model size is big. ROE runs gpt-oss:120b-cloud, a 120-billion-parameter model, while TechRAG gets by on qwen2.5:7b, about 17 times smaller, and results hold up. According to the README, a smoke test on a 50-chunk technical PDF yields 263 entities and 213 atomic facts. Questions then come back with evidence_coverage=0.6, from a context of only 4 chunks and 10 facts.

That suggests the quality of hybrid retrieval, vector similarity combined with an ontological graph, makes up for a weaker model. When the context is picked this carefully, even a modest LLM produces coherent and verifiable answers.

CLI instead of a web app

ROE ships a web app: chat, a graph view, a Facts Composer for editing facts, and an Audit Trail of past queries. TechRAG is command line only:

uv run roe ingest manual.pdf
uv run roe query "Which procedure reduces the risk associated with alarm A1?"
uv run roe stats

There's no GUI and no graph view, just the terminal. Still, query prints a full JSON document with the answer and its audit data: chunks used, facts, nodes, edges, paths, evidence_coverage, reasoning_score and recovered. Convenience is traded for easier use inside automated pipelines.

What stayed the same

Under the architectural differences, the core algorithm closely follows the original specification. Both systems run the same flow: parse documents (PDF through pypdf, Markdown or plain text), cut chunks of 1000 characters with a 200-character overlap, embed with bge-m3 on an Ollama endpoint, bootstrap the ontology from a distributed sample of chunks, and extract atomic facts per chunk with a back-reference to the source. The extraction prompts differ in form, since TechRAG's are written in German, but the idea is identical: have the model return JSON triples of subject, predicate, object and source.

Retrieval parameters match almost exactly. The table below sets them side by side:

The values agree, and so does the philosophy: begin with a small, highly targeted context and widen it only when needed.

Recovery logic is shared too. If the best chunk scores under 0.45, neither system blindly enlarges the context. Both rewrite the query, TechRAG through a dedicated prompt that asks the model for two alternative queries using different terminology, and retry retrieval. If evidence coverage falls below 40%, both launch a second attempt with more context, which TechRAG implements with max_recovery_rounds=1.

Choices beyond the original design

A few TechRAG decisions go past the specification.

Provider-agnostic client. Instead of Ollama's native API, it uses openai.OpenAI aimed at Ollama's /v1 endpoint, so identical code runs against Ollama, OpenAI, vLLM or any endpoint that is OpenAI-compatible:

# TechRAG clients.py
self._client = OpenAI(api_key=self.api_key, base_url=self.base_url)
resp = self._client.chat.completions.create(model=self.model, ...)

ROE is tied to Ollama, while TechRAG isn't. It's a subtle difference with real power: switching to GPT-4o later takes no code changes.

Modern packaging. TechRAG is a proper Python package with pyproject.toml, managed by uv, Astral's package manager, which runs far faster than pip. A clean roe/ package, a CLI in __main__.py and explicit dependencies make it reproducible, since uv sync is all that's needed.

A cheaper reasoning score. ROE derives its reasoning score from claim classification (Verified, Contradicted, Unsupported) performed by the model. TechRAG does the same, with an added heuristic: it matches the words "verified"/"verifiziert", "contradicted"/"widerspr" and "unsupported"/"nicht belegbar" in the response with a regex, so no second model call is needed. It's faster and less precise.

A real BFS over SQL. The graph_context_for function starts from seed entities taken from retrieved chunks, visits nodes breadth-first, gathers inbound and outbound edges, and stops at max_nodes or max_edges. The storage isn't a native graph, but the traversal behaves exactly like one.

Careful similarity handling. In vector_store.py, vectors get L2-normalized before insertion and before each query. Cosine similarity is then computed with a double fallback: it first assumes squared L2 distance, the Faiss default, and if the result lands outside the [-1, 1] range, it applies the linear-L2 formula. The returned score is therefore always a clean cosine value from 0 to 1, whichever index LanceDB uses underneath. Details like this separate a prototype that works sometimes from one that works every time.

Lessons

The first is confirmation. When an independent builder reimplements an architecture with another stack and gets working results, the idea wasn't bound to its original implementation. It was a transferable principle. Vector, ontology and graph layers don't need Qdrant or Neo4j, and they fit in LanceDB and SQLite just as well.

The second is simplicity. ROE needs Docker, Qdrant, MongoDB and a web app. TechRAG needs LanceDB, SQLite and a CLI. For prototypes, proofs of concept and CI/CD integration, the second path is objectively better, and the comparison is a reminder of how often developers over-engineer infrastructure that the problem doesn't call for.

The third concerns the model. A 7B model matching results of a 120B one, given well-structured context, shows that retrieval quality in GraphRAG matters at least as much as model size, and maybe more. A carefully chosen context of few chunks, relevant facts and graph paths lets a modest LLM reason precisely, while noisy context confuses even the strongest model.

With TechRAG in view, a redesign of ROE would likely take several steps:

• Use the OpenAI-compatible abstraction so any provider can be plugged in
• Evaluate LanceDB as an embedded stand-in for Qdrant to lower the barrier to setup
• Keep SQLite for document storage in lighter cases, and bring in Neo4j only when graph complexity justifies it
• Retain the web UI for graph exploration and fact correction, which remains essential, with a CLI added for scripted runs

TechRAG isn't a better ROE. It's a different one, aimed at accessibility and setup speed, and in many scenarios that's the right priority.

Why it matters

Both projects point at the same conclusion: keep ontology, knowledge graph and vector search separate, extract atomic facts with source back-references, limit the context to a few well-chosen elements, and verify answers with coverage metrics. They also show the same result can come from very different stacks, and that the simpler pick, LanceDB over Qdrant, SQLite over MongoDB and qwen2.5 over a 120B model, is often the right place to start.