Home / Articles

Hybrid Retrieval for RAG: Combining Keyword, Vector and Graph Search

How to combine BM25, vector search and graph retrieval in RAG: rank fusion with RRF, where GraphRAG helps, and a retrieve, fuse, expand, rerank architecture.

Publisher: REACTAPP.TOP6 min read1330 words

This article is published in English.

Sources: Hybrid Retrieval for RAG: Combining Graph, Vector, and Keyword Search

No single retrieval method answers every question. Keyword search is best at exact words, vector search at meaning and graph retrieval at relationships. Real questions tend to need all three at once, which is why search platforms and GraphRAG systems now tend to stack lexical, semantic and graph-aware retrieval rather than asking one technique to carry the whole load.

Why one method falls short

Keyword search stays essential because it's tied to the terms that really appear in the data. Classic information retrieval builds an inverted index out of a dictionary and postings lists, and BM25-style scoring refines plain term matching by accounting for term frequency and document length. Lexical engines also give controls that semantic systems don't offer naturally: quoted phrases, boolean operators, field constraints, token analyzers and predictable matching. When the exact wording matters as much as the meaning, those controls win.

Embeddings solve the opposite weakness. Text is mapped into vectors, and a query pulls back whatever lies closest, even when the wording differs completely from the question. The DPR paper made the case with numbers: across a handful of open-domain question answering datasets, dense retrieval scored 9 to 19 points above a tuned Lucene BM25 baseline when judged on top-20 accuracy. At scale, nearly every deployment uses approximate nearest-neighbor indexes, frequently HNSW, so lookups return in milliseconds.

A graph supplies what neither of the other two can: explicit structure. Nodes can stand for entities, and edges for relationships, with communities, claims and multi-document paths layered on top. Surveys of GraphRAG describe the gain as precision and completeness from entity-level structure. Microsoft's implementation separates local search, aimed at questions about one entity, from global search, aimed at questions about the entire corpus. KG²RAG shows another use: semantically matched seed chunks go in, and a knowledge graph grows them into a coherent, varied set of supporting evidence.

Two meanings of "graph"

The word graph is overloaded here. HNSW, among the most widely deployed approximate nearest-neighbor methods, is itself a graph, built so a search can hop quickly toward close vectors. Neo4j's vector index documentation exposes its settings, vector.hnsw.m and vector.hnsw.ef_construction.

When people discuss mixing graph, vector and keyword search, two different graphs are involved. One is an internal index structure that makes similarity search fast. The other is an external knowledge graph that models real entities and the links between them. The first helps locate similar vectors quickly, while the second supports reasoning over connected facts. Confusing the two leads to muddled architecture discussions.

How hybrid retrieval works in practice

The usual starting point is two searches at once, one lexical and one semantic. Weaviate calls its approach hybrid search: vector and keyword queries both run, and their result sets are fused. OpenSearch gets there through search pipelines that either normalize scores or fuse by rank before returning anything. Neo4j's HybridRetriever queries a vector index and a full-text index, then folds both into a single ranked list. Each side covers the other's blind spot. Keywords bring back exact evidence, and embeddings bring back passages that mean the same thing in different words.

Merging the lists

The next choice is how to combine the rankings. Reciprocal Rank Fusion (RRF) is a strong default because it works on rank positions instead of raw scores. Elastic's documentation describes it as needing little or no tuning and not requiring comparable score scales. That matters, since BM25 scores and embedding similarities usually live on very different numeric ranges.

When you want more control, normalized weighted fusion is an option. Weaviate supports score-based fusion, and OpenSearch offers both normalization-based processors and rank-based fusion. Elastic also points out that a linear combination can beat RRF if the weights are calibrated well, though that demands careful normalization and experimentation.

Where the graph comes in

The graph layer usually joins after first-stage recall. Neo4j's HybridCypherRetriever shows the idea: candidates come from vector and full-text search, and then a graph traversal returns richer context around them.

Microsoft's GraphRAG local search follows a comparable path, mixing context taken from the graph with the original text. Global search takes a different route and answers dataset-wide questions from community reports computed ahead of time. That's the step beyond "BM25 plus embeddings": find a good node or chunk, then walk outward and collect facts the matched passage never contained.

When GraphRAG earns its cost

Graph-aware retrieval pays off when a question is about connections rather than single passages. Microsoft's docs spell out the division: local search for specific entities, global search for whole-dataset reasoning, DRIFT for local reasoning extended with community context, and basic search for questions plain vector retrieval already answers well. Graphs help most when the user wants the system to connect dots, trace dependencies, summarize themes or follow several hops of evidence.

They aren't a free relevance upgrade, though. A newer GraphRAG benchmark paper asks directly when graphs are useful and finds that GraphRAG can underperform vanilla RAG on many real-world tasks. Its central point is that graph structure pays off when the workload truly calls for hierarchical retrieval, relationship-aware reasoning or broad contextual synthesis. For simple, local queries, building the graph, traversing it and summarizing it may buy little. A good hybrid design is selective instead of maximal.

A reference architecture

Preparation comes first. Chunk the documents, pull out entities and relationships where that's worthwhile, and keep a tidy mapping from each chunk to the graph objects it mentions. Three structures then sit side by side: a lexical index for matching tokens, a vector index for meaning, and a graph layer for traversal or community expansion.

Queries follow four moves, retrieve, fuse, expand and rerank. Lexical and vector searches go first, and their results get fused. Only the most confident hits are expanded through the graph, and a reranker orders the enlarged evidence before the generator sees it. Cost stays bounded, and both recall and context quality hold up.

Where things are heading

Vendors and researchers are converging on integrated stacks. Elastic, Weaviate, OpenSearch and Neo4j all ship hybrid retrieval as a standard capability instead of a workaround. Milvus documents a hybrid setup built on BGE-M3, a model whose paper describes dense, sparse and multi-vector output from one network. The trend points to fewer separate retrieval modes and more systems that derive several relevance signals from the same data and blend them when a query arrives.

Practical advice

The biggest mistake is treating hybrid retrieval as one algorithm. It's a strategy:

  • Begin with keyword precision paired with semantic recall.
  • Reach for RRF while score scales don't line up.
  • Switch to weighted fusion after you've built evaluation data.
  • Add graph expansion only where relationships matter.

Relevance isn't the only limit, either. Vector indexes take real memory and add latency, and Neo4j's operations guide says its Lucene-backed vector indexes lean heavily on the OS filesystem cache. Knowing when to stay simple is part of the design.

Hybrid retrieval works because it mirrors how people look for knowledge. Sometimes you remember the exact phrase, sometimes only the idea, and sometimes it's the chain of links that explains how two facts relate. Keyword, vector and graph retrieval each serve one of those needs, and the strongest search systems combine them on purpose.