Let Exact Terms and Meaning Work Together in Search

A developer searching for ERR_CACHE_STALE expects the matching error code. Another developer may describe the same issue as “the application keeps showing yesterday's data.” A useful knowledge search should handle both requests.

Lexical retrieval is valuable when the exact spelling carries information. Semantic retrieval can connect different descriptions of an idea. Combining the two gives each method a chance to find evidence the other misses.

The design decision is where that combination happens.

Give each retriever access to the corpus

Suppose semantic search retrieves twenty candidates, then a lexical scorer sorts those candidates. This can improve their order, but an exact match outside the initial twenty remains invisible.

For broader candidate coverage, run both retrieval methods over the eligible corpus and merge their results. Use the same source eligibility rules and filters in both paths. A document excluded because its identity is uncertain should not slip back in through the second retriever.

This builds directly on the source inventory: retrieval should operate on evidence you are prepared to use.

Merge ranks before inventing score arithmetic

A BM25 score and a cosine similarity score describe different calculations. Adding them directly gives a number, but the number has no automatic interpretation as combined relevance.

Reciprocal rank fusion offers a simple baseline. Each ranked list contributes a value based on a candidate's position:

fused_score(document) = sum(1 / (c + rank_in_list))

Ranks begin at one. A list contributes nothing for a document it did not retrieve. The positive constant c controls how sharply the contribution changes between positions.

The benefit is operational simplicity: the merge does not require the underlying score scales to match. The tradeoff is that it discards score magnitude. Two adjacent results receive similar contributions even when one retriever considered them very different.

Keep the constant and candidate counts in configuration, then evaluate them. They are design choices, not guarantees of relevance.

Deduplicate by identity, not by title

Two retrievers may return the same chunk. Merge that chunk by its stable identifier so both ranks contribute to one candidate. Do not count it as two independent pieces of evidence.

Titles are weak identifiers. Different articles can share a title, and one article can change its title. A useful chunk key includes a source identity, content version, and chunk location.

Also distinguish exact duplicate candidates from nearby chunks in one section. The latter may need consolidation later, as discussed in evidence-preserving chunking.

Test the query types separately

Use at least three groups of questions: exact identifiers, conceptual descriptions, and mixtures of both. Compare lexical search alone, semantic search alone, and the merged result.

Inspect whether the relevant evidence reaches the candidate pool before looking at answer quality. If fusion helps conceptual queries but harms exact error-code queries, the overall average can hide an important regression.

A small technical library may already perform well with lexical search. Add semantic retrieval when the measured misses justify it. Keeping a lexical baseline also makes the value of the extra machinery visible.

Pass a useful shortlist downstream

Fusion produces a shortlist, not a factual judgment. A highly ranked document may still be outdated, incomplete, or irrelevant to an important qualifier in the question.

The next stage can rerank candidates using a stronger relevance signal. But no later ranking stage can rescue evidence that never entered the shortlist. Measure candidate coverage first; otherwise, you may optimize the ordering of the wrong material.