Dartboard RAG: What to Do When Top-K Returns the Same Chunk Three Times

Plain top-k retrieval fails quietly. Whenever a corpus says the same thing in several places, the k nearest chunks for a query can turn out to be a single chunk saying it three times in different words.

Picture a 100-page report on climate change in which five sections each discuss greenhouse gases, framed a little differently but making one point. Someone asks "What causes climate change?" and the FAISS index returns this:

Top 3:
1. "Greenhouse gases trap heat in the atmosphere, causing warming..."  (sim 0.91)
2. "Atmospheric greenhouse gases are the primary driver of climate change..."  (sim 0.89)
3. "The trapping of heat by greenhouse gases leads to rising temperatures..."  (sim 0.88)

That's three chunks and effectively one fact. The user asked about causes, plural, and received one cause on repeat. Chunks about deforestation, industrial agriculture and land-use change were sitting at ranks 8, 12 and 17, so none reached the LLM's context. The model's answer mirrors its input: a fluent paragraph about greenhouse gases and no mention of the other causes.

Vector search isn't the culprit. It did what it was told, returning the top-k by similarity. The trouble is that similarity has no concept of redundancy. Three chunks can all be maximally close to the query and also be near-copies of one another, and a plain top-k will return all three while squeezing out different, relevant material.

Dartboard RAG changes the selection rule. Where standard retrieval asks for the k chunks closest to the query, this approach asks for k chunks that are each highly relevant and also far apart from one another. What you get covers the relevant parts of the corpus and not several copies of its single best match.

Picture a dartboard

Treat the query as the bull's-eye and every chunk as a dart already stuck in the board. The nearer the centre, the more relevant the chunk.

Chunks plotted by relevance to query
                    (closer to center = higher cosine sim)

                       ●●●         ← cluster of near-duplicate chunks
                      ●●●          (all about "greenhouse gases")
                      ● bull's-eye = QUERY

                                ●     ← chunk about deforestation
                                       (relevant but different topic)

                    ●           ●     ← chunks about agriculture, ocean carbon

                       ●  ●          ← chunks about historical climate


   STANDARD TOP-3 picks:           DARTBOARD TOP-3 picks:
   3 closest darts                 1 closest, then darts that are also
   = 3 darts in the same           good but spread across the board
     spot near bull's-eye          = better coverage of relevant content

Ordinary retrieval collects the three closest darts. The dartboard method collects three darts that are close and also spread out.

The overall pipeline

Architecturally this is standard retrieval with smarter selection stacked on top. The vector store and embeddings stay exactly as they were. The only new piece is a greedy selection step that decides which of 15 candidates make it into the final 5.

Step 1: fetch more than you need

The algorithm can't choose between candidates it doesn't have, so it needs a pool larger than the final k. The reference implementation multiplies by an oversampling factor of 3:

fetch_k = self.k * self.oversampling     # default: 5 × 3 = 15
candidates = vector_store.search(query_embedding, k=fetch_k)

So you pull 15 candidates and not 5. FAISS returns them ranked by cosine similarity, which puts candidate 0 closest to the query and candidate 14 furthest. The selection step then chooses the best subset of five by weighing relevance and diversity together.

The next step needs the embedding of each candidate and not just its text, so the pool is fetched as full Document objects with embeddings attached.

Step 2: build two distance matrices

From the candidate embeddings, compute two matrices:

# Normalize all vectors so dot product = cosine similarity
query_norm = query_vec / np.linalg.norm(query_vec)
cand_norm = candidate_matrix / np.linalg.norm(candidate_matrix, axis=1, keepdims=True)

# Distance = 1 - cosine_similarity
query_distances = 1.0 - np.dot(query_norm, cand_norm.T)        # (1, N)
document_distances = 1.0 - np.dot(cand_norm, cand_norm.T)      # (N, N)

• query_distances holds the distance from the query to each candidate. A low value means highly relevant. Its shape is (1, N).
• document_distances holds the pairwise distance between every two candidates. A low value means those candidates resemble each other. Its shape is (N, N).

Relevance comes from the first matrix and diversity from the second.

Step 3: turn distances into log-probabilities

Raw distances don't combine well. The gap between 0.3 and 0.4 can feel meaningful, yet plain subtraction gives 0.1 and understates it. So the algorithm maps distances to log-probabilities under a Gaussian kernel:

def lognorm(dist, sigma):
    return -np.log(sigma) - 0.5 * np.log(2 * np.pi) - dist**2 / (2 * sigma**2)

That is the log of a Gaussian density. In intuitive terms, a small distance gives a high probability (close means likely relevant, or likely similar), a large distance gives a low one, and sigma sets how sharp the curve is. A small sigma, with a default of 0.1, produces tight peaks that react strongly to small distance differences. A larger one such as 0.5 flattens things so that similar distances are treated more alike.

Working in log space matters for numerical stability. With small distances and a small sigma, raw probabilities shrink to tiny floats that underflow to zero, whereas log-probabilities stay bounded and comparable.

Step 4: the greedy loop

This is where the dartboard logic actually sits. The procedure goes like this:

1. Take the most relevant chunk first. No diversity test applies yet, since nothing has been selected to differ from.
2. Maintain max_distances, which records for each candidate its largest log-probability distance to any chosen document. Higher means more different from the current selection.
3. For each remaining slot, score every unchosen candidate by combining a relevance term (how close it is to the query, from query_probs) with a diversity term (how far it is from the nearest already-selected candidate, from max_distances). The combination is diversity_weight × diversity + relevance_weight × relevance.
4. Aggregate the combined scores with logsumexp, which is numerically stable.
5. Choose the candidate with the highest aggregate, add it, and refresh max_distances.
6. Repeat until k candidates are selected.

In code:

# Step 1: pick most relevant first
most_relevant_idx = np.argmax(query_probs)
selected_indices = [most_relevant_idx]
max_distances = doc_probs[most_relevant_idx].copy()    # diversity tracker

# Step 2-6: iteratively add diverse + relevant chunks
while len(selected_indices) < num_results:
    # For each candidate, compute "diversity from any selected"
    updated_distances = np.maximum(max_distances, doc_probs)

    # Combine relevance + diversity
    combined = (diversity_weight * updated_distances
                + relevance_weight * query_probs[np.newaxis, :])

    # Aggregate per candidate (logsumexp for numerical stability)
    normalized = logsumexp(combined, axis=1)

    # Mask already-selected
    for idx in selected_indices:
        normalized[idx] = -np.inf

    # Pick the best
    best_idx = np.argmax(normalized)
    max_distances = updated_distances[best_idx]
    selected_indices.append(best_idx)

The core idea is the combined score. Each pick weighs "is this relevant?" against "is this different from what is already held?" The first pick maximises pure relevance, and every later pick balances both.

What the maths is doing, informally

At each step the loop effectively asks: of the candidates not yet picked, which one best improves both its relevance score and the coverage of the relevant region?

Imagine a 2D scatter with relevance along the x-axis and distance from the selected set along the y-axis. The first pick is the point furthest right. Afterwards every other candidate has a y-value, which is its distance from that first pick.

For the second pick, you want something far to the right (relevant) and also high up (unlike the first). With both weights at 1.0 the trade-off is even, so the loop will take a chunk that's a bit less relevant but considerably more different.

Relevance →
            Low              ●◯

                   ●         ●           ←  Standard top-k picks these 3
                                              (highest relevance, regardless of diversity)
                    ●        ●
                       ●●●●●●  ●  ●  ●     ← Many similar high-relevance chunks
            High         (cluster)


            Diversity ↓
            from
            selected
                        ↓
                     ↓     ↓   ←  Dartboard picks 1 from cluster,
                                 then far-away ones with high relevance still

Plain top-k takes the rightmost points no matter how bunched together they are. Dartboard takes the rightmost first and then deliberately fans out to other relevant regions.

The two weights

relevance_weight = 1.0     # how much we care about chunks being close to query
diversity_weight = 1.0     # how much we care about chunks being different from each other

Both default to 1.0, meaning equal emphasis. Moving them shifts behaviour: raise diversity if duplicates keep appearing, and raise relevance if off-topic chunks are creeping in. The defaults suit most corpora.

A worked example with a duplicate-heavy corpus

The reference implementation has a duplicate_factor parameter that repeats every chunk N times, a useful way to stress-test diversity on a synthetically dense corpus. Set it to 5 and ask "What causes climate change?"

Standard top-5 retrieval gives this:

1. "Greenhouse gases cause warming..."  (sim 0.91)
2. "Greenhouse gases cause warming..."  (sim 0.91)  ← DUPLICATE
3. "Greenhouse gases cause warming..."  (sim 0.91)  ← DUPLICATE
4. "Greenhouse gases cause warming..."  (sim 0.91)  ← DUPLICATE
5. "Greenhouse gases cause warming..."  (sim 0.91)  ← DUPLICATE

Unique results: 1/5

The five results are literally one chunk, all equally close to the query, so the LLM receives one fact restated five times.

Dartboard top-5 gives this:

1. "Greenhouse gases cause warming..."  (highest relevance — wins first pick)
2. "Deforestation reduces the carbon sink..."  (different chunk, still relevant)
3. "Industrial agriculture emits methane..."  (third unique cause)
4. "Land-use changes alter surface albedo..."  (fourth unique cause)
5. "Fossil fuel combustion is the largest CO₂ source..."  (related to #1 but different angle)

Unique results: 5/5

Once the first chunk is chosen, the diversity term penalises every copy of it, since each has zero diversity relative to the selection. The loop therefore reaches for distinct content in slots two to five, and the LLM sees five different facts about the causes. That's the dartboard's best case. Real corpora seldom contain exact duplicates, so the effect is milder, though the same dynamic applies to near-duplicates.

The whole thing in about twenty lines

def dartboard_select(query_emb, candidate_embs, k=5, sigma=0.1):
    # 1. Compute distance matrices
    query_dist = 1 - cosine(query_emb, candidate_embs)        # query→each
    doc_dist   = 1 - cosine(candidate_embs, candidate_embs)   # each→each

    # 2. Convert distances to log-probabilities
    query_probs = lognorm(query_dist, sigma)
    doc_probs   = lognorm(doc_dist, sigma)

    # 3. Pick most relevant first
    selected = [np.argmax(query_probs)]
    max_distances = doc_probs[selected[0]].copy()

    # 4. Iteratively add diverse + relevant
    while len(selected) < k:
        updated = np.maximum(max_distances, doc_probs)
        combined = updated + query_probs[np.newaxis, :]   # equal weights = sum
        scores = logsumexp(combined, axis=1)
        for idx in selected:
            scores[idx] = -np.inf       # don't re-select

        best = np.argmax(scores)
        max_distances = updated[best]
        selected.append(best)

    return selected

The length is modest. The difficulty lies in what the maths means and not in the line count. Once the rule is clear (pick the most relevant chunk, then favour candidates that are both relevant and unlike what's already chosen) the code is easy to follow.

Parameters worth adjusting

Two settings deserve attention.

• oversampling sets the size of the candidate pool. The default of 3 means 15 candidates for a final 5. Going higher, to 5 or 10, gives the algorithm more to work with but slows the pairwise distance computation. In a badly duplicated corpus, 10× oversampling lets dartboard reach further down the ranking for truly distinct chunks.
• diversity_weight should go up if your evaluations show duplicates still slipping through and down if results scatter too much, with off-topic chunks selected merely for being "different". The default of 1.0 is balanced, and values of 1.5 or 2.0 are reasonable for very dense corpora.

sigma is more delicate and rarely needs changing. The default of 0.1 is reported to work for cosine distances on OpenAI embeddings. If you move to an embedding model with different distance scales you may need to recalibrate it, though most people can leave it alone.

Where it helps and where it doesn't

Dartboard suits several situations.

• Corpora with near-duplicates. Many sources covering one topic, repetitive policy documents, news archives with overlapping coverage, indexed forum posts. Standard top-k routinely stumbles here.
• Coverage mattering as much as precision. Research summaries, thorough Q&A and comparative analysis, where the user wants the whole space of relevant information and not one best chunk.
• Spare compute. The extra cost is matrix arithmetic on the candidate pool, not additional LLM calls. It's imperceptible for k=5 with 15 candidates and noticeable for k=20 with 60 or more.

It helps less in these cases.

• Naturally diverse corpora. If chunks are already topically distinct, through semantic chunking or well-structured documents, top-k returns varied chunks anyway, and dartboard adds overhead without altering the ranking.
• Very specific questions. "What's the boiling point of mercury?" has one good answer. Diversity logic may pick weaker chunks for the sake of variety and hurt the answer.
• Tiny k. With k of 1 or 2 there's little room to diversify, so ordinary top-k is fine.
• Pipelines that already combine fusion, reranking and windowing. Each of those tackles related problems in its own way. Adding dartboard on top of a rich pipeline seldom helps and adds complexity.

The idea to carry away

The interesting move in Dartboard RAG isn't the maths. It's the recognition that relevance alone is the wrong objective for top-k retrieval.

Standard top-k asks which k items are individually most relevant. That treats every item independently and ignores what's already in the result set, so when several equally relevant items exist, you get all of them even when they say the same thing.

The better objective works on the set: which k items, taken together, best cover what the query needs? That's a different optimisation, in which item k's score depends on items 1 through k-1 and not just its own relevance. Greedy methods like dartboard are tractable approximations to it.

The pattern turns up well outside retrieval:

• Training data sampling. Active-learning techniques use diversity-aware sampling for the same reason.
• Recommenders. Suggesting three slight variants of one product helps nobody, so diversity improves the list.
• Image search. Ten near-identical pictures serve a user worse than ten different relevant ones.
• Extractive summarisation. Choosing sentences needs variety, not just per-sentence importance.

The general lesson is that when you're choosing a set of things, you should score sets and not individual items. Dartboard is the RAG version of that principle, and once you've absorbed it you'll see the same trap in many other "rank, then take the top k" pipelines.

There is a price: extra computation, and it's harder to explain why one chunk beat another. The reward is better coverage, which often translates straight into better answers, especially when the corpus isn't neatly arranged as one chunk per distinct fact.