30 RAG and Vector Database Interview Questions With Concise Answers

These thirty questions come up repeatedly when teams interview for roles that touch retrieval-augmented generation and vector search. The answers are written the way a strong candidate might give them: short, with the trade-off stated and the measurement named.

The basics

1. What is RAG in one sentence?

It's a pattern where the system fetches evidence from an outside knowledge source at question time and puts it in front of the model before the model writes anything. The usual path runs from query processing through retrieval, reranking and context building to generation and a final verification step. It lets an organization use private, current and citable material without retraining a model. Retrieval doesn't guarantee truth, though, so grounding, citations, permissions and answer quality still need checking.

2. When does RAG make sense?

Reach for it when answers hinge on private documents, information that changes often, big collections, traceable evidence or per-user and per-tenant access rules. It is usually unnecessary for summarizing text the user already pasted in, for exact calculations, for changing writing style, or for live structured data that an API or database can return directly.

3. How do RAG, fine-tuning and long-context prompting differ?

RAG supplies knowledge at inference time. Fine-tuning changes the model's learned behavior. Long-context prompting stuffs more source text into the prompt itself.

• Pick RAG for knowledge that changes, is private or must be cited.
• Pick fine-tuning for behavior, tone, format or a specialized task.
• Pick long context when the corpus fits and the task needs comparison across all of it.
• Pick tools or APIs for live transactional data.

They combine well: a tuned model can pull policy documents through RAG and look up live account figures through an API call.

4. Is a vector database the same thing as a RAG system?

No, it is a single building block. It holds embeddings, answers similarity queries, filters on metadata and handles index upkeep. A working production system needs far more around it: document parsing, chunking, access control, hybrid retrieval, reranking, prompt assembly, generation, citations, evaluation, monitoring and deletion handling. When a RAG system disappoints, the cause is more often sloppy corpus preparation or no evaluation than the choice of database.

5. How do dense, lexical and sparse retrieval differ?

Dense retrieval compares embeddings and handles paraphrase and concepts well, but it can miss exact identifiers. Lexical retrieval such as BM25 scores terms by frequency and rarity and shines on names, product codes, error messages and jargon. Sparse neural retrieval such as SPLADE learns term expansion while keeping an inverted index. A strong production baseline normally blends lexical and dense results.

6. Why keep BM25 in a semantic-search stack?

It is excellent on exact words, rare terms, acronyms, error codes, legal citations and product identifiers, which embeddings may represent poorly. It is also cheap and transparent. Since the two methods tend to fail on different queries, combining them usually lifts recall.

7. Bi-encoder versus cross-encoder?

A bi-encoder embeds the query and each document separately, so document vectors can be computed ahead of time and searched across millions of records. A cross-encoder reads the query and a document together, which scores relevance more accurately but requires a separate pass per candidate. The standard design uses a bi-encoder for broad recall and a cross-encoder to rerank for precision.

8. Cosine, dot product or Euclidean distance?

Cosine compares the angle between vectors, dot product considers alignment and magnitude, and Euclidean measures straight-line distance. When embeddings are L2-normalized, cosine and dot product rank identically. Follow the embedding model's training objective and model card, since a mismatched metric can noticeably hurt retrieval.

Vector search and retrieval

9. What is approximate nearest-neighbor search?

ANN finds vectors likely to be closest to the query without comparing against every stored vector. It cuts latency and cost at scale but may miss some true neighbors, so measure recall against exact search. The core trade is speed and cost against possible recall loss.

10. How do flat, HNSW, IVF and quantization compare?

• Flat: compares everything and is exact, but costly at scale.
• HNSW: walks a navigable graph for high recall and low latency, at the cost of extra memory.
• IVF: groups vectors into centroid partitions and searches a few of them; raising nprobe improves recall and adds latency.
• Product or scalar quantization: compresses vectors to save memory and storage, with a measurable accuracy cost.

Choose by corpus size, filters, memory, update pattern, concurrency and the recall you need.

11. What is hybrid search and why use it?

It blends dense results with lexical or sparse ones. Semantic matches come from the embedding side and exact terminology from the keyword side, and the lists are merged by reciprocal rank fusion, weighted scores or normalized-score fusion. The benefit is greatest on enterprise data that mixes plain-language concepts with case numbers, policy codes, part IDs and error messages.

12. What is reciprocal rank fusion?

RRF merges ranked lists using positions and not raw scores. Each document receives, for every list, the value 1 divided by (k plus its rank), and the values are summed. Because dense and lexical scores live on different scales, adding them directly is unreliable. RRF sidesteps calibration and rewards documents ranked high by several retrievers.

13. Why rerank?

First-stage retrievers optimize for speed and recall, so their candidate lists contain weak matches. A reranker applies a more accurate model to a limited pool and lifts the best evidence to the top, which improves context precision and spares the generator irrelevant text. Tune the candidate count against Recall@k, reranker latency, cost and performance on key query types.

14. What does maximal marginal relevance do?

MMR trades relevance against redundancy when choosing passages: roughly, a lambda-weighted relevance score minus the complement of lambda times similarity to what has already been picked. Without it, the system may return several near-duplicate chunks from one section. It improves coverage when results are repetitive, but too much diversity admits less relevant passages.

15. Why can metadata filters hurt ANN quality?

If ANN search runs first and filters apply afterward, many top candidates may be discarded for belonging to the wrong tenant, product, date range or document version, leaving fewer results than requested. Remedies include integrated or pre-filtered search, metadata indexes, oversampling candidates, per-tenant partitions, iterative search, and exact search for very selective filters. Authorization filters must be enforced as trusted retrieval constraints and not left as prompt instructions.

Data preparation

16. How do you choose a chunking strategy?

Decide by experiment, guided by document structure and how people query. Small chunks match precisely but lose context, while large ones keep context yet dilute embeddings, raise cost and bury evidence. The menu includes fixed-token, recursive, structure-aware, semantic, sentence-window and parent-child chunking. Compare them per document type on retrieval recall, whether boundaries cut ideas in half, duplication, context size and the quality of final answers.

17. What is parent-child retrieval?

Small child chunks are indexed for precise matching, but the model receives the larger parent section. If one paragraph of a policy matches, the system returns the surrounding section with its definitions, exceptions and conditions. That balances precision with completeness, provided the expansion is capped so it doesn't flood the context window.

18. What are contextual retrieval and late chunking?

Contextual retrieval enriches each chunk with document-specific context, such as the title, section path or a short explanation, before embedding. Late chunking encodes a longer stretch of the document first and then pools token representations into chunk embeddings, so local chunks keep information from the wider text. Both help with ambiguous chunks but add ingestion complexity, so validate them against simpler baselines.

19. What does a solid ingestion pipeline contain?

1. Collect documents and record ownership, classification, ACLs and retention.
2. Parse text, headings, tables, pages and layout.
3. Clean OCR noise and repeated boilerplate.
4. Cut structure-aware chunks.
5. Attach metadata, provenance and parent-child links.
6. Generate dense embeddings and sparse representations.
7. Upsert with deterministic identifiers.
8. Verify counts, checksums, ACLs, deletions and retrieval canaries.
9. Version the parser, chunker, embedding model and index generation.

It must also handle updates, reprocessing, rollback and verified deletion.

20. How should hard PDFs and tables be handled?

Parse born-digital PDFs with a layout-aware tool, so headings, columns, reading order, captions, tables and page numbers remain linked. Scans need OCR with confidence scores, and sometimes bounding boxes or page images. Keep each table's headers, units, identity, page and cell coordinates. For numeric questions, retrieve the table structure and compute the result deterministically. Improving the parser often gains more than swapping the embedding model.

Advanced patterns

21. What are query rewriting, multi-query retrieval and HyDE?

Rewriting turns a vague or conversational question into a search-friendly one. Multi-query generates several variants and merges the results to raise recall. HyDE writes a hypothetical answer and embeds that to bridge the gap between question and evidence. All three add latency, cost, duplication and generated assumptions, so the original query should usually stay in the mix.

22. How do Self-RAG, CRAG, RAPTOR and GraphRAG differ?

• Self-RAG: the model decides whether to retrieve and reflects on what it got.
• CRAG: grades retrieval quality and triggers corrections such as rewriting or switching sources.
• RAPTOR: clusters chunks into a hierarchy of summaries so retrieval can work at different levels of detail.
• GraphRAG: extracts entities and their relationships into a graph, aimed at questions that hop across documents or span the entire corpus.

Adopt them only when evaluation reveals a specific baseline failure. GraphRAG is warranted when relational or corpus-wide questions can't be answered reliably by hybrid retrieval plus query decomposition.

23. What is agentic RAG?

The model picks from a set of controlled tools for retrieval and reasoning: document search, reading an exact file, SQL, APIs or a knowledge graph. A production version needs typed, permissioned tools, retrieval and token budgets, step limits, stop and refusal rules, authorization at each tool boundary, complete trajectory logs, replayable tests and human approval for consequential actions. Evaluation has to score the final answer and the path of tool choices.

Evaluation and diagnosis

24. Which metric comes first?

Candidate Recall@k: the share of queries whose required evidence appears in the retrieved set, computed as relevant items found in the top k divided by all relevant items. If the evidence is missing, neither the reranker nor the generator can recover it. After that, assess ranking with MRR or nDCG, then context precision, correctness, faithfulness, citation quality, latency and cost.

25. MRR versus nDCG?

Mean reciprocal rank averages 1 over the rank of the first relevant result, so it suits cases where one good passage is enough. nDCG scores the whole ranking and supports graded relevance, which fits situations where several passages have different usefulness. Use MRR for first-hit performance and nDCG when order across several results matters.

26. How do you evaluate answer quality?

Evaluate retrieval and generation separately. For retrieval, track Recall@k, Precision@k, MRR or nDCG, filter correctness and version freshness. For generation, track correctness, faithfulness to the evidence, completeness, citation support, appropriate refusal and policy compliance. Add end-to-end latency, cost, user acceptance and results on high-risk slices.

27. What is oracle-context testing?

Skip retrieval and give the generator known-correct evidence. If it answers well with that evidence but fails in the full pipeline, suspect retrieval, filtering, reranking or context building. If it still fails, suspect the prompt, reasoning, generation or verification. It is one of the quickest ways to separate retrieval faults from generator faults.

28. How do you diagnose high faithfulness with low correctness?

Faithful means the answer matches the supplied context, and incorrect means that context was incomplete, irrelevant, outdated or insufficient. Compare, in order: gold evidence against retrieved candidates, candidates against reranked results, reranked results against the final context, the final generation against an oracle-context run, and finally source versions, filters and conflicting documents. Typical culprits are low recall, bad ranking, stale evidence, missing exceptions, aggressive context compression or a flawed reference answer.

Running it safely at scale

29. How do you keep a multi-tenant RAG system secure?

Derive tenant and user identity from trusted authentication and never from the prompt. Useful controls include ACL filters enforced by the datastore, tenant-scoped indexes or partitions, ACL-aware caches, least-privilege service roles, encryption and audit logs, cross-tenant isolation tests, prompt-injection detection, source allowlists with provenance checks, and verified deletion end to end. Treat retrieved content as untrusted, because documents can carry malicious instructions.

30. How would you design a large-scale platform?

Start by clarifying document volume, query rate, tenants, update frequency, deletion SLA, latency, citation needs, security and quality targets. Then split the design into planes:

• Ingestion: connectors, parsing, chunking, enrichment, embedding, indexing, reconciliation and deletion.
• Query: authentication, routing, hybrid retrieval, ACL filtering, fusion, reranking, context assembly, generation, verification and refusal.
• Evaluation: labeled datasets, offline experiments, regression tests, human review and release gates.
• Observability: traces of each request, what was retrieved, which model and index versions served it, plus latency, token use, spend, freshness, quality and security signals.
• Control: tenant onboarding, configuration, quotas, releases, model selection and policy.

For resilience, add deterministic IDs, versioned indexes behind aliases, migrations that run two indexes side by side, circuit breakers, bounded workloads, caching, canary releases, rollback and explicit SLOs. GraphRAG or agentic retrieval should wait until a measured baseline failure justifies it.