Mapping RAG: Pipeline Stages, Core Components and the Variant Landscape

A language model can reason impressively and still know nothing about your internal documentation, last week's policy change or the ticket filed an hour ago. Retrieval-augmented generation (RAG) addresses that gap by letting the model look things up in an external knowledge source before it answers. This guide walks through the problems RAG solves and the ones it does not, the three stages of a RAG system, the components you need to assemble one, and a map for making sense of the long list of named RAG variants.

RAG in one sentence

RAG combines two processes:

1. Retrieval: locating relevant information in an external knowledge source.
2. Generation: having a language model produce an answer grounded in what was retrieved.

Put simply, RAG hands the model a reference library to consult before it responds. The research that introduced the approach framed it as pairing the model's parametric memory, the knowledge stored in its weights during training, with a non-parametric memory, an external index that can be searched and updated independently.

An open-book exam is a helpful comparison:

• The LLM is the student.
• The knowledge base is the textbook.
• The retriever is the student flipping to the right pages.
• The final answer is the student's response, written with those pages open.

The model contributes reasoning and fluent language; the retrieval system contributes the facts that are relevant to this particular question.

The problems RAG is meant to solve

Relying on a model's built-in knowledge alone runs into several practical limits. For each one, RAG offers a partial remedy.

Knowledge that stops at training time

A model cannot know about anything that was added or changed after its training data was collected. With RAG, current information lives in an external store that you can update whenever you like, without touching the model.

Unsupported or invented answers

Models sometimes generate statements that are wrong or have no basis. Retrieved evidence gives the model something concrete to anchor on, which reduces unsupported output. It does not eliminate it: retrieval can return the wrong passages, the sources themselves can be inaccurate, and the model can still make claims the evidence does not support.

Private data the model has never seen

A general-purpose model has no access to your organization's internal documents, proprietary code or private knowledge. RAG lets an application draw on authorized private sources at query time, with no retraining on that data. Access control still matters here: the retriever should only return what the current user is allowed to read.

Information that changes often

Policies, product and technical documentation, and internal records change constantly. Adding or updating entries in the retrieval index is far cheaper and faster than retraining a language model every time something changes.

Documents too large to send whole

Pasting entire documents into every request wastes context and money. RAG sends only the relevant portions of the knowledge base, which keeps prompts focused and smaller.

Noise from irrelevant context

Too much loosely related material makes it harder for the model to find the evidence that matters. Filtering, reranking and context compression all exist to raise the quality of what finally reaches the model.

RAG improves grounding and lowers the rate of hallucination, but it offers no guarantee of accurate or hallucination-free answers. Treat it as a reliability tool, not a correctness proof.

The three stages of a RAG system

At the simplest level, every RAG system does three things: it prepares the knowledge, retrieves from it, and generates with it.

Stage 1: indexing

Indexing happens before any user asks a question. Source material such as PDFs, web pages, documentation or database rows is:

• loaded into the system,
• split into smaller chunks,
• turned into vector embeddings,
• stored in a vector database along with useful metadata.

The outcome is a searchable representation of your knowledge. Conceptually, you are building a search index over your own data.

Stage 2: retrieval

When a question arrives:

• the query is converted into an embedding with the same model used during indexing,
• the knowledge base is searched for nearby vectors,
• the most relevant chunks are selected,
• optional filtering or reranking refines the selection.

The flow from question to candidate chunks looks like this:

User Query
    │
    ▼
[Embedding Model]
    │
    ▼
[Query Vector]
    │
    ▼
[Vector Database]
    │
    ▼
[Relevant Chunks]

The important property is that query and documents live in the same vector space. That is why the embedding model used at query time has to match the one used to build the index; mixing models produces vectors that cannot be meaningfully compared.

Stage 3: generation

The selected chunks are combined with the user's question into a prompt, and the model generates the response:

User Query
    +
Retrieved Context
    │
    ▼
[Prompt / Context]
    │
    ▼
[LLM]
    │
    ▼
Final Answer

The resulting answer draws on the model's general abilities and on the specific evidence pulled from your knowledge source. How the prompt presents that evidence, for instance with source labels the model can cite, strongly influences how faithfully the answer sticks to it.

Inside the indexing pipeline

Indexing is where most of the quality of a RAG system is decided. Its basic shape:

Raw Data
   │
   ▼
Document Loader
   │
   ▼
Text Chunking
   │
   ▼
Embedding Model
   │
   ▼
Vector Database

Raw data

Knowledge can come from almost anywhere:

• PDFs
• Websites
• Documentation
• Word documents
• Databases
• APIs
• Internal company files

Document loaders

A document loader reads a source and converts it into a uniform structure the rest of the pipeline can process. In LangChain, for example, loaders share a common interface that produces Document objects, and they can expose both load() and lazy_load(). The lazy variant yields documents one at a time instead of materializing everything in memory, which matters for large collections. LangChain and LlamaIndex are the two ecosystems most people start with.

Chunking

Large documents are cut into smaller pieces called chunks, because retrieving the relevant section of a document is usually more useful than handing the model the whole thing. A good analogy is turning a book into a stack of index cards, each covering one idea.

Chunk size and overlap have no universally correct values. They depend on how your documents are structured, what kind of questions you expect, the embedding model you use and how much context you can afford downstream. Chunks that are too small lose surrounding meaning; chunks that are too large dilute the signal of the one sentence that mattered. For a deeper look at this trade-off, see chunking that preserves evidence.

Embedding models

Each chunk is converted into an embedding, a list of numbers that represents its meaning:

"RAG retrieves relevant information"
                ↓
[0.24, 0.87, 0.11, 0.62, ...]

Embeddings let a system measure how related two pieces of content are, which is what powers semantic search and retrieval: texts with similar meaning end up close together in vector space even if they share few words.

In OpenAI's text-embedding-3 family, the small model returns 1536-dimensional vectors by default, and the large one returns 3072.

Both accept a dimensions parameter to request shorter vectors, trading a little accuracy for less storage and faster search. Model lineups change, so confirm current options in the provider documentation.

Vector databases

The embeddings are stored in a vector database or vector index designed for fast similarity search. Common choices include:

• Qdrant
• Pinecone
• Chroma
• Weaviate
• FAISS

A stored record typically holds more than the vector. It also keeps the original chunk text and metadata such as source, page number, document ID or timestamp. That metadata is what later enables filtering by date or document type, and citing where an answer came from.

The building blocks of a RAG application

Pulling this together, a typical RAG application is assembled from six kinds of components:

• Knowledge source: supplies the information, for example PDFs, websites, SQL databases or APIs.
• Text splitter: breaks documents into manageable chunks, using LangChain's text splitters or your own logic.
• Embedding model: turns text into vectors, with options from providers such as OpenAI, BGE or Cohere.
• Vector database: stores and searches the embeddings, for example Qdrant, Pinecone, Chroma or Weaviate.
• LLM: writes the final response, such as GPT, Gemini, Claude, Llama or Mistral.
• Orchestration framework: wires the pieces together, for example LangChain, LangGraph or LlamaIndex.

A map of the RAG family

Once the basic pipeline is clear, the ecosystem of named variants can look overwhelming. There is no single canonical list of RAG types; papers and engineering blogs use overlapping names for architectures, retrieval strategies, context techniques, knowledge sources and specialized systems. Grouping them by what they change makes the landscape manageable.

Basic architectures

• Naive RAG: the straight pipeline of embed, search, and generate.
• Advanced RAG: adds steps before and after retrieval, such as query rewriting and reranking.
• Modular RAG: treats each stage as a swappable module that can be recombined.

Retrieval strategies

• Single-pass RAG: one retrieval per question.
• Multi-Query RAG: generates several phrasings of the question and retrieves for each.
• Iterative RAG: retrieves, reasons, and retrieves again.
• Multi-Hop RAG: chains retrievals when the answer requires connecting several facts.
• Hybrid RAG: combines semantic vector search with keyword search.
• RAG Fusion: merges results from multiple queries into one ranking.
• Self-Query RAG: lets the model turn the question into a search plus metadata filters.

Context and chunking approaches

• Parent-Child RAG: matches on small chunks but returns their larger parent section.
• Sentence Window RAG: retrieves a sentence plus its neighbors.
• Hierarchical RAG: searches through levels, from documents down to passages.
• Contextual RAG: enriches chunks with document-level context before embedding them.
• RAPTOR: builds a tree of recursive summaries and retrieves across its levels.

Self-correcting and adaptive approaches

• Self-RAG: the model critiques whether retrieval is needed and whether its output is supported.
• Corrective RAG (CRAG): evaluates retrieved results and triggers corrective retrieval when they are weak.
• Adaptive RAG: routes each question to a strategy that fits its complexity.
• Feedback-based RAG: uses user or evaluator feedback to improve retrieval over time.

Agentic approaches

• Agentic RAG
• Multi-Hop Agentic RAG
• Multi-Agent RAG

In these, an agent decides when and where to retrieve, possibly across several tools or cooperating agents, instead of following a fixed pipeline.

Variants defined by the knowledge source

Several names simply describe where the evidence comes from. Vector RAG searches an embedding index, while SQL RAG and Table RAG query relational or tabular data, often by having the model write the query. GraphRAG retrieves from a knowledge graph of entities and relationships, Web or Search RAG pulls live results from a search engine, and Knowledge-Base RAG draws on a curated corpus such as a help center or wiki.

Multimodal variants

Other labels refer to the kind of content being retrieved. Text RAG is the default case; Image, Video and Audio RAG retrieve from those media, typically through embeddings or transcripts, and Multimodal RAG is the umbrella term for systems that mix several of them.

Specialized variants

A final group names systems tuned for a particular requirement: a single field such as law or medicine (domain-specific), time-aware questions where recency matters (temporal), answers adapted to an individual user (personalized), queries and sources in more than one language (multilingual), sensitive data that must stay protected (privacy-preserving), and indexes spread across several sites or organizations (distributed or federated).

These groups overlap freely. A real production system might be hybrid, agentic and domain-specific all at once; the labels describe design choices, not mutually exclusive products.

From naive to advanced pipelines

The most basic pipeline is a straight line:

Query
 ↓
Embedding
 ↓
Vector Search
 ↓
Top-K Chunks
 ↓
LLM
 ↓
Answer

This is what people usually mean by naive or basic RAG: embed the question, take the top-k nearest chunks, and pass them to the model. It is a sensible first version and often good enough for small, clean corpora.

More mature systems insert extra stages around retrieval:

User Query
     ↓
Query Rewriting
     ↓
Query Routing
     ↓
Hybrid Retrieval
     ↓
Metadata Filtering
     ↓
Reranking
     ↓
Context Compression
     ↓
LLM
     ↓
Answer

Each stage targets a specific weakness. Query rewriting fixes vague or conversational questions, routing sends a question to the right index or tool, hybrid retrieval catches exact terms that pure vector search misses, metadata filtering removes out-of-scope documents, reranking reorders candidates with a more precise model, and compression trims what is left to fit the budget. Add these only when you can measure the problem they solve; complexity is not a goal in itself.

Why RAG is more than a vector database plus an LLM

A widespread beginner assumption is that RAG equals "store embeddings, query them, call the model". A production pipeline tends to look more like this:

Document Processing
       ↓
Chunking
       ↓
Metadata
       ↓
Embedding
       ↓
Vector / Keyword Retrieval
       ↓
Filtering
       ↓
Reranking
       ↓
Context Compression
       ↓
Prompt Construction
       ↓
LLM
       ↓
Validation
       ↓
Final Answer

Every step in that chain can degrade the final answer: bad parsing, careless chunking, missing metadata, a mismatched embedding model, weak filtering or a poorly constructed prompt. That is why wiring a vector store to a model is only the beginning. However capable the model, it cannot fully make up for being handed the wrong information.

How to judge a RAG system

Producing an answer is not the bar. A good RAG system retrieves genuinely useful information and generates a response that the retrieved evidence actually supports. The dimensions worth measuring:

• Retrieval quality: do the relevant documents or chunks actually come back?
• Precision and recall: is the useful material retrieved without dragging in unnecessary noise?
• Reranking: do the most relevant chunks end up at the top of the list?
• Context quality: is what reaches the model complete and coherent enough to answer from?
• Grounding: does the generated answer follow the evidence, or drift beyond it?
• Evaluation: is there a repeatable way to tell whether a change made the system better or worse?

Measuring each stage separately is what makes failures diagnosable; the approach of evaluating RAG by failure stage shows how to structure that.

Where to go from here

The next layer of depth is the indexing path itself: how a PDF becomes text, the difference between loading, parsing and cleaning, the trade-offs between character, token, recursive, semantic and structure-aware chunking, what chunk overlap really does, how metadata should be designed, how text becomes a vector, how cosine similarity, dot product and Euclidean distance differ, and how a vector database performs similarity search at scale.

Key takeaways

• RAG pairs a model's trained knowledge with a searchable external store, so answers can reflect private and current information without retraining.
• It reduces unsupported answers but never guarantees correctness; retrieval, sources and generation can each still fail.
• Every RAG system indexes, retrieves and generates, and most quality problems originate in indexing choices such as parsing, chunking and metadata.
• Use the same embedding model for indexing and querying, and keep chunk text and metadata alongside each vector.
• Named RAG variants are combinable design choices; add stages like rewriting or reranking only when you can measure the problem they fix.
• Evaluate retrieval and grounding separately from the final answer, or you will not know which stage to improve.