16 RAG Architectures Compared: Which Pattern Fits Which Problem

Chatbots that state invented facts, forget everything between messages or fail to locate the right document usually suffer from the same root issue: a retrieval pipeline that was designed too simply for the job. Sixteen RAG architectures exist, ranging from the simplest retrieval pipeline to privacy-preserving federated systems, and each solves a specific weakness. What follows covers what each one is, when it fits and what separates it from the others.

The three problems behind RAG

Large language models are impressive, yet three basic flaws make them unreliable in real applications.

First, hallucination. A model will assert false information with confidence, and not occasionally but regularly, because it has no sense of what it doesn't know. Second, a knowledge cutoff: every model is frozen at the end of its training and knows nothing that happened afterwards. Third, blindness to private data. Internal documents, product databases and customer contracts were never part of its training.

RAG addresses all three by handing the model a cheat sheet of real documents before it answers. Compare the two behaviors. Without retrieval:

Without RAG:
  User: "What's our refund policy?"
  LLM:  "I believe you offer a 14-day return..." (guessing)

And with it:

With RAG:
  User: "What's our refund policy?"
  → Step 1: Search internal docs → Finds: "Refunds within 30 days..."
  → Step 2: LLM reads the doc and answers accurately
  LLM:  "Your refund policy allows returns within 30 days..."

The concept is simple and the results are strong, though the system grows more complicated as use cases do. The four-step pipeline underneath every variant is index, retrieve, augment, generate.

Sixteen patterns in four tiers

Sorting the patterns by real-world impact gives four tiers. The first covers what roughly 80% of production systems need, and the later ones address specialized problems.

Tier 1: must know

1. Standard RAG, the foundation

Difficulty: beginner. Best when you're starting out. Every RAG project begins here, and the approach has four stages:

1. Index: split documents into chunks, turn them into embeddings and store them in a vector database.
2. Retrieve: embed the user's query and look for similar chunks.
3. Augment: put the retrieved chunks into the prompt.
4. Generate: the LLM answers from that real context.

The central loop fits in about ten lines:

# The core loop in ~10 lines
query_embedding = embed(user_query)
relevant_docs = vector_store.search(query_embedding, top_k=5)
context = "\n".join(relevant_docs)
prompt = f"Answer using this context:\n{context}\n\nQuestion: {user_query}"
answer = llm.generate(prompt)

It's uncomplicated, it handles most real use cases, and everything else builds on it. Reach for it when constructing a first question-answering bot over company documents, PDFs, support tickets or any static knowledge base.

2. Hybrid RAG, the biggest single quality gain

Difficulty: beginner. Best when retrieval quality matters. Semantic search through embeddings misses exact keyword matches, and keyword search with BM25 misses meaning, so hybrid retrieval runs both.

User: "React useState hook"
→ Semantic search finds: "state management in React components"
→ BM25 finds: docs with exact string "useState"
→ Hybrid (RRF fusion): gets the best of both

Reciprocal Rank Fusion (RRF) merges the two ranked lists without having to normalize their scores, which makes it elegant and effective. Many teams report accuracy gains of 10 to 20 percent from this switch alone. It suits any production system, particularly code documentation, technical manuals and legal documents, where exact terms and meaning both count.

3. Contextual Retrieval RAG, built for dialogue

Difficulty: intermediate. Pick it once follow-up questions show up. A common failure goes like this: someone asks about the Premium plan and then asks how much it costs. Standard RAG searches for something like "it cost" and finds nothing useful. This variant rewrites the query using the conversation history before searching.

User turn 1: "Tell me about the Premium plan"
User turn 2: "How much does it cost?"

Before search, rewrite → "How much does the Premium plan cost?"
Now retrieve → finds the pricing document ✓

That rewriting step, often called query contextualization, is essential in any multi-turn chatbot, since conversation falls apart the moment someone uses a pronoun. If users can ask follow-ups, it's required.

Tier 2: competitive edge

4. Agentic RAG, the hot trend of 2025 and 2026

Difficulty: intermediate. Best when your bot needs to do more than search. Standard RAG only fetches documents, while agentic RAG gives the model a set of tools and lets it choose among them.

User: "What was NVDA's stock return last quarter vs its historical average?"

Agentic RAG decides:
  → Use web_search tool for current stock data
  → Use calculator tool for return calculation
  → Use document_retrieval tool for historical reports
  → Synthesize all three into one answer

The agent follows the ReAct pattern, reasoning and acting: think about what to do, do it, observe the result and think again. That amounts to planning and retrieval in a loop. It fits financial assistants, research tools and developer tools, wherever several data sources or live information are involved.

5. Self-RAG, for when errors are unacceptable

Difficulty: advanced. Best when accuracy is mission-critical. Quality control is built into the pipeline itself, and the system doesn't blindly answer after retrieval. It passes through four gates. The retrieval gate asks whether retrieval is even needed for this query. The relevance gate asks whether the fetched document is actually relevant. The faithfulness gate asks whether the answer sticks to the retrieved facts, and the usefulness gate asks whether the answer helps. Failing any gate means the system retrieves again, revises or declines to answer. Medical systems, legal AI and financial advisors are natural homes, anywhere a hallucinated answer can cause real harm.

6. HyDE RAG, the vocabulary bridge

Difficulty: advanced. Best when users speak differently from your documents. HyDE stands for Hypothetical Document Embeddings. Imagine a user asking why things fall down while the documents talk about gravitational acceleration of 9.81 m/s². The two share almost no vocabulary, so an embedding search is likely to miss. HyDE's trick is to generate a fake answer first and search with that.

User: "Why do things fall down?"
→ LLM generates hypothesis: "Objects fall due to gravitational force..."
→ Search with the hypothesis (technical vocabulary)
→ Find the actual document about gravitational acceleration
→ Answer using the real document

No training is needed and it works out of the box with any LLM. It suits consumer-facing products where casual users query technical documentation.

Tier 3: high-impact specialists

7. Graph RAG, for relational reasoning

Difficulty: intermediate. Best when you need to answer how X relates to Y. Vector search finds similar documents but can't answer a question like which companies are linked to a given person through at least two board members, which demands graph traversal. Graph RAG builds a knowledge graph out of the documents, with entities as nodes and relationships as edges, and queries walk that graph.

Text: "Dr. Chen collaborated with Dr. Patel on a 2022 study funded by NIH."

Graph nodes: Dr. Chen, Dr. Patel, 2022 study, NIH
Graph edges: COLLABORATED_WITH, FUNDED_BYQuery: "Who funded Dr. Chen's work?" → traverse: Chen → study → NIH

Legal compliance, knowledge management, scientific literature and organizational networks benefit most.

8. Memory-Augmented RAG, a bot that remembers you

Difficulty: intermediate. Best when a chatbot must know its users across sessions. With a standard bot every conversation starts from zero. This variant adds three kinds of memory: conversation memory for the current chat, semantic memory for facts about the user such as a preference for technical explanations, and episodic memory for past sessions, such as a question about some topic two weeks earlier.

user_profile = memory.get_user_facts(user_id)
recent_context = memory.get_recent_conversations(user_id, k=3)
answer = rag_with_context(query, user_profile, recent_context)
memory.update(user_id, query, answer)

Tutoring applications, customer success bots and personal productivity assistants gain the most.

9. Modular RAG, swap any part without rewriting

Difficulty: intermediate. Best when you need to iterate quickly. This is an architectural philosophy in which every pipeline component can be exchanged independently. Moving from FAISS to ChromaDB takes one line, and A/B testing BM25 against dense retrieval needs no refactoring.

Standard RAG:   [fixed retriever] → [fixed reranker] → [fixed LLM]

Modular RAG:    [pluggable retriever] → [pluggable reranker] → [pluggable LLM]
                     ↑ swap anytime         ↑ swap anytime        ↑ swap anytime

Teams that experiment actively, research setups, and any project whose stack is expected to evolve are the audience.

10. Domain-Specific RAG, built for an industry

Difficulty: advanced. Pick it for medicine, law or finance. The abbreviation MI is a heart attack for a cardiologist, yet a chatbot lacking domain knowledge reads it as the state of Michigan. Generic RAG fails in specialized fields, and not because retrieval is wrong. The system just doesn't grasp the jargon, regulatory requirements or safety constraints. Three additions make the difference: a domain vocabulary with the field's own terms, abbreviations and synonyms; compliance rules such as attaching disclaimers and citing sources every time; and safety guardrails, for instance refusing to suggest specific dosages and pointing the user to a professional instead. Clinical decision support, legal research, financial analysis and compliance monitoring are typical uses.

11. Enhanced RAG, three sources and one answer

Difficulty: advanced. Best when structured and unstructured data are needed together. A single query fuses SQL or other structured databases for precise numbers, knowledge graphs for relationships, and document embeddings for narrative context.

Query: "Tell me about the Mars Exploration Program"

→ SQL: "Mars: 1.52 AU from sun, 687-day orbit"
→ Graph: Mars → explored_by → Curiosity Rover, Perseverance
→ Text: "Mars is the fourth planet... known as the Red Planet..."→ Fused Answer: Combines all three for comprehensive, expert-level response

Scientific research tools, financial analytics platforms and enterprise knowledge bases with mixed data types profit from it.

12. Recursive or multi-step RAG, for complex questions

Difficulty: advanced. Choose it when a single lookup can't cover the question. Some questions need a chain of knowledge. How the invention of the transistor led to modern AI needs at least five hops: transistor, integrated circuit, CPU, GPU, deep learning, LLMs. Recursive RAG retrieves, learns, and retrieves again.

Round 1: Retrieve about transistors → learn about miniaturization
Round 2: Retrieve about integrated circuits → learn about computing power
Round 3: Retrieve about GPUs → learn about parallel processing
Round 4: Retrieve about deep learning → learn about compute requirements
Round 5: Synthesize the full chain into a coherent answer

Research assistants, academic tools, strategic analysis and any multi-hop question call for it.

Tier 4: specialized use cases

13. ODQA RAG, answer anything

Difficulty: intermediate. Best when the knowledge base is huge and open-ended. Open-domain question answering is RAG at massive scale, covering science, history, geography and culture. The challenge isn't retrieval quality but handling the scope and recognizing when the answer is unknown. ODQA adds confidence scoring, so when the retrieved documents lack a good answer the system admits it isn't confident and doesn't hallucinate. General-purpose assistants, trivia bots, research platforms and cross-domain internal knowledge bases use it.

14. Multi-Modal RAG, text, images and audio together

Difficulty: advanced. Best when the knowledge base includes images, charts or audio. Most RAG systems handle only text, yet real knowledge also comes as anatomical diagrams, product photos, recordings and charts. A multi-modal system builds one embedding space shared across modalities, so a text query can find an image and an image query can find a document.

User (text): "Show me the anatomy of the heart"
→ Retrieves both: text descriptions AND anatomical diagrams
→ Response includes both text explanation and relevant image

Medical education, product catalogs, e-learning platforms and media archives are good fits.

15. Streaming RAG, real-time knowledge

Difficulty: intermediate. Best when information shifts every few minutes. A question put at 9 AM and again at 11 AM deserves different answers whenever the data underneath has changed. Streaming RAG keeps a continuously updated knowledge base, indexing new documents as they arrive.

Time 9:00 AM: "What's happening with NVDA?"
→ Retrieves this morning's earnings report

Time 11:30 AM (after news breaks): "What's happening with NVDA?"
→ Retrieves the breaking news from 11:15 AM

Financial news assistants, live support bots, event monitoring and operations dashboards rely on it.

16. Federated RAG, privacy first and multi-source

Difficulty: advanced. Best when data can't leave its source. It resolves a basic tension in enterprise AI: several data owners want a shared system but none may share raw data. Hospitals contributing to a medical assistant are the typical picture, because patient records can't be centralized. Each source runs its own local retriever, queries go to all sources at once, and only retrieved snippets, never raw data, reach the central system.

Central Query: "What's the survival rate for treatment X?"

→ Hospital A retriever: returns relevant anonymized snippets
→ Hospital B retriever: returns relevant anonymized snippets
→ Hospital C retriever: returns relevant anonymized snippets
→ Central LLM synthesizes — never saw raw patient recordsResult: HIPAA/GDPR compliant, collaborative AI

Healthcare networks, financial consortiums, legal databases and any multi-organization setup with data governance requirements are the target.

A month-by-month build order

All sixteen aren't needed on day one. Most production systems grow incrementally.

Month 1:  Standard RAG           ← Get something working
Month 2:  + Hybrid RAG            ← Biggest retrieval quality win
Month 3:  + Contextual Retrieval  ← Multi-turn conversations work now
Month 4:  + Self-RAG              ← Stop hallucinations
Month 5:  + Agentic capabilities  ← Tools beyond just search

This covers 80-90% of production RAG needs.

Combining patterns

Production systems nearly always mix several patterns. A few recipes:

• An enterprise chatbot pairs Standard, Hybrid, Contextual and Memory, for multi-turn conversation, good retrieval and personalization.
• A medical assistant pairs Domain-Specific, Self-RAG and Enhanced, because accuracy matters and structured data is involved.
• A research tool pairs Agentic, Hybrid and Recursive, for complex questions, multiple steps and tools.
• Customer support pairs Standard, Contextual and Streaming, for follow-ups and real-time updates.
• Legal research pairs Domain-Specific, Graph and Hybrid, for entity relationships and exact citations.
• A news analyzer pairs Streaming, ODQA and Multi-Modal, for real-time data, broad topics and images.

A quick chooser

For a first question-answering bot over documents, start with Standard RAG. For better search in production, add Hybrid. A multi-turn chatbot needs Contextual Retrieval, and a bot that calls tools or APIs needs the agentic variety. High-accuracy work in medicine or law points to Self-RAG, and casual users facing technical documentation point to HyDE. Relational questions call for Graph RAG, a personalized assistant for Memory-Augmented, experiments with different stacks for Modular, and regulated fields for Domain-Specific. Mixed structured and unstructured data suits Enhanced RAG, multi-hop questions suit Recursive RAG and general-purpose Q&A suits ODQA. Images and audio mean Multi-Modal, live data means Streaming, and multi-organization setups under HIPAA or GDPR mean Federated.

Running the examples

Each implementation is meant to be run directly: clone a repository, install the requirements, set an OpenAI API key and start with the first pattern.

# Clone the repo
git clone https://github.com/Manjunadh86/RAG-Materials.git
cd RAG-Materials

# Install dependencies
pip install -r requirements.txt# Set your OpenAI API key
export OPENAI_API_KEY="sk-your-key-here"# Run your first RAG system
cd 01-Standard-RAG
python hands_on.py

Each module contains a README that explains the concept, the architecture and when to use it, plus a runnable script to modify, with comments on every step.

RAG keeps moving fast. Agentic systems are becoming more capable, multi-modal retrieval is maturing, and federated approaches are turning critical as AI governance requirements tighten. The best way to learn RAG is to build it.