This article is published in English.
Production-Shaped HR Policy RAG with LangChain and LangGraph
Ingestion, MMR, history-aware rewrite, grounded answers, guardrails, eval, citations, and graph orchestration for an HR policy assistant.
A walkthrough covering ingestion, MMR retrieval, history-aware queries, safety checks, eval hooks, and multi-turn chat UX.
Introduction
Fluent generative answers are not enough for enterprise HR. A policy assistant must not invent leave rules from pretraining. It should retrieve approved documents and ground every claim in those sources. That is the job of Retrieval-Augmented Generation (RAG).
A modular, production-style HR Policy Q&A stack can combine Python, LangChain, LangGraph, an OpenAI-compatible chat model, ChromaDB, Streamlit, embeddings, MMR retrieval, history-aware query rewriting, prompt design, input moderation, prompt-injection checks, retrieval evaluation, conversational memory, summarization, and PDF export. The aim is not a demo chatbot — it is a RAG system that takes retrieval quality, conversation context, security, evaluation, and usability seriously.
1. The problem
When someone asks “How many sick-leave days are allowed?”, a plain LLM may invent an answer. An HR assistant instead should:
- Understand the question
- Search organizational HR documents
- Retrieve the most relevant sections
- Pass those sections to the model
- Generate an answer grounded in them
- Cite sources so the employee can verify
High-level flow: HR documents → ingestion → chunking + metadata → embeddings → ChromaDB → retriever → history-aware query → relevant docs → grounded prompt → answer → citations.
2. Document ingestion
Raw policies become searchable chunks. Full documents are too large to shove into every prompt and can blow context limits. Chunks carry metadata such as filename, path, folder, document type, and source labels — useful later for citations, debugging, and inspecting retrieval hits.
3. Embeddings and vector storage
Each chunk becomes an embedding vector:
"Employees receive annual leave..."
↓
Embedding Model
↓
[0.12, -0.43, 0.87, ...]
Vectors and metadata land in ChromaDB. User questions are embedded the same way so semantically close policies surface even when wording differs (“time off because of illness” vs “sick leave entitlement”).
4. Retrieval with MMR
Naive top-k similarity can return four near-duplicate leave-policy chunks:
Chunk 1 → Leave policy
Chunk 2 → Leave policy
Chunk 3 → Leave policy
Chunk 4 → Leave policy
Maximal Marginal Relevance balances relevance with diversity. A larger fetch_k candidate pool, then MMR selection of final k, reduces redundancy:
10,000 chunks
↓
Similarity search
↓
20 candidate chunks
↓
MMR
↓
4 diverse + relevant chunks
↓
LLM
5. History-aware retrieval
Follow-ups like “What about managers?” after an annual-leave answer are ambiguous alone. A history-aware step rewrites the query using conversation history before retrieval:
Conversation History
+
Current Question
↓
LLM
↓
Standalone Search Query
That separates query contextualization (what did the user mean?) from answer generation (what should the reply say given the docs?).
6. Grounded answer generation
Retrieved chunks feed a prompt that tells the model to use only HR documents, avoid inventing policy, surface exclusions, stay concise, and admit gaps. Architecture: question → contextualization → retriever → docs → QA prompt → LLM → grounded answer. The model writes prose; the corpus supplies facts.
7. Why LangGraph
As stages multiply — validation, moderation, injection detection, query processing, retrieval, generation, memory, summarization — a linear chain gets brittle. LangGraph models the workflow as states and nodes:
User Input
↓
Validation
↓
Guardrails
↙ ↘
Safe Unsafe
↓ ↓
RAG Workflow Reject
↓
Final Response
↓
Conversation
Management
The graph stays easier to extend than one mega-function.
8. Guardrails
Production input needs checks before RAG:
- Moderation — block policy-violating content early
- Prompt-injection detection — refuse “ignore previous instructions…” style attacks
Order matters: user input → security checks → RAG, not user input → raw LLM.
9. Conversational memory and summarization
Multi-turn assistants need history, but unbounded transcripts burn tokens. Summarizing older turns preserves signal while capping active context — a trade between preservation and cost.
10. Retrieval evaluation
An answer can look fluent while retrieval failed. Measure whether the right passages arrived, not only whether text came back. Evaluate retrieval quality, grounded generation, and application behavior as separate layers.
11. Source citations
Prefer “Employees receive 20 days of annual leave (Leave Policy §3)” over bare claims. Citations improve trust, debugging, and traceability when employees can open the original PDF.
12. PDF conversation export
Exporting a conversation to PDF lets employees keep a record for later. It is a usability detail that signals the system is meant for real work, not a throwaway chat widget.
Closing
A production-shaped HR RAG assistant is a pipeline of ingestion, retrieval strategy, conversational rewrite, grounding, graph orchestration, guardrails, evaluation, and citations. LangChain and LangGraph help assemble those stages; Chroma and MMR shape what the model is allowed to see. The non-negotiable product rule remains: policy answers come from approved documents, not from the model’s memory of the internet.
Design choices that matter in practice
Chunk size and overlap are not cosmetic. Too-large chunks dilute the embedding signal; too-small chunks lose surrounding policy constraints. Overlap helps when a rule spans a boundary. Metadata is not optional decoration — without filename and section hints, citations become vague and eval sets hard to judge.
MMR parameters (fetch_k, k, diversity lambda) should be tuned against a labeled question set, not gut feel. A candidate pool that is too small never surfaces diverse passages; one that is too large wastes latency.
History-aware rewriting must not invent facts. The rewrite step should only expand pronouns and incomplete follow-ups into standalone search queries. If the rewrite model drifts into answering, retrieval never sees a clean query.
Guardrails belong before retrieval. Moderation and injection detection that run after the model has already seen private policy text are late. Fail closed on injection suspicion; fail open only where product policy explicitly allows soft blocks with logging.
Evaluation should score both retrieval (recall@k of expected doc ids) and generation (faithfulness to retrieved text). A pretty answer from the wrong clause is still a failure. Store traces: rewritten query, retrieved ids, final answer, citation list.
Streamlit (or any thin UI) should expose citations and “unknown” states clearly. Employees trust systems that admit gaps more than systems that invent generous leave policies.
Export-to-PDF is a retention feature: people paste policy answers into tickets. Treat exported text as potentially sensitive and apply the same access controls as the chat session.
Finally, keep the graph’s happy path obvious. New engineers should be able to draw validation → rewrite → retrieve → generate → respond without spelunking through optional branches. Optional branches (summarization, export) hang off clear nodes rather than nesting inside generation.
That discipline turns a weekend RAG demo into something an HR team can pilot without fearing silent policy hallucinations.
Putting the pieces on a timeline
HR DOCUMENTS
│
▼
DOCUMENT INGESTION
│
Chunking + Metadata
│
▼
EMBEDDINGS
│
▼
CHROMADB
│
▼
RETRIEVER
(MMR Search)
│
│
USER ──→ GUARDRAILS ─────┤
│
▼
HISTORY-AWARE QUERY
CONTEXTUALIZATION
│
▼
RETRIEVAL
│
▼
RELEVANT DOCUMENTS
│
▼
QA PROMPT + LLM
│
▼
GROUNDED ANSWER
│
┌────┴────┐
↓ ↓
Citations Memory
│
▼
Summarization
│
▼
PDF Export
Day one often stops at “embed PDFs and chat.” Days two through ten are where production character appears: metadata schemas, MMR tuning, follow-up rewrite prompts, moderation hooks, injection classifiers, eval spreadsheets, citation formatting, and conversation summarization. Skipping those steps leaves a system that demos well on happy-path questions and fails on the second turn, on adversarial prompts, or on near-duplicate retrieval.
LangChain helps wire model and retriever objects; LangGraph helps make the control flow inspectable. Neither replaces a product decision about which HR folders are authoritative, who may ask which policies, and what “unknown” looks like in the UI. Those decisions belong in design docs next to the graph diagram.
When something goes wrong in production, the fastest debug path is usually: inspect the rewritten query, list retrieved chunk ids, read those chunks, then read the grounded prompt. If that trail is missing from logs, fix observability before adding another model.