Home / Articles

A Local RAG Pipeline with Qdrant, fastembed and Gemini in Python

A small RAG pipeline in Python: chunk local docs, embed them offline with fastembed, keep vectors in Qdrant, and answer questions with Gemini from the CLI.

Publisher: REACTAPP.TOP6 min read1485 words

This article is published in English.

Sources: How to Build a Simple RAG Pipeline using Qdrant

A custom RAG (retrieval-augmented generation) setup sounds more daunting than it is. The hype suggests an enterprise stack is needed just to let an LLM read local files. It isn't. A fully working, incremental pipeline can run on one machine with Qdrant and Gemini, with no heavy framework and no server to manage. Below is how it works, what the code does and how to run it.

The problem RAG solves

Language models only know their training data. They've never seen your notes, your reports or the PDF you saved this morning. RAG closes that gap. Instead of hoping the model guesses from general knowledge, the system searches your documents for the relevant pieces, hands those passages to the model and tells it to answer from that context alone. It stops inventing things because it's reading your material.

Plain keyword search could find those passages, but it breaks down quickly, which is why vector search is the usual choice.

A vector is a list of numbers that stands for the meaning of a piece of text. Turn "how are you" and "how old are you" into vectors and you can compute how similar they are. Vector search applies that idea to find answers even when the wording differs. The question and every document chunk get embedded into the same numeric space, and whatever lies closest in meaning wins. That matters because people seldom phrase a question the way the document does. Someone asking "how do I get my money back" still needs the section titled "Refund Policy".

A RAG pipeline is the plumbing that connects all this. It usually has six steps:

  1. Ingestion: a file lands in a folder, documents/ in this code.
  2. Chunking: the script cuts the file into smaller overlapping passages. Searching a 50-page document as one unit doesn't work, so splitting is required.
  3. Embedding: a small local model turns each chunk into a vector, and passages on similar topics end up near each other.
  4. Storage: the vectors go into Qdrant, a vector database built to take a new vector and quickly return the closest matches from millions. It's fast, open source and runs locally, with no server setup or account.
  5. Retrieval: a question is embedded too, and Qdrant returns the chunks nearest to it, the ones most likely to hold the answer.
  6. Generation: the retrieved chunks are placed in a prompt and sent to Gemini, which writes an answer grounded in the real documents.

Steps 1 to 4 run automatically through the -i flag or the -w watcher, so documents are always searchable. Steps 5 and 6 happen on demand when -q is used.

The Python implementation

The code stays lightweight and skips big frameworks, so it's easy to see where data goes.

Reading files

First, find files and pull out raw text, whether they're Markdown notes or untidy Word documents:

def read_document(path) -> str:
    # dispatches on extension: .txt/.md read directly,
    # .pdf via pypdf, .docx via docx2txt -> returns plain textdef file_status(path, state) -> str:
    # "not_indexed" if never seen, "modified" if file_hash(path)
    # differs from the hash stored in state, else "indexed"

Chunking

Next, the text is split with a simple sliding window, so a sentence spanning a cut-off keeps its context:

def chunk_text(text, chunk_size=500, overlap=50) -> list:
    words = text.split()
    step = chunk_size - overlap
    return [" ".join(words[i:i+chunk_size])
            for i in range(0, len(words), step)]

Embeddings and stable IDs

Embedding happens locally, with no external API call per sentence. Each chunk also gets a predictable ID derived from a hash of its text, so unchanged text keeps the same ID from run to run:

def chunk_ids_for(chunks) -> dict:
    # id = uuid5(namespace, chunk_text) for each chunk
    # -> same text always gets the same id, across re-indexes
    ...

The embedding itself happens inside the Qdrant call:

# inside index_file():
client.add(collection_name=index_name, documents=new_chunks, ids=new_ids)
# Qdrant's fastembed integration embeds `documents` automatically

Syncing files with the database

Here local files and the database are reconciled. A typo fix doesn't wipe the index and restart. The code compares IDs, deletes chunks that vanished and adds new ones:

def index_file(path, state) -> bool:
    chunks = chunk_text(read_document(path))
    new_ids = chunk_ids_for(chunks)
    old_ids = state[path.name].chunk_ids  # from last time
    client.delete(collection_name, ids=old_ids - new_ids)   # removed chunks
    client.add(collection_name, ids=new_ids - old_ids, ...)  # new/changed chunks
    state[path.name] = {...updated record...}

Files removed from the folder are cleaned up too:

def prune_deleted(state) -> int:
    # for each file in state no longer present in DOCUMENT_DIR:
    #     delete its Qdrant collection, drop its state entry

Answering a query

When a question arrives, Qdrant is asked for the best matches across the chosen documents. A helper first works out which indexes to search:

def resolve_index_names(spec, state) -> list:
    # "all" -> every indexed collection; "a,b" -> ["a","b"], validated

Then the query runs against each one and merges the results:

def cmd_query(index_spec, question, top_k, show_sources):
    for name in resolve_index_names(index_spec, state):
        hits += client.query(collection_name=name, query_text=question, limit=top_k)
    hits = sorted(hits, key=score, reverse=True)[:top_k]

Generation

Finally the retrieved fragments go straight to Gemini, which is held to the supplied text:

def generate_answer(question, context) -> str:
    prompt = f"Answer using only this context:\n{context}\n\nQuestion: {question}"
    return genai.Client(api_key=...).models.generate_content(
        model=GEMINI_MODEL, contents=prompt
    ).text

The rest is supporting code. get_client() opens the Qdrant connection, load_state() and save_state() read and write .rag_state.json, and sanitize_index_name() turns filenames into valid collection names. The command functions (cmd_list_files, cmd_index, cmd_watch) tie everything to the command-line flags.

Running it locally

Clone the repository and move into the project folder:

git clone https://github.com/Shivvam/rag-pipeline-qdrant.git
cd rag-cli

Then create an environment, install requirements and set up the config file:

python3 -m venv venv
source venv/bin/activate   # on Windows: venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env   # then edit in your GEMINI_API_KEY

Activate the virtual environment again in any new terminal before running app.py. GEMINI_API_KEY must be set for -q to produce answers, while the other commands work without it.

Command-line usage

Everything about ingestion and querying is driven from the terminal. To see which files are in the documents folder and which need attention, or which indexes exist:

python app.py -lf                        # list files in ./documents + status
python app.py -li                        # list existing indexes

To process new documents or update changed ones:

python app.py -i myfile.md               # index one file (or re-index if changed)
python app.py -i all                     # index every not_indexed/modified file

For automatic indexing while editing, keep this running in a second terminal:

python app.py -w                          # watch ./documents, auto-index on add/edit

To ask questions against one file, several files or everything:

python app.py -q report "What is X?"      # query a single index
python app.py -q report,notes "What is X?"  # query multiple indexes together
python app.py -q all "What is X?" --sources # query everything, show retrieved chunks
python app.py -q report "..." -k 8        # retrieve top 8 chunks instead of default 5

The status shown by -lf takes three values. not_indexed means the file has never been processed, indexed means it's current, and modified means it changed since the last run and -i should be repeated.

Design choices

A few decisions keep things quick and simple.

  • Qdrant: each file gets its own collection, named from the cleaned filename, with a numeric suffix added on a collision. By default it uses on-disk local mode (./qdrant_storage), so no server is needed. For concurrent access, set QDRANT_URL in .env to point at a real Qdrant server. Local mode locks the directory to one process, so -w and -q can't run simultaneously.
  • Embeddings: the BAAI/bge-small-en-v1.5 model runs through fastembed. It downloads once and is cached, so there are no API costs and no network dependency at query time.
  • Gemini: it only writes the final answer from the retrieved chunks, and defaults to gemini-3.5-flash.
  • State: .rag_state.json records file hashes, status, index names and chunk counts, which makes -lf and -li instant since Qdrant doesn't need rescanning. Chunk IDs come from the text itself, so re-indexing is incremental and only changed chunks are re-embedded and upserted.
  • File types: .txt and .md work immediately. .pdf and .docx need pypdf or docx2txt, both already listed in the requirements.