Meaning as Coordinates: From Embeddings to Vector Databases and RAG

A search box that returns nothing for "a good pet for an active person" even though the knowledge base is full of dog advice shows exactly where keyword matching falls short. Fixing it means treating meaning as a position in space. That idea underlies semantic search, vector databases and retrieval-augmented generation, and one tiny example is enough to follow all of it.

The example used throughout

Picture a miniature knowledge base of four sentences:

1. Dogs are loyal and need daily walks and exercise
2. Cats are independent pets that groom themselves
3. Golden retrievers are friendly family dogs
4. The capital of France is Paris

A user asks:

"What's a good pet for an active person who exercises a lot?"

The question never contains the word dog or walks. A plain keyword search would find nothing useful. Yet a machine can still return the right answer, because it compares where texts sit in a meaning space and not which words they share.

Similar meaning, nearby points

A vector is simply a row of numbers that encodes what a piece of text means. Once similar meanings produce similar vectors, finding related text becomes finding nearby points. That turns a language problem into a geometry problem, which is the base of modern search, of recommendation, and of RAG, the technique of giving a language model an outside library to consult so that it stops inventing things.

When meaning is a set of coordinates, working with meaning means working with distance and direction, something computers do extremely well.

Embeddings for whole passages

Word-level vectors are not enough for searching documents. A single vector is needed for a sentence, a paragraph or a page. Transformer models can produce one: feed in a chunk of text and receive one vector, usually called an embedding, that sums up its meaning.

"Dogs need daily walks"  ──►  [ 0.91,  0.08, -0.30,  ... ]   (e.g. 384 numbers)
 "The capital is Paris"   ──►  [-0.05,  0.62,  0.44,  ... ]

A sentence embedding compresses a passage into a fixed-length list of numbers. Two texts on the same topic yield similar lists, and unrelated texts yield very different ones. The individual numbers mean nothing to a person. What counts is how they relate to each other, and nobody reads them directly. They are only compared.

The picture to keep in mind

Imagine every text as a dot in space. Texts with similar meaning sit close together and unrelated ones sit far apart.

(a simplified 2-D "meaning space")

   dog walks •   • golden retriever          ← the "dogs" neighborhood
                \  /
                 (pets)
   cat grooming •                                    • Paris   ← far away, different topic

In that simplified map, the two dog sentences cluster together, the cat sentence sits near them because it is still about a pet, and the Paris sentence lies in a different region altogether. Real embedding spaces have hundreds of dimensions, but the intuition is identical: related meaning means nearby points. Everything that follows is a way of finding the nearest dots.

Keyword search against semantic search

Keyword search matches words. Searching for exercise will skip a document that says daily walks, since no word is shared. It is fast but literal, and it misses meaning.

Semantic search matches meaning. The query is turned into a vector, and the document vectors closest to it are returned, whatever words each one happens to use. Here is the ranking produced for the example, using actual similarity scores:

Query: "a good pet for an active person who exercises"

 0.999  #############################  Golden retrievers are friendly family dogs
 0.995  #############################  Dogs are loyal and need daily walks and exercise
 0.398  ###########                    Cats are independent pets that groom themselves
 0.059  #                              The capital of France is Paris

The query shares no keywords with the dog sentences, yet both score highest at roughly 1.0. The cat sentence lands in the middle because it is still a pet, and the Paris sentence is near zero. Keyword search would have returned nothing. That is why semantic search feels intelligent: it recognizes that an active person who exercises and a dog needing daily walks are about the same thing. The gap between keyword and semantic search is the gap between no results found and the answer that was actually wanted.

Measuring closeness with cosine similarity

The usual tool for measuring how close two vectors are is cosine similarity. It looks at the angle between two arrows and ignores their length.

• Arrows pointing the same way give a similarity near 1, meaning very similar meaning.
• Arrows at right angles give a similarity near 0, meaning unrelated.
• Arrows pointing opposite ways give a similarity near -1, meaning opposite meaning.

A small angle between the query and a dog document looks like this:

^  query
        │ /
        │/  small angle  →  cosine ≈ 1  (similar)
        └──────► dog doc

And a wide angle to the Paris document:

        ^ query
        │
        └──────► Paris doc      big angle  →  cosine ≈ 0  (unrelated)

Angle is used instead of length so that long and short texts compare fairly. The math fits in a line: multiply the vectors together and normalize.

import numpy as np

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

That small function is behind every score in the ranking above. Scores run from -1 to 1, and a value near 1 says that the two texts mean the same thing.

Vector databases for search at scale

The example has four documents. Real systems hold millions or billions, such as every support ticket, product page or paragraph of company documentation. Comparing a query against a billion vectors one by one on every search is far too slow.

A vector database exists for this. It is a store built to do one job well: take a query vector and quickly return the closest stored vectors. It relies on approximate nearest neighbor search, a family of clever indexes that find nearly the closest matches while skipping most comparisons. A sliver of accuracy is traded for a huge gain in speed.

documents ─► embed ─► [ vector database ]
                              │  stores millions of vectors
                              │  + a smart index (ANN)
   query ─► embed ─────────►  │
                              ▼
                     top-k nearest matches (in milliseconds)

The documents go through the embedding model a single time and are saved alongside an index. When a question arrives, it gets embedded too, and the database replies with the k best-matching items in a blink, however many vectors it holds. Products in this space include FAISS, Pinecone, Weaviate, Chroma and Milvus, plus pgvector, an extension that brings vector search into ordinary PostgreSQL. Details differ, but all of them store embeddings and find neighbors fast. In short, a vector database is a search engine for meaning.

What plain language models get wrong

Large language models impress, but on their own they have three stubborn weaknesses:

• Hallucination. When unsure, they produce fluent, confident and wrong answers.
• A knowledge cutoff. They know only what was in the training data, frozen at some earlier date.
• No knowledge of your data. Internal documents, private notes and last week's meeting are all unknown to the model.

You: "What's our refund policy?"
 Plain LLM: "Typically, refunds are offered within 30 days..."   ← a guess, not YOUR policy

Asked about a refund policy, a plain model answers from memory with a plausible guess, not the company's actual policy. For anything factual or private that is risky. The need is to supply the right facts exactly when the model answers, with no retraining involved, and RAG does exactly that.

Retrieval-augmented generation

RAG combines everything above into one loop. Instead of asking the model to answer from memory, the relevant documents are retrieved first through semantic search, and then the model is asked to answer based on them. The name is a description: retrieval finds relevant text, augmented means adding it to the prompt, and generation lets the model write the answer.

1. Question ─► embed ─► search the vector DB ─► top relevant chunks
                                                        │
 2. Build a prompt:  "Using ONLY the context below, answer the question.
                      Context: [retrieved chunks]
                      Question: [user's question]"
                                                        │
 3. LLM generates an answer grounded in the retrieved text ─► with sources

Running the example through it:

Question: "What's a good pet for an active person?"
   → retrieves docs 1 & 3 (the dog sentences, from Figure 3)
   → prompt: "Using this context: [Dogs need daily walks and exercise;
              Golden retrievers are friendly family dogs] — answer the question."
   → LLM: "A dog — especially a golden retriever — is a great fit, since
           they need daily walks and exercise. (Source: docs 1, 3)"

The answer is now grounded. It draws on real documents that the owner controls, and it can name where a fact came from. Far fewer invented answers slip through. RAG is semantic search plus a language model: retrieve the right facts, then let the model phrase the reply.

The practical pipeline and its main trap

RAG has two phases. Setup happens once, and querying happens each time someone asks.

── SETUP (once) ──                    ── QUERY (every question) ──
 documents                             user question
    │ split into chunks                    │ embed
    │ embed each chunk                     │ search vector DB
    │ store in vector DB                   │ get top-k chunks
    ▼                                      │ build prompt with chunks
 [ vector database ]  ◄───── search ──────┤ send to LLM
                                           ▼
                                     grounded answer

Documents are chunked, embedded and stored once. Every question is embedded, matched against the store, and the best chunks go to the model to write the final answer.

The trap is chunking, meaning how documents are split before embedding. Chunks that are too large blur several topics into one vector and give fuzzy matches. Chunks that are too small lose context, and an answer may get cut in half. Picking the chunk size is among the biggest practical decisions in a working RAG system, and a reminder that unglamorous data preparation shapes everything downstream.

The same thing in code

The retrieval half of RAG, a bare-bones semantic search, takes only a few lines of NumPy. It receives vectors for some documents plus one for the query and sorts the documents by cosine similarity:

import numpy as np

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))def search(query_vec, doc_vecs, docs, k=2):
    scores = [(cosine(query_vec, v), d) for v, d in zip(doc_vecs, docs)]
    return sorted(scores, reverse=True)[:k]      # top-k most similar

Fed the four pet and Paris vectors and the active-person query, it returns the two dog documents first, matching the ranking shown earlier. That is semantic search in a handful of lines.

A real project does not hand-build vectors. It calls an embedding model and a vector store, and the code looks roughly like this:

# Illustrative real-world usage (libraries download a model, so not run here)
from sentence_transformers import SentenceTransformer
import chromadb

model = SentenceTransformer("all-MiniLM-L6-v2")     # turns text into vectors
db = chromadb.Client().create_collection("pets")db.add(documents=my_docs, embeddings=model.encode(my_docs).tolist(),
       ids=[str(i) for i in range(len(my_docs))])q = "a good pet for an active person"
hits = db.query(query_embeddings=model.encode([q]).tolist(), n_results=2)
# hits now holds the top-2 most semantically similar documents

The libraries do the heavy lifting, but underneath they perform the same cosine-similarity search. Semantic search is a few lines of math on vectors, and real tools wrap that idea with speed and scale.

The whole arc

Text starts as characters and tokens, then becomes embeddings, then gains context through attention inside a transformer, and finally turns into a searchable map of knowledge that makes language models accurate and useful.

raw text
   → characters / tokens / embeddings        [article 1: From Characters to Meaning]
   → attention turns them into context-aware
     vectors inside a transformer            [article 2: Attention, Explained Simply]
   → those vectors become coordinates we can
     store, search, and retrieve             [article 3: this one]
       → semantic search  (find by meaning)
       → vector databases (search at scale)
       → RAG              (ground an LLM in real data)

The thread through all of it is a single idea growing up: meaning is a point in space. First it applies to words, then to sentences in context, and finally to entire libraries of documents that can be searched and reasoned over.

One more distinction is worth keeping in mind. RAG gives a model access to knowledge without changing the model itself, like handing it notes to read at answer time. Fine-tuning is a different route, where the model's internal weights are adjusted by training on examples. RAG supplies knowledge to read, and fine-tuning changes what the model is. Knowing which to reach for is a practical skill in its own right.