Choosing an Embedding Model for RAG: MTEB, BGE, Voyage and OpenAI Compared
Choosing an embedding model for RAG: reading MTEB properly, notes on BGE, Voyage, OpenAI and three more, a five-step Milvus pipeline and a decision framework.
This article is published in English.
Sources: Picking the Right Embedding Model for Your RAG Pipeline: What I’ve Learned the Hard Way
Three weeks is a long time to chase a RAG chatbot that gives eerily confident, completely wrong answers. Retrieval metrics looked fine on paper, the LLM was GPT-4 and the vector database was solid. The culprit turned out to be the embedding model, grabbed from the first tutorial that came up and never questioned again. It was badly mismatched for the domain. A lesson like that forces a proper look at what sits at the front of a RAG pipeline, so here is what matters when choosing an embedding model: the MTEB leaderboard, SBERT-style models, and the trade-offs between open-source and proprietary options.
Where embeddings sit in a RAG pipeline
RAG runs in three stages.
Ingestion. Every document chunk passes through an embedding model, which produces a vector embedding, a fixed-size numerical summary of the chunk's meaning. The vectors are stored in a vector database such as Milvus.
Retrieval. When a user asks something, the question is embedded with exactly the same model, and a nearest-neighbor search finds the most similar chunks.
Generation. The top-K chunks are injected into an LLM prompt as context, so the model answers from domain knowledge instead of relying only on its training data.
The critical constraint is that query and document embeddings share a vector space only when one model produced both. Swap the model on one side without reindexing the other and retrieval quality collapses. There is also research behind keeping top-K tight. The Lost in the Middle paper showed that answer quality drops when too many retrieved chunks crowd the context window, so retrieval should stay focused and relevant rather than broad and noisy.
SBERT: the design behind most embedding models
Most embedding models you will meet descend from SBERT, Sentence-BERT. Plain BERT was built to understand individual tokens in context. SBERT goes further by training the model to emit one fixed-size vector that captures the meaning of a whole sentence. The method is siamese training: two sentences are encoded separately, and the model learns to place semantically similar ones close together in vector space.
In practice, SBERT can tell that a sentence about a cat sitting on a mat differs in meaning from the same words with cat and mat swapped, a distinction basic BERT wouldn't reliably catch. For retrieval, where questions get matched to related passages, that sentence-level understanding is what makes the approach work.
LLMs such as GPT-4 grow from the decoder side of the transformer and are tuned for generation. Embedding models grow from the encoder side and are tuned for representation. They are different tools doing different jobs inside the same pipeline.
Using the MTEB leaderboard properly
The Hugging Face MTEB leaderboard is the industry standard for comparing embedding models, and it is often misused. MTEB scores models on 8 task types, including retrieval, clustering, classification and semantic textual similarity, across 58 datasets. For a RAG pipeline, one column matters: Retrieval Average, measured as NDCG@10, the normalized discounted cumulative gain at the tenth result. That metric weights top-ranked results most heavily, which matches RAG reality, where the first few chunks carry most of the weight.
A workable selection routine:
- Sort the leaderboard by the Retrieval column, highest first.
- Keep only models that fit the memory and latency budget.
- Choose the smallest model whose retrieval scores are acceptable.
- Test on a sample of real domain queries, since MTEB is known to reward overfitting in some models.
The final step is non-negotiable. Plenty of models rank high on MTEB and then struggle with technical or niche-domain writing. Treat the ranking as a place to begin, not a final answer.
Six production models and their quirks
The comparison covers three vendors:
- BAAI
bge-base-en-v1.5: 768 dimensions, 512-token context, open source, MTEB retrieval score 53. - BAAI
bge-base-zh-v1.5: 768 dimensions, 512-token context, open source, score 69 on Chinese benchmarks. - VoyageAI
voyage-2: 1024 dimensions, 4K tokens, proprietary. - VoyageAI
voyage-code-2: 1536 dimensions, 16K tokens, proprietary. - OpenAI
text-embedding-3-small: 512 to 1536 dimensions, 8K tokens, proprietary, score 62. - OpenAI
text-embedding-3-large: 256 to 3072 dimensions, 8K tokens, proprietary, score 65.
BGE base English and Chinese
These are the default picks for prototyping or whenever infrastructure cost should sit near zero. They are available on Hugging Face, run acceptably on CPU and carry no API fees. The 512-token window is the real limit. Most paragraph-level chunking stays under it, so it goes unnoticed, but long technical passages that resist clean splitting get truncated, and that quietly erodes retrieval quality. For Chinese-language content, bge-base-zh-v1.5 is the most practical option, and its score of 69 on Chinese benchmarks is genuinely strong.
VoyageAI voyage-2 and voyage-code-2
A published case study showed clearly better NDCG@10 on technical documentation than the ada-002 generation, and that is often what draws people to these models. voyage-2 was trained on dialog and conversational data, so it matches questions to passages better than general-purpose models in certain domains. In customer-support RAG systems it clearly beat bge on short, intent-heavy queries.
voyage-code-2 is the more interesting one. It is trained on code and offers a 16K context window, the longest in this group. For code search or documentation RAG, that means whole functions or long docstrings can be embedded without chunking at awkward boundaries. VoyageAI reports 14% higher recall on code retrieval, a figure that felt credible in testing. The downside is that both are proprietary and API-only, with no self-hosting and full dependence on the vendor's availability and pricing.
OpenAI text-embedding-3
These models replaced ada-002, and the gain is meaningful: higher MTEB retrieval scores, better multilingual performance and lower prices. The most interesting engineering choice is Matryoshka Representation Learning. Rather than training at one fixed dimension, the model learns representations at several scales at once, so the embedding can be truncated at query time with surprisingly little accuracy loss.
For text-embedding-3-large, cutting from 3072 dimensions to 256 lowers the MTEB retrieval score from 65 to 62. That is roughly a 5% accuracy drop in exchange for a 12x cut in storage and memory, a trade often worth making when tens of millions of vectors are stored. In RAG tests on Milvus technical documentation, text-embedding-3-small at 256 dimensions gave answers indistinguishable from the 1536-dimension version for most queries. The cases where higher dimensionality mattered were subtle disambiguation questions, perhaps 5% of real traffic.
End-to-end code: OpenAI embeddings with Milvus
The full pipeline below connects to Zilliz Cloud, which is managed Milvus, though the same code works against a self-hosted Milvus instance.
Step 1, connect:
from pymilvus import connections, utility
import os
from dotenv import load_dotenv
load_dotenv()
TOKEN = os.getenv("ZILLIZ_API_KEY")
CLUSTER_ENDPOINT = "https://in03-xxxx.api.gcp-us-west1.zillizcloud.com:443"
connections.connect(
alias='default',
uri=CLUSTER_ENDPOINT,
token=TOKEN,
)
Step 2, define the embedding model:
import openai
from openai import OpenAI
openai_client = OpenAI()
EMBEDDING_MODEL = "text-embedding-3-small"
EMBEDDING_DIM = 512 # Using reduced dim - good enough for most use cases
Step 3, create the collection:
from pymilvus import MilvusClient
COLLECTION_NAME = "my_rag_collection"
mc = MilvusClient(uri=CLUSTER_ENDPOINT, token=TOKEN)
mc.create_collection(
COLLECTION_NAME,
EMBEDDING_DIM,
consistency_level="Eventually",
auto_id=True,
overwrite=True
)
Step 4, chunk, embed and insert:
# Assumes `chunks` is a list of LangChain Document objects
chunk_list = []
for chunk in chunks:
response = openai_client.embeddings.create(
input=chunk.page_content,
model=EMBEDDING_MODEL,
dimensions=EMBEDDING_DIM
)
embeddings = response.data[0].embedding
chunk_list.append({
'vector': embeddings,
'chunk': chunk.page_content,
'source': chunk.metadata.get('source', ''),
})
mc.insert(COLLECTION_NAME, data=chunk_list, progress_bar=True)
mc.flush(COLLECTION_NAME)
Step 5, query and generate:
SAMPLE_QUESTION = "What do the parameters for HNSW mean?"
response = openai_client.embeddings.create(
input=SAMPLE_QUESTION,
model=EMBEDDING_MODEL,
dimensions=EMBEDDING_DIM
)
query_embeddings = response.data[0].embedding
results = mc.search(
COLLECTION_NAME,
data=[query_embeddings],
output_fields=["chunk", "source"],
limit=3,
consistency_level="Eventually"
)
context = [r['entity']['chunk'] for r in results[0]]
contexts_combined = ' '.join(context)
llm_response = openai_client.chat.completions.create(
messages=[
{"role": "system", "content": f"Answer using only the context below. Context: {contexts_combined}"},
{"role": "user", "content": SAMPLE_QUESTION}
],
model="gpt-3.5-turbo",
temperature=0.1,
)
print(llm_response.choices[0].message.content)
A complete working notebook lives in the Milvus bootcamp repository on GitHub.
A decision framework
After many experiments, the choice usually falls out like this:
- Prototyping, cost-sensitive or self-hosted work: begin with
bge-base-en-v1.5. It is free, fast and good enough to validate the pipeline architecture. - Production English or multilingual chatbots:
text-embedding-3-smallat reduced dimensionality is hard to beat on cost versus quality. - High-stakes domain RAG such as legal, medical or technical documents: evaluate
voyage-2seriously, because MTEB doesn't capture everything and domain retrieval quality can surprise. - Code search and developer tooling:
voyage-code-2and its 16K window justify the API dependency. - Severe memory limits or billions of vectors:
text-embedding-3-largeat 256 dimensions, where the Matryoshka approach is clever engineering.
Whatever the choice, test on a slice of real production queries. MTEB is a proxy, and retrieval quality on actual data is the only number that counts.