Why Production RAG Is Moving Beyond Dedicated Vector Databases
Explores why standalone vector databases are losing ground in production RAG systems and how hybrid retrieval, filtering, and unified data layers are reshaping the stack.
This article is published in English.
Sources: The Vector Database Era Is Ending. Here’s What Production RAG Is Moving Toward
Retrieval pipelines used in production are rarely a single mechanism. A modern setup can blend:
- Dense retrieval to catch semantic similarity
- Sparse retrieval for matching exact terms
- BM25 for lexical scoring
- Hybrid search that widens recall
- Reranking to sharpen precision
- Query rewriting for hard or ambiguous questions
- Self-checking and validation of retrieved results
- Graph-based retrieval for interconnected knowledge
- Agentic workflows that pull information in multiple steps
This piece looks at why the classic vector-database-first design for Retrieval-Augmented Generation is starting to fall short in many real-world deployments.
If RAG is a new topic for you, it's worth reviewing the basics before diving into how modern systems are evolving beyond them — concepts like what RAG is, why LLMs hallucinate, how a RAG pipeline is typically built, and the difference between dense and sparse retrieval are good starting points.
For quite a while now, the go-to architecture for Retrieval-Augmented Generation has felt almost obligatory:
LLM → Embedding Model → Vector Database → Retrieved Context → LLM
If you were architecting a serious RAG product, the expectation was that you'd pick a dedicated vector store — Pinecone, Weaviate, Qdrant, Milvus, Chroma, or similar.
But real-world production use is reframing the question.
It's no longer really about:
"Which vector database should I pick?"
The more relevant question has become:
"Do I actually need a dedicated vector database?"
That shift in framing matters a lot.
Recent field reports and benchmark comparisons show a growing number of teams folding vector search into infrastructure they're already running — PostgreSQL with the pgvector extension, Elasticsearch, and various unified data platforms are common choices.
This isn't a signal that vector databases are about to vanish overnight.
Rather, it suggests that treating a standalone vector database as the default architectural choice is becoming harder to justify.
And this pressure is showing up exactly where it counts most in AI engineering:
production costs, operational overhead, filtering capability, data consistency, and overall retrieval quality.
1. The Vector Database Solved a Genuine Problem
It helps to first revisit why vector databases caught on in the first place.
Conventional databases handle queries like this with ease:
SELECT *
FROM documents
WHERE category = 'insurance'
AND effective_date > '2025-01-01';
But semantic search poses a fundamentally different kind of question:
"Find documents whose meaning is close to this query."
Answering that requires turning text into numerical vectors:
"How do I file a health insurance claim?"
↓
Embedding Model
↓
[0.021, -0.183, 0.492, ...]
From there, the database runs an approximate nearest-neighbor search.
At a conceptual level, this looks like:
Query Vector
│
▼
┌─────────┐
│ ANN │
│ Index │
└────┬────┘
│
├── Document A 0.91
├── Document B 0.87
├── Document C 0.84
└── Document D 0.81
Once you're dealing with millions or billions of vectors, specialized indexing methods like HNSW or IVF can speed this process up enormously compared to a brute-force scan.
So the original motivation made complete sense:
Design a database purpose-built for retrieving vectors efficiently.
For specific kinds of workloads, that reasoning still holds up today.
The complications arose from what came afterward.
2. Production RAG Involves Far More Than Vector Lookup
Early prototypes of RAG systems tend to follow a simple flow:
Question
↓
Embedding
↓
Vector Search
↓
Top 5 chunks
↓
LLM
Real production systems almost never stay that simple.
What you typically end up with instead looks more like:
┌── Metadata filters
│
├── Tenant isolation
│
Query → Rewrite → Embed ─┼── Keyword search
│
├── Vector search
│
├── ACL filtering
│
├── Recency
│
└── Reranking
│
▼
LLM
At this point, the database can't just answer:
"Which vectors are nearest to this one?"
It has to answer something far more layered:
"Which documents are semantically related, belong to the right customer, are still valid, are accessible to this particular user, and satisfy these additional filters — all at once?"
That requirement reshapes the whole problem.
3. The Hidden Cost of Running Two Databases
Imagine your company already relies on PostgreSQL.
Your core application data sits there:
PostgreSQL
users
customers
policies
claims
documents
permissions
metadata
Then a vector database gets added to the stack:
PostgreSQL Vector DB
────────── ──────────
Documents ───────► Embeddings
Users Metadata
Permissions ANN index
Claims
From this point on, updating a document may require pushing changes into two separate systems.
A typical update flow ends up looking like:
Document Updated
│
├──────────────► PostgreSQL
│
└──────────────► Embedding Pipeline
│
▼
Vector DB
This setup opens the door to synchronization headaches.
Consider what needs to happen when:
- a document gets deleted
- a customer's permissions change
- a policy gets a new version
- metadata is updated
- a tenant is removed from the system
- the embedding model itself is swapped out
At that point, you've effectively built distributed data consistency problems into your retrieval layer.
A 2026 research paper examining unified data layers for RAG points to exactly this kind of split — between relational and vector storage — as the root cause of stale data, tenant-isolation failures, and overly complex query composition. Their PostgreSQL-plus-pgvector design showed meaningful gains on filtered retrieval tasks in their evaluation and did away with the synchronization issues that come from maintaining two separate systems.
That turns out to be a far more consequential production issue than debating whether one ANN indexing method edges out another by a small margin in raw speed.
4. "Good Enough" Vector Search Starts Showing Up Inside Databases You Already Run
This is where PostgreSQL enters the picture.
Thanks to pgvector, a PostgreSQL table can hold both your regular columns and embeddings side by side:
CREATE TABLE documents (
id BIGSERIAL PRIMARY KEY,
content TEXT,
metadata JSONB,
embedding VECTOR(1536)
);
And you can run a similarity query directly against that same table:
SELECT
id,
content,
embedding <=> query_embedding AS distance
FROM documents
WHERE tenant_id = 123
ORDER BY embedding <=> query_embedding
LIMIT 10;
What matters here isn't simply that PostgreSQL is capable of vector search.
It's that this vector search runs right alongside the rest of your application's data.
You get to combine, in a single query, in a single system:
SQL filtering
+
Vector similarity
+
Transactions
+
Access control
+
Metadata
For teams running production systems, that's a compelling proposition.
Several production comparisons published in 2026 now treat pgvector as the go-to option for many RAG use cases whenever PostgreSQL is already part of the stack — especially once vector counts are moderate rather than massive.
5. The Cost Case Is More Significant Than It Appears
Cost is arguably the strongest reason teams revisit this architecture.
Suppose your stack already includes something like:
PostgreSQL
$300/month
Adding vector capability to that setup can be as simple as:
Existing PostgreSQL
+
pgvector
+
additional CPU/RAM
Compare that to running a separate managed vector service alongside PostgreSQL:
PostgreSQL
+
Pinecone
+
network traffic
+
index management
+
monitoring
+
data synchronization
The gap won't be dramatic for every workload.
But architectural overhead adds up over time.
A number of 2026 cost studies point to a real price gap between managed vector services and PostgreSQL-based vector search under moderate load. One benchmark found Pinecone's per-query cost noticeably higher, while pgvector needed extra tuning work to hit similar throughput.
The takeaway isn't:
"pgvector is always the cheaper option."
That claim doesn't hold universally.
The real takeaway is:
The cheapest vector database is frequently the one you're already paying for.
6. Cost Isn't What Actually Breaks This Model
Here's the less comfortable reality.
If the only issue with vector databases were price, the fix would be simple: cut costs.
But teams running these systems in production are running into a different, deeper issue.
Similarity search alone often isn't the right retrieval approach.
Take a question along these lines: a customer wants to know what cancellation terms apply to a premium subscription tier in a specific state, starting from a particular date early next year.
This kind of question actually carries several distinct retrieval signals at once:
Semantic meaning
+
Exact terms
+
Customer segment
+
Geography
+
Effective date
Vector search alone might grasp the general meaning of the sentence while completely missing the precise constraints embedded in it.
That's why current RAG systems are increasingly built to blend:
BM25
+
Dense Retrieval
+
Metadata Filtering
+
Reranking
instead of depending purely on semantic similarity.
A recent study examining retrieval quality as corpus size scales — comparing lexical, dense, graph-based, and agentic approaches — found that BM25 held up remarkably well, staying on the low-cost Pareto frontier throughout the tested range of corpus sizes.
That result alone should give RAG engineers pause.
Years of effort went into refining semantic retrieval.
Yet keyword-based search was never actually obsolete.
7. Elasticsearch Has Emerged as a Legitimate RAG Backend
This trend further undercuts the case for standalone vector databases.
Search engines have long handled:
- inverted indexes
- BM25 scoring
- filtering
- aggregations
- faceted search
- text relevance ranking
- distributed search
Now they handle vector search too.
So rather than architectures built around:
PostgreSQL
+
Vector DB
+
Search Engine
teams can consolidate toward something like:
Search Engine
┌───────────────┐
│ BM25 │
│ Vector Search │
│ Filters │
│ Ranking │
└───────────────┘
A 2026 community benchmark pitting pgvector against Elasticsearch, Qdrant, Pinecone, and Weaviate went beyond raw ANN latency, testing filtered search, hybrid search, write/update behavior, and concurrent access under realistic production-style conditions.
That's the category of benchmark that actually matters in practice.
Nobody gets an emergency page because their cosine similarity numbers were 12% slower than a competitor's.
They get paged because:
a customer was shown the wrong document.
8. Filtered Retrieval Is Where the Real Competition Is Happening
This might be the single most important architectural insight in current RAG design.
Picture a dataset of 100 million vectors.
A given user is only permitted to see results matching:
tenant = ACME
region = US
department = legal
document_status = active
A naive pipeline might run something like:
Vector Search
↓
Top 100
↓
Metadata Filtering
↓
Top 10
But by the time filters get applied afterward, you may have already discarded the documents that actually mattered, because they never made it into the initial shortlist of candidates pulled by the vector search step.
Modern production retrieval increasingly needs those filtering constraints to shape the retrieval process itself, not just clean it up afterward.
In other words, the query you actually want looks more like:
Semantic Similarity
+
Structured Filtering
+
Authorization
+
Keyword Matching
+
Ranking
At its core, this is a database query optimization challenge.
It isn't simply a vector search challenge.
9. Vector Databases Are Facing Their Own “Unified Data Layer” Pressure
Database engineering has spent decades merging different categories of data into single systems. AI is now pushing that same consolidation instinct even further.
The architecture taking shape looks roughly like this:
Unified Data Layer
│
┌───────────────────┼───────────────────┐
│ │ │
Relational Vector Search
│ │ │
SQL / joins ANN/HNSW BM25
│ │ │
└───────────────────┼───────────────────┘
│
Graph
│
Relationships
And potentially extends into something like:
RAG Query
│
▼
Query Planning Layer
│
┌────────────┼────────────┐
▼ ▼ ▼
Vector BM25 SQL
Search Search Filters
│ │ │
└────────────┼────────────┘
▼
Reranker
│
▼
LLM
This mirrors how retrieval actually functions inside real enterprise systems far more closely than a standalone vector store does.
Current research in database systems is actively investigating engines that execute vector similarity search, graph traversal, and relational filtering as one coordinated operation rather than as separate, sequential retrieval steps. That direction represents a significant shift in how retrieval infrastructure gets designed.
10. Are Pinecone, Qdrant, and Milvus Finished?
Not at all.
This is the part of the argument that needs nuance.
Purpose-built vector databases remain the right choice in several situations.
When your vector collection is massive. If you're operating at:
500M+
1B+
10B+ vectors
and vector retrieval is a core piece of your infrastructure, dedicated systems justify their complexity.
When you need very high query throughput. If serving vector queries fast, at volumes reaching:
Millions of queries
per second
is essentially your product, routing everything through a general-purpose transactional database isn't practical.
When you need advanced ANN tuning. Specialized platforms give you fine control over HNSW, IVF, quantization, sharding, replication, distributed approximate nearest neighbor search, and GPU acceleration.
When you want infrastructure isolation. Keeping retrieval workloads separate from your core transactional systems can be a deliberate, sensible choice, so your AI features don't become the source of database contention.
11. What's Actually Fading Away
It isn't vector search itself. It isn't embeddings, ANN indexing, or RAG as a pattern.
What's losing ground is a specific assumption:
"Every production RAG system requires its own dedicated vector database."
That premise is becoming harder to justify by default. For a large share of applications, the setup can simply look like this:
PostgreSQL
│
├── Relational Data
├── Metadata
├── Permissions
├── Full-text Search
└── pgvector
│
▼
RAG
rather than this:
PostgreSQL
│
├──── Vector DB
│
├──── Search Engine
│
└──── Cache
Each additional system you introduce adds another operational seam, and every seam is a potential point of failure.
12. A More Useful Way to Decide
Rather than asking which vector database is the fastest, ask questions tied to your actual scale.
Below 1 million vectors: keep it simple. If PostgreSQL is already part of your stack, pgvector deserves to be your first evaluation. There's no need to bring in another database purely because a tutorial did.
Between 1 million and 50 million vectors: this is where real benchmarking matters. Compare options such as:
pgvector
vs
Qdrant
vs
Weaviate
vs
Pinecone
vs
Elasticsearch
But evaluate p95/p99 latency, filtered recall quality, hybrid retrieval performance, ingestion throughput, update and delete behavior, memory footprint, operating cost, and failure recovery — not just raw top-k ANN speed.
Between 50 million and 500 million-plus: dedicated systems start making strong sense, including:
Qdrant
Milvus
Pinecone
Weaviate
Elasticsearch
specialized vector infrastructure
Your choice now hinges on QPS targets, vector dimensionality, index type, replication needs, latency budgets, memory constraints, and filtering complexity.
Above 500 million vectors: stop relying on generic "best vector database" rankings and build a benchmark specific to your workload. At this scale, small architectural missteps can translate into infrastructure costs measured in six figures.
13. What the RAG Stack Looks Like Now
The typical 2023 RAG stack looked like this:
LLM
│
Embedding Model
│
Vector Database
│
Retriever
│
LLM
The production stack emerging for 2026 looks noticeably different:
User Query
│
▼
Query Planner
│
┌─────────────┼─────────────┐
│ │ │
▼ ▼ ▼
BM25 Vector SQL
Search Search Filters
│ │ │
└─────────────┼─────────────┘
▼
Reranker
│
▼
Context Selection
│
▼
LLM
What's missing from that picture is telling: the assumption that vector search sits at the center of retrieval. It no longer does. Vector search has become just one retrieval primitive among several, not the anchor the whole system revolves around.
14. A Broader Takeaway for AI Engineers
This pattern says something about AI infrastructure as a whole.
Early on, the industry tended to spin up a dedicated system for every new capability:
LLM → New database
Embeddings → New database
Vector search → New database
Agents → New orchestration layer
RAG → New retrieval layer
Eventually, though, production teams start asking a simpler question:
"Could we merge some of this?"
Every additional service you run adds:
More cost
More monitoring
More networking
More credentials
More failure modes
More backups
More synchronization
More operational knowledge
Specialized tools tend to win on benchmark charts.
Consolidated systems tend to win in production.
That's precisely why PostgreSQL has become such a compelling option in the AI stack right now.
It doesn't have to be the fastest vector engine available.
It only has to be:
fast enough, cheap enough, dependable enough, and well-integrated enough.
For a large share of real applications, that mix is genuinely hard to beat.
15. Vector Databases Aren't Disappearing — They're Being Redefined
The realistic outcome is more layered than a simple "vector databases are over" narrative.
Expect the ecosystem to settle into three broad tiers.
Tier 1 — Vector search built into the database
PostgreSQL + pgvector
Elasticsearch
OpenSearch
other unified databases
This tier suits teams that want to minimize the number of moving systems.
Tier 2 — Purpose-built vector databases
Pinecone
Qdrant
Weaviate
Milvus
This tier makes sense when vector retrieval alone justifies running separate, dedicated infrastructure.
Tier 3 — Specialized retrieval engines
For workloads that are unusually large or demanding:
GPU indexes
quantized indexes
distributed ANN
custom retrieval engines
hybrid vector/graph systems
The category isn't vanishing.
It's splitting into layers.
Final Thought
The real shift isn't just that PostgreSQL can now hold vectors.
It's that engineers are starting to tune the whole retrieval pipeline rather than a single piece of it.
A production RAG system doesn't care whether your ANN benchmark reports:
8 ms
vs
12 ms
if your real request path is actually spending:
40 ms → query processing
30 ms → network
80 ms → reranking
500 ms → LLM
The vector database was never really the end goal.
Retrieval quality was.
Once you weigh the full picture — cost, latency, filtering, security, consistency, maintenance and relevance — justifying yet another standalone database gets a lot harder.
So the honest conclusion is:
Vector databases aren't dying.
Something more interesting is unfolding.
Vector search is turning into a database feature.
And once a technology shifts from being its own product category to being a feature, everything about the surrounding architecture changes for good.