Designing Portable Multi-Tenant Semantic Search on Kubernetes and Qdrant
How a semantic search platform uses Kubernetes for portability and tenant-aware routing to keep its vector database replaceable and its workloads isolated.
This article is published in English.
Sources: Stop Searching by Coincidence: Portability and the Storage Decoupling Layer
Once a system is built, its architecture diagram seems the obvious answer. A gateway tied to an embedding model, a vector store, a broker and a relational database looks like the only reasonable arrangement: queries go here, ingestion goes through a queue, the model lives apart from the app.
Real systems seldom appear that tidily. They accumulate around constraints, where one decision creates pressure elsewhere and the fix adds yet another requirement. The finished picture then looks planned. The more useful question is why each boundary is where it is.
The platform in question
The system described here is multi-tenant infrastructure for semantic search and retrieval, and it reaches well beyond a single WordPress search plugin. It has to ingest documents, convert them into retrievable representations, store those, run low-latency hybrid searches and stay flexible enough that no particular vector database or ML model hardens into a permanent dependency.
The building blocks:
- Kubernetes as the infrastructure base.
- A Go API gateway coordinating requests.
- Snowflake Arctic as the main embedding model.
- Qdrant as the main vector database.
- MySQL holding tenant and credential configuration.
- NATS carrying the asynchronous ingestion path.
Choosing Kubernetes before anything else
Picking the orchestration layer ahead of most application parts sounds backwards, since the usual order is to build first and decide deployment later. Here, portability of infrastructure was a product requirement, so deployment couldn't be an afterthought. Two operating modes had to stay open.
One is a public cloud with a managed control plane. The other is a cheaper environment where the team runs the cluster itself once cost outweighs convenience. These aren't identical, and Kubernetes doesn't erase provider differences. Storage, networking, load balancers, GPU supply and node provisioning stay provider-specific. What it does provide is a steady operational boundary sitting above them.
Under that boundary the workloads map cleanly. Model servers are pods and the gateway is a deployment. Qdrant is stateful infrastructure with persistent volumes, and NATS is one more network service. Configuration is declarative, and service discovery works the same whether nodes come from a large cloud or from a self-managed cluster.
That matters because cost curves shift with growth. A managed service is a fine pick when simplicity beats price. The math changes once traffic turns predictable, once workloads justify dedicated nodes or once costly inference hardware dominates the bill. A later move to cheaper infrastructure shouldn't require redesigning the application at the same moment. Kubernetes thus works less as a deployment tool and more as a portability layer for the architecture.
ML infrastructure is also unusually uneven, which gave a second reason. An API service, an embedding model and a vector database each scale in their own way, with distinct memory needs, startup times and failure causes. Stateless services can fan out in seconds, a model server may first need to load a hefty artifact, and a vector store wants steady local disks and deliberate placement. The base layer should host all three without treating them as the same.
It also explains a taste for plain Kubernetes manifests, Kustomize and FluxCD. Helm isn't ruled out in principle. When one person operates and pays for the system, though, keeping the deployment model visible is worth more. Inspecting a StatefulSet should show what's running, how storage mounts, how a node boots and where configuration originates.
Keeping the vector database replaceable
Semantic search systems tend to organize themselves around the vector store. That's natural: it sits on the critical query path, holds expensive derived data and sets latency, recall, memory use and running cost through its indexing strategy. After enough data builds up, leaving it hurts. So the rest of the platform shouldn't depend structurally on any one database.
Qdrant is the primary engine today, but it acts as a routing target and not as the identity of the platform. The reasons for choosing it were covered elsewhere, so the focus here stays on structure.
It would be wrong to call the decoupling layer a universal database API. It consists of tenant-aware placement, backend-specific adapters and a gateway that shields clients from both. Each backend's quirks sit behind a small storage adapter, and for every request the gateway reads the tenant's configured backend, then forwards the operation to that adapter. Vector databases don't share identical semantics, and nothing pretends otherwise. The point is keeping placement and query building out of the public API and the wider request flow. No collection or index structure is presumed: tenant config names the backend type and the indexing node, and the gateway resolves both when the request arrives.
The current production gateway still carries both Qdrant and RediSearch paths behind this layer. Backward compatibility with Redis is only part of the story. No single vector database is guaranteed to suit every retrieval workload, because the ecosystem moves too fast for comfort. Databases keep adding sparse vectors, new quantization methods, disk-backed indexes, late-interaction support, GPU acceleration and alternative distributed search designs. What's optimal now can turn into a bottleneck sooner than expected.
Why route at the application level
There's a deeper distributed-systems argument too. Horizontal scaling isn't infinite the way cloud marketing implies. The phrase "scales horizontally" glosses over coordination cost. As a dataset spreads, queries touch more partitions, replication eats more bandwidth, rebalancing grows pricier, coordination surfaces widen and tail latency starts to track the slowest participant. Depending on the workload, extra capacity in one logical cluster eventually yields diminishing returns.
Multi-tenancy changes the picture because tenants are a natural partition. Nothing requires every independent website to share one giant vector cluster. Some tenants can live on one Qdrant deployment, others on a second, and a particularly heavy one can get a dedicated database node if its load warrants it.
Imagine a deployment hosting thousands of small sites, nearly all comfortable on a shared cluster. Then one client shows up whose catalogue outweighs hundreds of existing tenants put together, with steady query traffic that rewrites the capacity plan. Scaling the shared cluster for everybody shouldn't be the only response. With routing, that tenant moves to separate hardware, the client API stays untouched and unrelated tenants are spared the consequences.
Separate deployments also give a failure boundary. Scaling isn't the sole motive for splitting workloads. Sometimes it's about stopping one pathological tenant or overloaded cluster from becoming everyone's incident.
Sharding asks how to spread data within one database system. Application-level routing asks whether all the data needed to be in the same system at all. A strongly multi-tenant workload benefits from having both. That's why the indexing node and backend live in tenant configuration instead of being hardcoded into application logic. Adding another vector engine later then stays mostly a local adapter task and doesn't become a platform-wide migration.
The next assumption to question
Decoupling storage dealt with one future migration and exposed the next hidden assumption. A hybrid search request doesn't start at the database. It starts at a machine learning model, and models change even faster than storage engines. Once the vector database is accepted as unable to define the platform, the embedding model can't be allowed to define it either.