Switching RAG Embedding Models Without Reindexing: How rebasis Works

Upgrading the embedding model behind a retrieval-augmented generation system sounds routine until the existing vector index gets in the way. An open-source Python tool called rebasis targets that exact problem. It trains a small adapter that translates query vectors from a new embedding model into the space the old index already understands. A full reindex can then be put off, or carried out slowly and under supervision.

The tool tries several kinds of mapping (centered Orthogonal Procrustes, affine, low-rank and small residual networks) under one shared evaluation protocol. It reports more than alignment quality. It also estimates how much better the new model really is on the corpus in question, and whether the adapted system beats staying put. Confidence intervals from a paired bootstrap, several retrieval metrics, fingerprints of the encoding setup, reproducible audit records and a crash-safe migration routine together make an upgrade a measurable decision instead of a leap of faith.

The topics are RAG, embedding drift, vector databases, backward compatibility, model migration, retrieval evaluation and approximate nearest-neighbor search.

Why the embedding model gets locked in

Building a RAG system commits you to more than a language model and a vector database. You also commit to the coordinate system of whichever embedding model was current on the day of indexing. Documents, whether thousands or millions, get chunked, encoded and written into an index. Sooner or later a faster, better or more fitting embedding model shows up, and that's where trouble starts: a query vector from the new model means nothing to an index built by the old one.

"Just reindex everything" is the stock advice. It's technically right and often costly. A big personal knowledge base, an internal documentation set, a codebase or the long-lived memory of an agent has to be re-chunked and re-embedded, which takes time, compute, API budget and carries operational risk. Many systems therefore stay on a stale model long after a better one exists.

rebasis was built to break that lock-in. Its stated goal is to let a local RAG system change its embedding model without deleting the vector index it already has.

Transform the query, not the index

Two embedding models don't produce comparable spaces. Encode one sentence with each and the dimensionality may differ, but the differences go deeper: what each axis means, where the space is centered, how it is scaled and how semantic neighborhoods are arranged can all shift. A query vector from the new model, sent straight to the legacy index, returns largely meaningless results. Some vector databases refuse the query outright because the dimension doesn't match the collection.

The tool's answer is a small learned adapter. It takes a query vector from the new model and maps it into the space the existing index speaks. To learn the mapping, the same texts are encoded by both the old and the new model, producing paired vectors. If the relationship between the two spaces is learnable enough, a fresh query vector can be converted into something meaningful to the old index.

The practical upshot is large: the new model can serve the query side immediately, and the document vectors in the index stay untouched. Integration into an application is kept deliberately small. First the import:

from rebasis import Bridge

Then loading the adapter and putting it on the query path:

bridge = Bridge.load("adapters/minilm-to-bge.rbs")query_vector = new_model.encode(["how do I deploy to production?"])
query_vector = bridge.to_index_space(query_vector)
results = collection.query(query_embeddings=query_vector, n_results=10)

Bridge is the thin layer sitting on the hot path of the RAG application. When the adapter loads, it validates model identities, encoding profiles, dimensions and the integrity of the adapter file. After that, each query costs only the matrix operation. Logging, dictionary construction and repeated validation are intentionally left off this path, since a small convenience shouldn't turn into a cost paid on every request. No GPU is needed either, because at this scale shuttling data between devices tends to cost more than the multiplication.

The math behind the default adapter

Call the old embedding model f_old and the new one f_new, and take n documents encoded by both. That gives two data matrices, one in each space. The aim is to learn a function mapping the new space into the old one, and in its general form the training objective minimizes the Frobenius-norm distance between the mapped new vectors and the old ones.

The default method is centered Orthogonal Procrustes. Both matrices have their column means subtracted first. For spaces of equal dimension the constraint is that the transformation must be a rotation-like orthogonal matrix. Take the singular value decomposition of the product of the centered matrices, and the best orthogonal matrix has a closed form, built from the left and right singular vectors multiplied together. A query is then transformed by centering it with the new space's mean, rotating it, and adding back the old space's mean. This is the classic Procrustes problem, studied since the 1960s.

No iterative optimization is involved in that solution, so there's no learning rate and no choosing of epochs. The same input data always gives the same transformation.

Vectors are length-normalized before fitting and again after the adapter has been applied, so cosine similarity reduces to an inner product. Centering matters because embedding spaces are anisotropic and share a common mean component. Removing it spares the transformation from having to represent a translation that a plain rotation can't express. If a pair of models needs more freedom than that, affine, low-rank or residual alternatives take over.

One operationally important property follows: training cost depends on the number of matched pairs, not on the size of the corpus. An adapter can be fitted from a few thousand representative pairs even when the index holds hundreds of thousands of records. A full-matrix transformation costs on the order of the product of the two dimensions per query and uses parameter memory of the same order. A rank-r factorization brings the query cost down to roughly r times the sum of the dimensions. As a concrete figure, a 768 by 768 float32 matrix takes about 2.36 MB, which is tiny next to the index it saves from rebuilding.

Why one transformation method isn't enough

The relationship between two embedding spaces is frequently close to a rotation, but not for every pair of models. rebasis therefore doesn't assume a single algorithm can work everywhere. It starts from Orthogonal Procrustes, the closed-form way to best align two matched point sets. Real spaces, though, can show mean shifts, differences in scale per dimension and nonlinear components.

With --method auto, the tool fits several candidates on the same data split and with the same preprocessing, then compares them:

• Procrustes, in centered and classical variants.
• A ridge-regularized linear map.
• A low-rank affine map.
• Procrustes with a scale per dimension.
• When PyTorch is available, a small residual MLP.

CSLS isn't forced onto the mapping; it too competes as a candidate. The reasoning is practical. In high-dimensional spaces it can ease the "hubness" problem for weak adapters, yet it can harm an alignment that was already strong. The aim isn't a long list of algorithm names but a winner picked by measurement on held-out data. When several candidates fall inside the same uncertainty region, the tool prefers the one with fewer parameters and lower cost.

Training and evaluation data stay strictly disjoint throughout, and that isn't just a convention. The tool checks for overlap at runtime. Scoring an adapter on the examples it learned from would give an impressive number that proves nothing.

Asymmetric embedding models get explicit treatment as well. In the E5 family, for instance, the query: and passage: prefixes mean the same text is encoded differently depending on its role, and other models use query or document instructions for the same purpose. Fit the adapter on document-encoded pairs, apply it to query-encoded vectors, and you've created a textbook train-serve mismatch. For that reason an encoding profile in rebasis fingerprints much more than the model name: dimensionality, normalization, pooling, the query and document prefixes and any Matryoshka dimension are all included. An adapter built under the wrong profile is rejected at load time rather than quietly returning poor results.

How this fits with earlier research

Making embeddings compatible across versions is an old question. Backward-Compatible Training constrains a new model during training so its representations stay comparable with the legacy ones. Forward Compatible Training goes the other way, preparing side information while the old model is trained so that later upgrades are easier. Both suit big retrieval systems that control the model's whole lifecycle from the design stage.

rebasis addresses another situation. The index already exists, the old and new models were trained separately, and nobody can touch either training process. That makes it a post-hoc compatibility layer. It draws on Orthogonal Procrustes, low-rank transformations and the more recent Drift-Adapter line of work, and its own contribution is engineering: packaging these ideas into one local tool that also adds measurement, adapter selection, decision theory, abstractions over stores, safe migration and a workflow for reproducible audits.

CSLS comes from the literature on cross-lingual embedding alignment. It accounts for the local neighborhood density of the query and of the target, and penalizes "hub" vectors that become the nearest neighbor of too many queries. Roughly speaking, it scores a pair as twice the cosine similarity minus a density term for each side. The project's own real-corpus experiments reportedly found that it helped weak mappings and sometimes hurt strong ones, which is why it's measured instead of applied blindly.

A tool that sometimes tells you not to use it

Early in the project, the question "can a bridge be built between these two spaces?" turned out to be the wrong one to ask on its own. Three questions matter more:

1. Is the new embedding model really better on this particular data?
2. How much of the quality of a full reindex does the adapter keep?
3. Is bridging better than simply staying with the current model?

That's why probe, rather than fit, is presented as the key command. It folds sampling, embedding, ground-truth construction, candidate fitting, metric computation and the final decision into one measurement pipeline, and it never writes to the existing index.

One central measure is ARR, Adapted Recall Retention: the quality of the adapted search as a fraction of what a complete reindex with the new model would deliver. A high ARR isn't enough by itself. If the new model barely improves on the old one, an adapter has little operational value. If the new model is a big step forward but the adapter can't carry that gain over, a full reindex may be the right answer.

The decision therefore turns on a break-even measure:

bridge_advantage = ARR × upgrade_gain

Writing it out makes the logic clearer. Let Ra be Recall@k for the adapted system, Rn the same for a system fully reindexed with the new model, and Ro for the current old system. ARR is then the gain of the adapted system over the old one divided by the gain of the full reindex over the old one, and the upgrade gain is the ratio of Rn to Ro. Multiply them and the Rn terms cancel, leaving Ra divided by Ro: the adapted system compared straight against the user's real alternative of doing nothing.

Each factor answers a plain question. ARR asks how much of the ideal reindex was retained. Upgrade gain asks how much better the new model is. Their product asks whether bridging today actually moves things forward. A value above 1 means bridging beats staying on the current model; below 1 it doesn't. When a measurement lands too close to the threshold, the tool doesn't manufacture certainty. It labels the result borderline and recommends a larger sample.

Given a genuine query log plus human-judged relevant document IDs, probe is able to measure the new model's real gain on that corpus. Without such a log, held-out documents can stand in as queries, and keyword-based synthetic queries give an approximate evaluation. Each result states its evidence tier, so the output carries both a score and the basis on which it was produced.

A panel of measurements, not one number

Recall@k is the main decision axis, but retrieval quality isn't squeezed into a single figure. A single probe run produces several complementary signals:

• MRR@k shows how high the first relevant hit appears.
• nDCG@k weighs graded relevance and rank position together.
• Top-k overlap reveals how much of the oracle's neighborhood the adapted result set keeps.
• Spearman correlation follows how badly the internal ordering of shared results degrades.
• Kolmogorov-Smirnov distance measures how far calibration moves the distribution of similarity scores.
• Tail ARR sets the sparsest clusters' quality against the global average, so drift that hits some regions harder gets exposed.

The distinctions matter. Two systems may return the same documents in different orders, which recall can't see and nDCG can. Similarly, ranking can look fine while the shape of the score distribution changes enough to defeat a filter with a hard cutoff. If recall-based and nDCG-based break-even results disagree, rebasis reports the disagreement and doesn't quietly substitute one metric for the other.

Uncertainty belongs in the result

An ARR measured on a sample is a point estimate, so the tool attaches a 95% confidence interval from 2,000 bootstrap resamples, following Efron's method. At the T1 evidence tier, ARR is a ratio of two means. Resampling the numerator and denominator independently would wreck their statistical dependence, so a paired ratio bootstrap reuses the same query indices for both. The bounds are the 2.5th and 97.5th percentiles of the resulting distribution.

Pairing preserves a real correlation: a query that is hard for the adapter is usually hard for the oracle too. When the interval straddles a decision threshold, the result isn't treated as settled. Instead of mistaking decimal places for scientific certainty, the resolution of the measurement becomes part of the decision. The seed, sampling strategy and metric version go into the audit record, so the interval can be reproduced.

The project says measurement steered its direction directly. According to its published report, it compared the old model, a naive model swap, adapted retrieval and a full reindex on identical query sets, across 11 corpora, 398,010 documents and 10,346 real user queries. Human relevance judgments were used, and the main results were scored independently with ranx instead of relying only on the tool's own metric code. By those numbers, a direct swap kept roughly 12.5% of the quality of a full reindex, whereas bridging kept about 71%, or about 6.4 times more recovery than the naive swap.

The more telling result is that bridging wasn't right every time. It was truly worthwhile in just 12 of the 62 model and corpus pairs measured. The decision rule pointed the right way in 61 of 62 cases, including 32 of 33 on corpora it hadn't seen before the rule was frozen. A tool earns trust by saying "reindex here" or "you don't need this upgrade" when that is the honest answer, as well as by saying "use me."

The command-line workflow end to end

rebasis isn't only a library function. Its command line turns a model change into a manageable process. It starts with a check of the environment and of which stores and embedders are available:

# Inspect the environment and available store/embedder support
rebasis doctor

Next come the three stages of the actual process: measure first and pick a path, fit the adapter, then migrate the index in background batches when ready.

# Measure first and choose the right path
rebasis probe --store chroma:///path/to/db#notes \
  --old sentence-transformers/all-MiniLM-L6-v2 \
  --new BAAI/bge-base-en-v1.5 \
  --queries queries.jsonl --sample 10000 --report report.html# Fit the appropriate adapter
rebasis fit --store chroma:///path/to/db#notes \
  --old sentence-transformers/all-MiniLM-L6-v2 \
  --new BAAI/bge-base-en-v1.5 \
  --method auto --pairs 4000 \
  --out adapters/minilm-to-bge.rbs# When ready, migrate the index in background batches
rebasis migrate --adapter adapters/minilm-to-bge.rbs \
  --store chroma:///path/to/db#notes --priority access --limit 5000

The resulting .rbs adapter holds more than weights. Its manifest records the identities of the old and new models, the vector dimensions, the direction and the encoding-profile fingerprints. Tensors are stored in the safetensors format, and the score calibration and evaluation results travel along. An adapter is therefore a portable, auditable record of which models and conditions produced it, and the tensor hashes can be recomputed to verify integrity.

Score calibration deserves emphasis. A transformation can preserve ranking and still shift the absolute similarity scores. An application with a fixed cutoff such as similarity > 0.7 could then silently drop results. To handle this, the tool fits a monotonic isotonic calibrator that moves scores back toward the legacy scale without altering their order, and it still warns systems that depend on fixed thresholds.

Compute and memory behavior

A tool that doesn't theoretically need the whole corpus in memory might still load it by accident. In rebasis, corpus iterators stream records, so peak memory on the migration path is on the order of batch size times vector dimension. Runtime grows with the record count, but the user's corpus is never materialized in RAM. As memory approaches the --max-memory ceiling, the batch size can shrink and the checkpointed job can pause instead of being killed.

The similarity matrix used for ground-truth search, queries by sample size, isn't built in full either. The queries are processed in blocks. Each block does a matrix multiplication, pulls out the top-k results and throws the intermediate score matrix away. A full sort would cost S log S per row. Using argpartition keeps the expected selection cost per row linear in S. For normalized float32 vectors cosine search collapses into one matrix multiplication, which keeps both peak memory and dtype-related platform differences under control.

On this project the CPU is the numerical reference rather than a second-class fallback. A GPU can speed up embedding generation a great deal, but host-device transfer can erase the benefit for a tiny single-query adapter operation. Device-equivalence tests compare score tolerances and also retrieved neighbor sets and, above all, whether identical data leads to the identical final decision. A numerically small difference that pushes an ARR across a threshold like 0.95 isn't acceptable hardware variation; it counts as a defect in product behavior.

Migration that survives interruptions

Bridging means the index needn't change right away; the system can keep running through the adapter as long as wanted. Should the index be moved permanently, migrate shifts records over to the new space batch by batch, and in this part data safety comes before raw speed.

Every batch begins with a read of the current vectors, which are copied into a shadow store ahead of any overwrite. The transformed vectors are written next, and a sample of the written records is read back to verify them. At the end the tool doesn't trust only its current client handle. It reopens the store through a fresh connection and samples again for durability, so client-side caching can't hide what actually reached persistent storage.

The migration engine rests on four invariants:

1. An original float32 vector must exist in the shadow store before its record is changed.
2. A post-write sample from each batch marked complete must be readable within tolerance.
3. A finished job must still verify after the store is reopened through a new connection.
4. Rollback must restore the original vectors without changing the record count or the payloads.

The ordering is the backbone of the crash-safety argument. If the process dies after the shadow write but before the main-store write, the result is a harmless extra copy. Swap those two steps and the same interruption could destroy an original vector for good.

The migration queue doubles as the checkpoint, so --resume picks up after the last completed batch. The tool can watch memory pressure and battery state, and an access log can put the most-requested records first. If the outcome disappoints, rollback restores the originals from the shadow copy. Contracts shared across the backends (Chroma, LanceDB, sqlite-vec, Qdrant, FAISS and an in-memory one) keep these operations meaning the same thing everywhere.

Plugins for different stacks

Local RAG doesn't revolve around a single vector database or embedding provider, so storage and embedding sit behind plugin-based registries. Stores covered: Chroma, LanceDB, sqlite-vec, Qdrant and FAISS, plus a backend that lives in memory. For producing embeddings there are sentence-transformers, fastembed, Ollama, OpenAI-compatible APIs, llama.cpp and precomputed vectors.

The core installation is meant to work without PyTorch. Core probe features, Procrustes fitting and the decision logic run on NumPy, SciPy and scikit-learn, and heavy or platform-specific dependencies are installed only by those who need them. Third-party packages can expose a rebasis.stores or rebasis.embedders entry point to add an integration without forking the project.

Privacy, logging and reproducibility

By design rebasis sends no local data anywhere. It has no phone-home telemetry and no usage counter, and document text and vectors never reach the logs. Since information can be reconstructed from an embedding, vectors are treated as just as sensitive as plain text. OpenTelemetry support exists as an option, is off by default and exports only to the user's own collector.

Each decision can be recorded in an append-only audit log so it stays reproducible. The entry holds model identities, profile fingerprints, the sampling method, the seed, thresholds, the metric version, the rebasis version and environment details. Entries are linked in a hash chain, which makes later corruption or partial edits visible. rebasis audit replay reruns an earlier measurement, to find out whether a difference stems from a software regression or from a changed corpus.

What the open-source packaging aims to prove

rebasis is released under the Apache 2.0 license, and the stated aim is to make its claims inspectable as well as its code readable. In the repository you'll find architecture decision records for the key choices, along with golden scenarios, measurements on real corpora, performance budgets, contract tests per backend, CLI tests that run end to end, and contributor docs. The test strategy goes beyond the happy path to cover numerical consistency across devices, memory ceilings, adapter properties, store contracts and the durability of migration and rollback.

The CI pipeline targets Python 3.12 and 3.13 across Linux and macOS, two systems whose storage semantics differ. Minimum supported dependency versions get a run of their own, and a separate job checks that optional extras such as PyTorch or OpenTelemetry don't leak into the core package. Type checking uses mypy --strict, code quality is handled by Ruff, layer boundaries by import-linter and documentation links by MkDocs in strict mode. The thinking behind all this is that open-source trust depends less on bold README claims than on showing which tests and measurements stand behind them.

Model changes without the all-or-nothing choice

The underlying problem isn't picking a technology but the cost of changing it. As RAG systems grow, their indexes become infrastructure investments, and picking an embedding model turns into a decision far harder to reverse than it should be. The idea behind rebasis is to loosen that coupling.

When a new model appears, the first step should be measuring whether it's actually better on your own data. If it is, the next is to find out whether a reliable bridge can link it to the current index. If the bridge is good enough, the new model can be used immediately. If full quality is wanted over time, migration can proceed in controlled batches with verification and rollback guarantees. And if the bridge falls short, that should be discovered before an expensive production experiment, not during one.

That's the decision space the tool offers: measure first, bridge next, migrate safely when needed. It's an open tool meant to make local RAG infrastructure sturdier, embedding-model choice less binding and model upgrades more accountable, and it is open for developers to inspect, try on real corpora and help develop.