Spring AI RAG With an In-JVM Vector Store: Drop Qdrant and Docker

Retrieval-Augmented Generation doesn't automatically mean running and looking after a separate vector database. A Spring AI application can keep its vector index inside the JVM, in the same process as everything else. Here's how to set it up.

Where the in-memory option falls short

Spring AI works with many vector stores, among them Chroma, Pinecone, PostgreSQL with pgVector, and Qdrant. It also ships a built-in class, SimpleVectorStore. That class suits demos and tests but not production. It doesn't run a real vector search. For every query it scans each document one by one. It also has no support for document metadata, so there's no way to narrow results by metadata on top of vector similarity.

A newer library called Vectors fills the gap. It's a true vector index that lives entirely inside the JVM, with metadata support and efficient similarity search. You get semantic search without deploying or operating another database.

The plan here is to take a RAG application that currently uses Qdrant and swap the store for Vectors, so the whole app runs with no outside service.

Swapping the dependency

Start by adding the Vectors starter:

implementation 'com.integrallis:vectors-spring-boot-starter:0.1.4'

The version has to be 0.1.4 or newer. Earlier releases lacked auto-configuration, so you had to create the VectorStore bean yourself, along with a VectorCollection bean. From 0.1.4 on, auto-configuration builds both for you.

Next, remove three things: the Qdrant dependency, the Docker Compose support dependency, and compose.yaml itself. None of them are needed, because Vectors runs inside the application and there's no server to start or supervise.

The rest of the app stays as it was. Since Spring AI hides the store behind the VectorStore abstraction, the ingestion code in load() runs exactly as before, and the QuestionAnswerAdvisor set up on the ChatClient needs no edits.

One JVM flag before the first run

Vectors relies on the Java Vector API, which is still an incubating JDK feature, so that module has to be turned on. For Gradle, add this to build.gradle:

tasks.named('bootRun') {
    jvmArgs('--add-modules', 'jdk.incubator.vector')
}

That covers runs started with Gradle, such as ./gradlew bootRun.

Launching from IntelliJ IDEA means adding the argument to the run configuration as well:

1. Go to Run → Edit Configurations.
2. Choose the run configuration for your application.
3. Find the VM options field. If it isn't showing, click Modify options and switch on Add VM options.
4. Type --add-modules=jdk.incubator.vector into the field.
5. Click Apply, then OK.
6. Start the application as usual.

On startup, log messages should confirm that the document chunks were loaded into the vector store.

Why nothing else has to change

The abstraction does the heavy lifting. During ingestion, the app splits the rulebook into chunks and hands them to whatever VectorStore bean exists. At question time, the QuestionAnswerAdvisor asks that same bean for the chunks closest to the user's question and adds them to the prompt before the model answers. Neither step knows or cares whether the bean talks to a Qdrant container over the network or to an index sitting a few objects away in the same heap. That's the whole reason the swap amounts to a dependency change plus one JVM flag.

Try some questions

With the Sagrada board game rulebook as the source document (the same one used with Qdrant), ask things like these:

• What is the name of the game?
• How many players can take part?
• How many dice get rolled in each round?

Answers should be grounded in the rulebook, just as they were with Qdrant. The difference is that now everything happens inside the JVM.

Saving the index between runs

Vectors has more features than the basic setup above touches, and its documentation is worth a read. Persistence is the one most apps will want, so vectors survive a restart and nothing needs to be reloaded each time. Switching it on takes a single property:

java-vectors.storage-path=/tmp/vectors/rag-recipe

Pick any directory for the storage path. After that, the store keeps its contents from one run to the next, and documents don't have to be ingested again at every startup. One consequence: after the first run, remove the load() method from RagIngestionConfig, or each restart will add duplicate copies of the same document.

When it fits

For applications that don't need a standalone vector database, Vectors sits in a handy middle ground. It gives real vector search, metadata support and optional persistence, all running within the application's own JVM. Anyone who wants RAG in a Spring AI app without adding yet another service to deploy and maintain should give it a look.

Trade-offs to weigh

Running the index in-process has costs that are easy to forget. The vectors take up memory in the same JVM that serves requests, so a large corpus competes with the rest of the application for heap. With persistence enabled, the data sits on that machine's disk, which ties it to a single application instance. If several instances need to share one index, or the corpus is far larger than a single JVM should hold, a standalone database like Qdrant is still the better choice. For a documentation helper, an internal rulebook bot or a prototype, though, dropping a whole service from the deployment is often a very good trade.