Beyond Similarity Search: Metadata Filtering for RAG with S3 Vectors
Metadata filtering for RAG on Amazon S3 Vectors: where filter values come from, when a classifier earns its place, and filterable vs non-filterable metadata.
This article is published in English.
Sources: Beyond Similarity Search: Metadata Filtering for RAG with Amazon S3 Vectors
Semantic search is the best part of RAG. People don't have to phrase a question the way a document does. The system matches meaning and surfaces content about a related idea. But similar isn't always what you want.
Picture an online store whose knowledge base covers products, shipping, warranties, returns, payments and accounts. A shopper asks whether a damaged item can be returned after 30 days. Several documents mention damage, refunds, warranties or time limits, and all of them look relevant to a vector search, yet only some answer the question. Metadata filtering helps here, but first you have to settle where the filter value comes from.
Where the filter comes from
Sometimes the application already knows. A customer on the Returns Help page who asks that question gives you the category for free, with no extra model call needed.
User Question +Known Application Context
↓
category = returns
↓
Vector Search
Selected product, region, language or app section can serve the same role.
Other systems only have the user's words. A general chatbot opens blank, and nothing tells it that the right category is returns. In a small, focused knowledge base that's fine: run plain semantic search with no filter.
User Question
↓
Query Embedding
↓
Vector Search
↓
Relevant Results
As the base grows, topics overlap. Returns, warranties, refunds and damaged goods blur together. Then a small query-understanding or classification step before retrieval can help. The system already knows its own categories (returns, warranty, shipping, payments), and a light model, rules or an LLM can pick the likely ones.
User Question
↓
Query Understanding
↓
returns
↓
Vector Search
↓
category = returns
That step costs time and money, so don't add it to every RAG app just because filtering exists. If plain semantic search already works, keep the flow simple.
One more caution: when the classifier isn't confident, don't force a category. A damaged item might involve both returns and warranty, so search both or skip the filter. A wrong filter can hide the one document that held the answer. The aim is not to filter everything. It's to filter when you have a dependable reason.
Storing metadata with vectors
Filtering only works if metadata is stored alongside the embedding. A chunk from the return policy might carry a document name, category, region and status.
{
"document_name": "Return Policy",
"category": "returns",
"region": "us",
"status": "active",
"chunk_text": "Damaged products can be returned within the allowed return period."
}
The two serve different jobs. The embedding captures meaning, so "send an item back" and "return a product" land close together. Metadata says where the content belongs: here, returns, US region, active document.
Filtering the query
A plain Amazon S3 Vectors query finds the nearest vectors across the whole index.
const result = await s3Vectors.send(
new QueryVectorsCommand({
vectorBucketName: "rag-demo-vectors",
indexName: "store-knowledge",
queryVector: {
float32: queryEmbedding,
},
topK: 5,
returnMetadata: true,
returnDistance: true,
})
);
If you reliably know the search should stay in returns, put that condition in the query itself.
const result = await s3Vectors.send(
new QueryVectorsCommand({
vectorBucketName: "rag-demo-vectors",
indexName: "store-knowledge",
queryVector: {
float32: queryEmbedding,
},
topK: 5,
filter: {
category: "returns",
},
returnMetadata: true,
returnDistance: true,
})
);
Now the query combines both signals.
(Meaning of the Question + category = returns)
↓
Amazon S3 Vectors
↓
Relevant Return Information
S3 Vectors evaluates metadata conditions together with similarity, so it looks for similar vectors that also match, instead of grabbing the nearest hits first and discarding the unwanted ones. That helps when a base holds many related topics.
Filterable and non-filterable fields
Not every field should be searchable. Category, region and status make good filters because you narrow retrieval with them.
{
"category": "returns",
"region": "us",
"status": "active"
}
In S3 Vectors metadata is filterable by default unless you say otherwise. Large text chunks and similar context are better kept non-filterable, since you want them back with the result but never filter on them.
{
"chunk_text": "Damaged products can be returned within the allowed return period."
}
metadataConfiguration: {
nonFilterableMetadataKeys: ["chunk_text"],
}
The text still comes back when metadata is requested. A way to remember it: filterable metadata narrows the search, non-filterable metadata carries what you want back afterward.
More than one condition
Often one condition is enough. If you know more, combine them, for instance only active US return content.
filter: {
$and: [
{
category: {
$eq: "returns",
},
},
{
region: {
$eq: "us",
},
},
{
status: {
$eq: "active",
},
},
],
}
When a question could belong to returns or warranty, use $in.
filter: {
category: {
$in: ["returns", "warranty"],
},
}
S3 Vectors also offers $or, $exists and several comparison operators. Most RAG apps need only one or two conditions.
Plan metadata at ingestion
Metadata is far easier to use when you think about it while ingesting documents. For the store example, a small structure may be plenty.
{
"document_name": "Return Policy",
"category": "returns",
"region": "us",
"status": "active",
"chunk_text": "..."
}
You don't need every possible detail on every vector. Ask whether a field could genuinely help retrieval later. If it won't narrow a search, it likely doesn't need to be filterable.
How the two pieces fit
Metadata doesn't replace semantic search. Similarity captures the meaning of the question, and metadata sets a useful boundary when you know where the answer should come from.
User Question
↓
(Semantic Similarity + Reliable Metadata Context)
↓
Amazon S3 Vectors
↓
More Focused Results
The key word is reliable. If the app already knows category, region or product, filtering is simple and effective. If it only has the question, first decide whether similarity alone is enough. For larger or noisier bases, query understanding can find useful metadata first, and uncertain results are safer handled with a broader search than with the wrong filter.
One practical test helps decide whether you need the classifier at all: collect twenty real questions, run them with and without a category filter and compare the top results by hand. If the unfiltered answers already point at the right document nearly every time, skip the extra step. If related categories keep crowding each other out, that is the signal to add filtering, and the same twenty questions become a regression set for later changes.\n\n## Closing
Vector similarity matters because people rarely repeat a document's exact words, but real knowledge bases hold many related kinds of information, and metadata gives semantic search extra context. Not every query needs a filter or an extra model call. Use known metadata directly, and add classification only when it pays for itself. The point isn't more complexity, only a search that looks in the right place when you have enough information to say where that is.