Practical notes: Beyond Similarity Search: Metadata Filtering for RAG with

The following notes reconstruct a practical path around “Beyond Similarity Search: Metadata Filtering for RAG with Amazon S3 Vectors”. Emphasis stays on contracts, checks, and drop-in code placeholders rather than motivational framing. When working through the Overview stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

Where Does the Filter Come From?

The Where Does the Filter stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Separate chunking policy from retrieval policy. Changing one should not force a rewrite of the other when quality metrics move.

User Question +Known Application Context
      ↓
category = returns
      ↓
Vector Search

User Question
      ↓
Query Embedding
      ↓
Vector Search
      ↓
Relevant Results

User Question
      ↓
Query Understanding
      ↓
returns
      ↓
Vector Search
      ↓
category = returns

Adding Metadata to the Vectors

The Adding Metadata to the stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Separate chunking policy from retrieval policy. Changing one should not force a rewrite of the other when quality metrics move.

{
  "document_name": "Return Policy",
  "category": "returns",
  "region": "us",
  "status": "active",
  "chunk_text": "Damaged products can be returned within the allowed return period."
}

Filtering the Search

The Filtering the Search stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Separate chunking policy from retrieval policy. Changing one should not force a rewrite of the other when quality metrics move. The Filtering the Search stage works best when treated as a measurable surface. Capture one golden transcript, one failure case, and the rollback note before expanding scope. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

const result = await s3Vectors.send(
  new QueryVectorsCommand({
    vectorBucketName: "rag-demo-vectors",
    indexName: "store-knowledge",
    queryVector: {
      float32: queryEmbedding,
    },
    topK: 5,
    returnMetadata: true,
    returnDistance: true,
  })
);

const result = await s3Vectors.send(
  new QueryVectorsCommand({
    vectorBucketName: "rag-demo-vectors",
    indexName: "store-knowledge",
    queryVector: {
      float32: queryEmbedding,
    },
    topK: 5,
    filter: {
      category: "returns",
    },
    returnMetadata: true,
    returnDistance: true,
  })
);

(Meaning of the Question + category = returns)
        ↓
Amazon S3 Vectors
        ↓
Relevant Return Information

Filterable and Non-Filterable Metadata

For the Filterable and Non-Filterable Metadata stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Cite the passages that actually grounded the answer. Without citations, operators cannot tell hallucination from an indexing gap.

{
  "category": "returns",
  "region": "us",
  "status": "active"
}

{
  "chunk_text": "Damaged products can be returned within the allowed return period."
}

metadataConfiguration: {
  nonFilterableMetadataKeys: ["chunk_text"],
}

Using More Than One Condition

For the Using More Than One stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Cite the passages that actually grounded the answer. Without citations, operators cannot tell hallucination from an indexing gap.

filter: {
  $and: [
    {
      category: {
        $eq: "returns",
      },
    },
    {
      region: {
        $eq: "us",
      },
    },
    {
      status: {
        $eq: "active",
      },
    },
  ],
}

filter: {
  category: {
    $in: ["returns", "warranty"],
  },
}

Think About Metadata Early

For the Think About Metadata Early stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Document the happy path and the recovery path together. Retries, human gates, and dead-letter handling are part of the product, not later polish. Cite the passages that actually grounded the answer. Without citations, operators cannot tell hallucination from an indexing gap. For the Think About Metadata Early stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state. Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

{
  "document_name": "Return Policy",
  "category": "returns",
  "region": "us",
  "status": "active",
  "chunk_text": "..."
}

Similarity and Metadata Work Together

When working through the Similarity and Metadata Work stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Record timings and token or query cost next to functional results. Cost visibility early prevents surprise bills when the path moves from demo to shared environments. Measure recall on a fixed question set before tuning prompts. Prompt churn rarely fixes a weak retrieval surface.

User Question
      ↓
(Semantic Similarity + Reliable Metadata Context)
      ↓
Amazon S3 Vectors
      ↓
More Focused Results

Conclusion

When working through the Conclusion stage, write down the contract first: required inputs, success signal, and what happens on partial failure. That checklist keeps later code changes honest. Keep configuration outside application code. Environment files, secret stores, and feature flags belong in one place operators can audit without reading the whole graph. Measure recall on a fixed question set before tuning prompts. Prompt churn rarely fixes a weak retrieval surface.

Operational checklist

For the Operational checklist stage, define the inputs, the owner of the step, and the exit criteria before changing code. Operators should be able to re-run the step from a known checkpoint without guessing hidden state.

Prefer small, testable units over sprawling scripts. When a step fails, the failure should point at a single responsibility rather than a tangled pipeline.

Cite the passages that actually grounded the answer. Without citations, operators cannot tell hallucination from an indexing gap.

Write a short runbook: how to rotate keys, how to drain the queue, how to roll back the last ingest.

Treat this stage as a contract between inputs and validated outputs. Name the artifacts, define success checks, and refuse silent partial completion.

Cite the passages that actually grounded the answer. Without citations, operators cannot tell hallucination from an indexing gap.

Before promoting the stack, freeze versions, capture a golden transcript for the critical path, and confirm rollback steps. Shared environments need rate limits, tenancy checks, and a clear owner for secret rotation. Prefer boring reliability over clever one-off demos.

Batch note for 5ab755d9f682: keep provider keys out of the repo, set a per-session token ceiling, and store transcripts next to the eval fixtures so later model swaps stay comparable.