Building Multi-Tenant RAG on Amazon S3 Vectors: The Pipeline

Giving every tenant of a RAG service its own addressable resource, in place of a shared index with a tenant tag, makes isolation something AWS enforces and a customer's auditor can check. This is the build: each requirement mapped to an API call, a policy or a log line, followed by the parts that still live in application code. The premise is that isolation belongs to the storage layer, not to a WHERE tenant = ? clause somebody might forget.

Layout

One AWS account holds one vector bucket per region served, and inside each bucket sits one index per tenant. The snippets are illustrative TypeScript using @aws-sdk/client-s3vectors and @aws-sdk/client-bedrock-runtime, so check parameter names against the current SDK.

Ingest:  S3 (tenant prefix) ──▶ Lambda: extract + chunk ──▶ SQS ──▶ Lambda: embed (Bedrock Titan V2)
                                                                     └──▶ PutVectors → index/org-<tenant>
Query:   eID provider (OIDC) ──▶ API authorizer (tenant, role) ──▶ STS session policy (one index ARN)
                        ──▶ embed question ──▶ QueryVectors(index/org-<tenant>, filter=classification)
                        ──▶ LLM, grounded on returned chunk_text, citations = document_id + version
Audit:   CloudTrail data events, resource type AWS::S3Vectors::Index, queried by resources.ARN

The shared regulatory catalogue (laws, regulations, guidance) is copied into every tenant index. Duplicating it costs little, and it keeps each tenant's data self-contained.

Creating an index

Onboarding a tenant is a single call, with two irreversible decisions inside it: the embedding dimension and the list of non-filterable metadata keys.

import { S3VectorsClient, CreateIndexCommand } from "@aws-sdk/client-s3vectors";
const s3v = new S3VectorsClient({ region: "eu-west-1" });

await s3v.send(new CreateIndexCommand({
  vectorBucketName: "kb-eu-west-1",      // the bucket is regional → residency
  indexName: "org-b",                       // the index is the tenant → isolation
  dataType: "float32",
  dimension: 1024,                          // Titan Text Embeddings V2, 1024-d
  distanceMetric: "cosine",
  metadataConfiguration: {
    nonFilterableMetadataKeys: ["chunk_text"],   // returned, never scanned, never filterable
  },
}));

The new index gets an ARN built from the service name, the region (eu-west-1), the account, the bucket kb-eu-west-1 and finally index/org-b. The region sits right there in the string, which means listing the indexes of each bucket gives you the data-residency report for free.

Ingestion: pick the keys yourself

Of everything in the pipeline, the choice of vector key matters most. Let the store invent keys and "delete this document" turns into a search for its chunks followed by deleting whatever the search finds, and approximate search can never prove that nothing is left behind. Derive the key from the document id and erasure is a simple lookup.

import { PutVectorsCommand } from "@aws-sdk/client-s3vectors";

async function ingestDocument(tenant: string, doc: Document, chunks: string[]) {
  const vectors = await Promise.all(chunks.map(async (text, i) => ({
    key: `${doc.id}-k${String(i + 1).padStart(3, "0")}`,      // chosen by us
    data: { float32: await embed(text) },
    metadata: {
      document_id: doc.id,                 // filterable
      version: doc.version,                // filterable → appears on the citation
      classification: doc.classification, // filterable → role filter at query time
      chunk_text: text,                    // non-filterable → rides with the vector
    },
  })));  for (const batch of chunk(vectors, 500)) {                    // ≤ 500 per call
    await s3v.send(new PutVectorsCommand({
      vectorBucketName: "kb-eu-west-1", indexName: `org-${tenant}`, vectors: batch,
    }));
  }
  await manifest.put(tenant, doc.id, doc.version, vectors.map(v => v.key));   // written down
}

Notice what the metadata leaves out: the tenant. Nothing on a chunk says "org-b". The chunk is in org-b, and that is what makes it B's. Writing a tenant tag as well would bring back the weakness of tag-based designs, where a wrong tag looks just like a right one, for no gain.

PutVectors writes are strongly consistent, and each call is billed as at least 128 KB, so group writes into batches.

Embedding, for completeness:

import { BedrockRuntimeClient, InvokeModelCommand } from "@aws-sdk/client-bedrock-runtime";
const bedrock = new BedrockRuntimeClient({ region: "eu-west-1" });

async function embed(text: string): Promise<number[]> {
  const res = await bedrock.send(new InvokeModelCommand({
    modelId: "amazon.titan-embed-text-v2:0",
    contentType: "application/json",
    body: JSON.stringify({ inputText: text, dimensions: 1024, normalize: true }),
  }));
  return JSON.parse(new TextDecoder().decode(res.body)).embedding;
}

Inference runs in the region you call, and Bedrock doesn't retain prompts or use them for training, which is part of the residency story.

Querying with credentials that name one index

In this design the tenant boundary doesn't live in the query as a filter. It is the destination of the query, the one destination the caller's credentials are allowed to open. The API creates those credentials fresh for every request.

import { STSClient, AssumeRoleCommand } from "@aws-sdk/client-sts";

function indexArn(tenant: string) {
  return `arn:aws:s3vectors:eu-west-1:${ACCOUNT}:bucket/kb-eu-west-1/index/org-${tenant}`;
}async function tenantCredentials(tenant: string) {
  const res = await sts.send(new AssumeRoleCommand({
    RoleArn: QUERY_ROLE_ARN,                       // broad role
    RoleSessionName: `q-${tenant}`,
    DurationSeconds: 900,
    Policy: JSON.stringify({                       // session policy: intersection, never a widening
      Version: "2012-10-17",
      Statement: [{
        Effect: "Allow",
        Action: ["s3vectors:QueryVectors", "s3vectors:GetVectors"],
        Resource: indexArn(tenant),                // exactly one ARN
      }],
    }),
  }));
  return res.Credentials!;
}

A session policy can only narrow a role and never grant beyond it. The credentials last a few minutes and never leave the API. At scale, attribute-based access control achieves the same with a tag on the index matched against a principal tag, so the policy doesn't grow along with the tenant list.

Next comes the query itself, with the intra-tenant control expressed as a filter:

import { QueryVectorsCommand } from "@aws-sdk/client-s3vectors";

async function retrieve(tenant: string, role: Role, question: string) {
  const client = new S3VectorsClient({ region: "eu-west-1", credentials: await tenantCredentials(tenant) });
  const res = await client.send(new QueryVectorsCommand({
    indexArn: indexArn(tenant),
    queryVector: { float32: await embed(question) },
    topK: 10,
    filter: { classification: { $in: allowedClassifications(role) } },   // evaluated during search
    returnMetadata: true,
    returnDistance: true,
  }));
  return res.vectors ?? [];    // each: key, distance, metadata incl. chunk_text
}

So there are two boundaries and two mechanisms. IAM on the ARN decides which tenant you can reach. A metadata filter, applied within the search itself and not afterwards, decides which classification you see inside that tenant. Filtering afterwards, meaning fetch ten and throw away what the user can't see, quietly wrecks recall for restricted roles and never says so. Filtering inside the search gives back ten results the user is allowed to see. The allowed classifications are computed from the authorizer's claims and never from the request body.

Grounding and citations fall out of the shape of the response. The model is shown only the chunk_text of the returned vectors, and each citation is the document id and version read from that same metadata. What gets cited is exactly what the model read, since both are the same bytes.

The deliberate bug

There's a test that separates a design where the application holds the boundary from one where the platform does. Point the application at the wrong index while it holds tenant-scoped credentials:

const client = new S3VectorsClient({ region: "eu-west-1", credentials: await tenantCredentials("b") });
await client.send(new QueryVectorsCommand({ indexArn: indexArn("a"), queryVector: { float32: q }, topK: 10 }));
// → AccessDeniedException: User ... is not authorized to perform: s3vectors:QueryVectors on resource: .../index/org-a

Nothing comes back. In a design that relies on application filtering, the same bug would return A's data with a 200. Keep this as an integration test, since it's the cheapest evidence you have that the boundary isn't in your code. The refused call also appears in the audit log with errorCode: AccessDenied against index/org-a.

Erasure by key, verified by key

Deletion uses the manifest written at ingest, and no search is involved.

import { DeleteVectorsCommand, GetVectorsCommand } from "@aws-sdk/client-s3vectors";

async function eraseDocument(tenant: string, docId: string, version: number) {
  const keys = await manifest.get(tenant, docId, version);
  for (const batch of chunk(keys, 500)) {
    await s3v.send(new DeleteVectorsCommand({ vectorBucketName: "kb-eu-west-1", indexName: `org-${tenant}`, keys: batch }));
  }
  // verify — strongly consistent, so this is valid immediately
  for (const batch of chunk(keys, 100)) {
    const res = await s3v.send(new GetVectorsCommand({ vectorBucketName: "kb-eu-west-1", indexName: `org-${tenant}`, keys: batch }));
    if ((res.vectors ?? []).length) throw new Error(`erasure incomplete: ${res.vectors!.length} keys remain`);
  }
  await audit.record({ tenant, docId, version, keyCount: keys.length, verifiedAt: new Date() });
}

An auditor can rerun the second loop with her own read-only credentials limited to that tenant's index ARN. The call is identical, only the principal differs, and the result comes from AWS instead of a screenshot supplied by the vendor. Calling GetVectors against another tenant's index returns AccessDenied, which is the isolation test once more.

A whole tenant leaving is just DeleteIndex. Two operational notes. A deleted vector drops out of query results right away, though it may still count toward index size, and so scan billing, for as long as a day. Also, when a new document version is ingested, the previous version's keys should be removed the old version's keys through this same path and not through a separate cleanup job.

Audit: CloudTrail data events by ARN

Index creation and deletion (CreateIndex, DeleteIndex) are management events and CloudTrail records them by default. Records of individual queries are data events: they must be switched on, and each one is billed. Skip that step and any claim of auditability is hollow.

// CDK
new cloudtrail.Trail(this, "Trail", { sendToCloudWatchLogs: false })
  .addEventSelector(cloudtrail.DataResourceType.S3_VECTORS_INDEX, [
    `arn:aws:s3vectors:eu-west-1:${account}:bucket/kb-eu-west-1/index/*`,
  ]);
// Check the CDK enum/name for the S3 Vectors index resource type; the underlying resource type is AWS::S3Vectors::Index.

A query event, trimmed down:

{
  "eventSource": "s3vectors.amazonaws.com",
  "eventName": "QueryVectors",
  "eventTime": "2026-09-09T10:41:07Z",
  "userIdentity": { "type": "AssumedRole", "arn": "arn:aws:sts::123456789012:assumed-role/kb-query/q-b" },
  "resources": [{ "type": "AWS::S3Vectors::Index",
                  "ARN": "arn:aws:s3vectors:eu-west-1:123456789012:bucket/kb-eu-west-1/index/org-b" }],
  "errorCode": null
}

"Who read tenant A's data?" becomes one predicate in CloudTrail Lake or Athena:

SELECT eventTime, eventName, userIdentity.arn, errorCode
FROM   cloudtrail_events
WHERE  eventSource = 's3vectors.amazonaws.com'
  AND  element_at(resources, 1).arn LIKE '%/index/org-a'
ORDER  BY eventTime DESC;

AWS produces the record the moment the call arrives, ahead of the application's own handler, and the platform operator has no way to edit it. Hand the auditor read access to the trail, or export that tenant's events, and she can answer the question unaided. That is what changes for the audit requirement: the auditor verifies, and nobody has to take the vendor's word.

Four additions that fit the model

One KMS key per index. Vector buckets can use SSE-KMS, so each tenant's index can be encrypted with a customer-managed key dedicated to that tenant. Every key carries an ARN and a region, which is a second residency artefact. Disabling it leaves everything encrypted under it unreadable, copies included, a firmer deletion than DeleteIndex offers alone.

Export. ListVectors can stream a whole index, text that isn't filterable included, across as many as 16 parallel segments. With credentials scoped to its own ARN, a tenant can walk out with its complete knowledge base. That covers GDPR Article 20 portability and removes lock-in with one call.

A private network path. Send S3 Vectors traffic over a VPC endpoint so queries don't cross the public internet. Security officers at tenants subject to NIS2 or a sector security code will ask exactly this.

Cost per tenant. With cost-allocation tags on every index, the vector layer shows up per tenant on the AWS bill as a line item and not as a modelled estimate. The same tags make the ABAC form of the session policy possible.

What the index boundary doesn't solve

All of the above answers which tenant can reach which data, and AWS enforces it. The following items aren't covered by that boundary, and they're where most of the engineering time actually went.

Roles inside a tenant. The classification filter covers it. Two rules matter: apply the filter inside the query and not after it, and take the role's rank from the authorizer, never from the client.

Existence leakage. Public-sector teams hit a failure mode that others rarely meet: a model must not even reveal that a restricted document exists, because "I can't show you the child-welfare report on this family" is a disclosure in itself. So retrieval never hands back a chunk the user can't see, which the filter guarantees, and the prompt never hints at anything held back. The model answers from what it has or admits it has nothing. This sits in retrieval and not in the prompt because retrieval can be tested and the model's phrasing can't.

Cross-language similarity. A corpus that mixes a local language with English legal text gets noticeably different scores from a single embedding model for passages that are equally relevant. A fixed cosine cutoff tuned on one language then drops good results in the other without any warning. The remedy was swapping the absolute threshold for a window relative to the top match in each result set. Multilingual tenants should measure this first, because the right value depends on the corpus.

No keyword search. Identifiers like section numbers or case ids should go into filterable metadata and be matched exactly, not embedded and left to chance. If real keyword search is needed, put OpenSearch in front and let S3 Vectors act as the storage engine.

Chunk boundaries. The least glamorous finding: a fortnight spent on where documents are split improved answer quality more than months of retrieval architecture. Chunk on document structure, meaning sections and clauses, and not on token counts alone.

Changing the embedding model. The dimension is fixed when the index is created, so switching models means creating another index for each tenant. Recompute embeddings from the original documents still in S3, switch the configuration, then run DeleteIndex on the old index. Thanks to the per-tenant layout it rolls out gradually, and the delete itself shows the old copy is gone.

Bedrock Knowledge Bases. It can store vectors in S3 Vectors and is a sound default. Running your own ingestion pays off for one narrow reason: Knowledge Bases picks the vector keys itself, and the erasure proof described earlier depends on you picking them.

Requirements revisited

Five of the six requirements are a result of where data lives and who can address it, not of code the team wrote. That was the aim, and it's why such a platform can safely be offered to someone who doesn't trust the vendor.

The code here is illustrative. Check it against the current AWS SDK v3 and CDK, because the CloudTrail event shown is abbreviated, and confirm VPC endpoint support and index tagging for S3 Vectors in the latest documentation. Tenants, documents, regions and account ids are placeholders, and no production volumes or evaluation thresholds appear.