azure rag ai-search ai-foundry embeddings agents

Grounded PDF Q&A on Azure AI Search and Foundry agents

Index private course PDFs with embeddings, wire a Foundry agent to the vector index, and debug the auth and instruction gaps that make RAG look broken.

Publisher: REACTAPP.TOPAdded to this site Sep 26, 20269 min read2147 words

This article is published in English.

Sources: Building a Grounded RAG Agent on Azure AI Search and Azure AI Foundry

Introduction

A digital-forensics training org sits on years of PDF courseware—slides, labs, reference chapters. Analysts need one concept buried across dozens of decks. Keyword search across scattered files is slow and brittle. The desired product is plain-language Q&A answered from that corpus, not a generic textbook paraphrase.

That is the RAG shape: retrieve relevant chunks from a private document set and ground generation in them. This build stands the pattern up on Azure end to end—Azure AI Search indexing real PDFs with embeddings, plus an Azure AI Foundry agent that answers from the index rather than general knowledge.

It also became a debugging narrative. Almost nothing failed because the idea was wrong; failures came from small mismatches among resources, auth models, and default agent behavior. Those friction points are documented rather than smoothed away.

Understanding the architecture

Four Azure services own four jobs.

Blob Storage holds source documents—here, forensics PDFs; any private domain corpus fits the same shape.

Azure AI Search indexes and retrieves. A skillset chunks documents, calls an embedding model, and stores text plus vectors. At query time Search returns semantically similar chunks, not only keyword hits.

Azure AI Foundry hosts models and the agent: an embedding deployment for index/query vectorization and a chat deployment for answers. The agent wraps the chat model with a knowledge source and instructions.

Managed Identity and RBAC gate who can query what. Connections (Search→Blob, Search→embeddings, agent→index) should prefer scoped roles over embedded keys. This build did not reach a fully key-free chain; that gap is called out below.

Stage one: source documents

A private blob container (knowledge-docs) received the course PDFs. Access stayed private—no anonymous read—consistent with treating materials as sensitive.

Stage two: model deployments in Azure AI Foundry

Two deployments inside a Foundry project:

Chat (chat-deploy) for answers
Embeddings (embed-deploy, text-embedding-3-small) for document and query vectors

Confirm both exist before wiring Search skillsets or agents; missing deployments produce opaque indexing failures later.

Stage three: Azure AI Search and the vectorized index

Create a Search service, then an index whose fields carry chunk text, vector, and document metadata. Attach a skillset that:

Pulls blobs from knowledge-docs
Splits into overlapping chunks
Calls the embedding deployment
Writes vectors into the index

Indexers need permission to read blobs and to call the embedding endpoint. Role assignments that look “almost right” still fail at runtime—verify the exact resource IDs.

Web Api response status: 'NotFound'
DeploymentNotFound: The API deployment for this resource does not exist.

Web Api response status: 'Unauthorized' (401)
Access denied due to invalid subscription key or wrong API endpoint.

Access denied when connecting to the MCP server ... HTTP 403 Forbidden

You are an assistant that answers questions about digital forensics course material.
For every user question, you MUST first search the connected knowledge base before answering.
Only answer using information retrieved from the knowledge base.
If the knowledge base returns no relevant results, say so explicitly rather than answering from general knowledge.

Stage four: indexing failures, and what they meant

Indexing rarely fails with a single clear sentence. Common shapes in this build:

Skillset cannot reach the embedding deployment (auth or wrong endpoint)
Indexer schedule runs but documents stay at zero (container path or skillset reference)
Vectors write with wrong dimensions (model mismatch between index schema and deployment)

Treat each as a configuration bug, not a RAG-concept failure. Capture the indexer execution history before changing three settings at once.

Stage five: wiring the index to an agent

In Foundry, attach the Search index as a knowledge source for the agent. Instructions should require citing retrieved passages and refusing when the index returns nothing relevant. Test with a question whose answer exists only in one PDF page—success means a grounded quote, not a fluent guess.

Stage six: two behavioral gaps

Beyond configuration, two behavioral issues showed up:

Over-helpful fallbacks — the chat model answered from prior knowledge when retrieval was empty. Tighten instructions and, where available, groundedness checks so empty retrieval yields “not in the corpus.”
Auth inconsistency — some links still used keys while others used managed identity. Mixed auth makes rotations painful and hides which principal actually succeeded.

Neither is a portal checkbox; both are product decisions encoded in instructions and identity design.

What’s actually happening behind the scenes

A user question is embedded, Search returns top chunks, the agent prompt receives those chunks plus instructions, and the chat model generates an answer constrained (ideally) to that context. Blob storage never talks to the agent directly; Search is the retrieval boundary. Foundry is the orchestration and model host.

Challenges and lessons learned

Region and model availability must be checked before resource creation.
Embedding dimension in the index schema must match the chosen embedding model exactly.
RBAC errors look like empty indexes until you read indexer details.
Agent instructions are part of correctness, not decoration.
Document the auth model you actually shipped, including temporary keys.

Key takeaways

Azure RAG is a chain of small contracts: blob privacy, skillset→embed permissions, index schema↔model dimensions, agent knowledge attachment, and grounded answer policy. Get those right and the architecture is boring—in the good way. Miss one and you debug for days while believing “RAG doesn’t work.”

Operational tip: keep a one-page runbook that lists the exact resource names, deployment names (not base model names), index name, skillset name, and the managed identities or keys each hop uses. When an indexer fails at 02:00, that page beats archaeology through portal screenshots. Add a weekly job that asks three golden questions whose answers live on known pages and fails the build if groundedness regresses.

← All articles Send the brief