Microsoft Foundry turns model API calls into governed AI operations
Microsoft Foundry turns one-off model API calls into governed operations: model catalogs, agents, RBAC, tracing, observability and evaluation in one place.
This article is published in English.
Sources: Why Use Microsoft Foundry: From Model API Calls to Enterprise AI Operations
From a model POST to enterprise AI operations
A single chat completion can answer a prompt. It will not inventory a large model catalog, drive multi-step agents, apply team-scoped access control, record every tool invocation, or gate releases on quality checks. The gulf between a one-off completion and a system you can operate at company scale is exactly what a unified Azure AI platform is meant to close.
Microsoft Foundry (and the surrounding Azure AI stack) aims to turn fragmented LLM experiments into governed, observable, production-grade systems. The value is not another SDK wrapper; it is the operating model: catalogs, deployments, identity, tracing, evaluation, and agent wiring under one roof.
What “calling the API” actually buys you
Direct provider calls are fine for prototypes:
| Dimension | Previous | Current |
| - - | - - | - - |
| Brand | Azure AI Studio / Azure AI Foundry | Microsoft Foundry |
| Brand | Azure AI Services | Foundry Tools |
| Agent API | Assistants API (Agents v0.5/v1) | Responses API (Agents v2) |
| API versioning | Monthly `api-version` params | v1 stable routes (`/openai/v1/`) |
| Resource model | Hub + Azure OpenAI + Azure AI Services | Foundry resource (single, with projects) |
| SDKs & endpoints | Multiple packages against 5+ endpoints | Unified project client (`azure-ai-projects` 2.x) + `OpenAI()` against one project endpoint |
| Surface | Best for |
| - - | - - |
| Foundry portal | Exploring models, prototyping prompts, building prompt agents without code. |
| SDKs (Python, C#, JavaScript, Java) | Building applications with `azure-ai-projects` 2.x and `OpenAI()` against one project endpoint. |
| Azure Developer CLI (azd) | Scaffolding, running, testing, and deploying hosted agent projects from the command line. |
| VS Code extension | Building and debugging agents in your editor. |
| Foundry Skill + MCP | Driving Foundry from coding agents (GitHub Copilot, Claude Code). |
| Dimension | Azure OpenAI / Raw API | Microsoft Foundry |
| - - | - - | - - |
| Scope | Model inference (GPT and OpenAI models) | Full AI lifecycle: models, agents, tools, eval, governance |
| Model catalog | OpenAI models only | 10,000+ models from multiple providers |
| Agent orchestration | Roll your own (LangChain, custom code) | Built-in prompt agents and hosted agents |
| Observability | Application Insights (manual setup) | Built-in tracing, evaluation, dashboards |
| Governance | Per-resource RBAC | Unified RBAC, Entra, content filters, Azure Policy |
| API stability | Monthly `api-version` changes | v1 stable routes (`/openai/v1/`) |
| Best for | Single-model, latency-critical, stable apps | Multi-model, agentic, governed, evolving AI products |
They fail production tests when you need:
- consistent deployment names across environments
- network isolation and private endpoints
- per-team quotas and cost attribution
- tool-call traces that security can audit
- evaluation gates before a prompt change reaches customers
Foundry’s operating layers
Think in layers rather than features:
- Model catalog & deployments — pick, version, and pin models instead of hardcoding base names in app code.
- Agent / orchestration surface — attach tools, knowledge sources, and instructions with a durable identity.
- Identity & RBAC — managed identities and least-privilege roles instead of leaked keys on laptops.
- Observability — traces, metrics, and content safety hooks that survive across services.
- Evaluation — offline and online checks so “it felt better in the demo” is not the release criteria.
When Foundry is the right move
Choose the platform path when multiple teams will share models, when data residency matters, when agents call tools that touch customer data, or when finance needs a single bill with attribution. Stay on raw APIs when you are still discovering product-market fit and the experiment count is one engineer and one notebook.
Migration habits that prevent pain
- Treat deployment name ≠ model name as a hard rule in every client.
- Put secrets in Key Vault / managed identity early—even for demos that “won’t last.”
- Version prompts and agent instructions next to code.
- Add a tiny golden-question suite before the first production traffic.
- Prefer private networking when legal already asked about egress.
Closing
Enterprise AI is an operations problem wearing a model costume. Foundry’s pitch is to make the operations half first-class: catalogs, agents, identity, traces, and evaluation. Use it when those concerns are already on the roadmap—not as a vanity platform for a hello-world chat box.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Operational checklist: name owners for model deployments, agent instructions, and evaluation datasets. Without owners, the catalog decays into an unmarked junk drawer of stale endpoints.
Add a weekly review of orphaned deployments and unused agent endpoints. Platform sprawl is how teams recreate the fragmented experiments Foundry was supposed to replace.
Add a weekly review of orphaned deployments and unused agent endpoints. Platform sprawl is how teams recreate the fragmented experiments Foundry was supposed to replace.