Plan and Manage an Azure AI Solution
This domain accounts for 28% of the AI-103 exam and tests your ability to select the right Foundry services for a given scenario, architect solutions correctly, apply responsible AI principles in practice, control costs, and manage, monitor, secure and govern AI workloads once deployed. Questions are scenario-driven: you are given a business requirement and must identify the correct service, tier, architecture pattern, or policy setting. Precision matters — many distractors differ by only one configuration option or service tier.
Azure AI Foundry Projects, Hubs, and Resource Planning
Core Concepts
Azure AI Foundry (formerly Azure Machine Learning Studio + Azure AI Studio unified) is the platform for building, deploying, and managing AI solutions on Azure. Understanding its two-level hierarchy is essential.
| Concept | What it is | Scope | Key facts |
|---|---|---|---|
| AI Foundry Hub | Top-level governance container | Subscription / resource group | Holds shared connections, compute, storage, Key Vault, ACR. One hub can support many projects. |
| AI Foundry Project | Working environment for a team/use case | Within a hub | Inherits hub resources. Has its own deployments, indexes, evaluations. |
| Hub-linked resources | Storage account, Key Vault, Azure Container Registry, Application Insights | Created at hub provisioning | Shared across all projects in the hub. |
| Connection | Credential-bearing pointer to an external service | Hub or project level | Hub-level connections are shared; project-level connections are isolated. |
Resource planning rules:
- One hub per organization or business unit sharing governance.
- One project per team, product, or experiment that needs isolated deployments or data.
- Compute clusters and serverless compute are attached at the hub level.
- Model deployments (e.g., Azure OpenAI endpoints) are configured at the project level.
Common traps:
- Confusing a hub with a project — hubs are governance/shared-resource containers; projects are where actual work happens.
- Thinking each project needs its own storage account — storage is shared from the hub.
- Assuming you need separate Azure subscriptions for team isolation — projects within one hub provide sufficient isolation.
- Forgetting that deleting a hub deletes all associated projects and shared resources.
Worked scenario: A bank has three AI teams (fraud, loans, customer service) that must share a GPU cluster but keep their model deployments and prompt flows isolated. → Create one hub (attaches shared compute), create three projects (one per team, isolated deployments).
Model Selection: Azure OpenAI Models vs Open Models in the Catalog
Decision Framework
Bold the decision criteria: latency requirements, customization depth, data sovereignty, cost, and whether the task needs a flagship reasoning model or a smaller, cheaper model.
Azure AI model catalog contains models from multiple providers (OpenAI, Meta, Mistral, Cohere, Hugging Face, etc.) deployable as Managed Online Endpoints or via Serverless API (pay-per-token, MaaS).
| Dimension | Azure OpenAI Service | Open/Partner Models (Model Catalog) |
|---|---|---|
| Models available | GPT-4o, GPT-4o mini, GPT-4 Turbo, o1, o3, DALL-E 3, Whisper, text-embedding-3 | Llama 3, Mistral, Phi-3/Phi-4, Cohere Command R+, etc. |
| Deployment type | Standard, Provisioned (PTU) | Serverless API (MaaS) or Managed Compute |
| Data processing | Your Azure tenant; Microsoft manages infra | MaaS: hosted by provider in Azure. Managed: your VNet |
| Fine-tuning | Supported (GPT-4o, GPT-3.5 Turbo) | Supported for select models (Llama, Phi) via AI Foundry |
| Content filtering | Azure AI Content Safety integrated by default | Varies; MaaS models may have provider-side filters |
| SLA / enterprise support | Full Microsoft SLA | Depends on model tier |
| Best for | Enterprise, compliance-heavy, need GPT-4 class | Cost optimization, open-source flexibility, on-prem portability |
Phi-3 / Phi-4 models (Microsoft Small Language Models) are optimized for edge/on-device scenarios and low-cost inference — expect exam questions contrasting them with GPT-4o.
Common traps:
- Assuming Azure OpenAI is the only way to deploy LLMs on Azure — the model catalog supports 1,700+ models.
- Confusing Serverless API (MaaS) with a managed compute deployment — MaaS is pay-per-token with no infrastructure management; managed compute requires provisioning an endpoint.
- Thinking fine-tuning is only available for Azure OpenAI models — Llama 3 and Phi models support fine-tuning in AI Foundry.
- Selecting GPT-4o when a question emphasizes "lowest cost, summarization of structured data" — Phi-3 small or GPT-4o mini is the correct answer.
Worked scenario: A startup needs multilingual text classification, must minimize cost, and the model will run on a low-memory IoT gateway. → Deploy Phi-3 Mini from the model catalog as a managed endpoint or use its ONNX version for edge inference. GPT-4o is incorrect (cloud-only, high cost).
Sign in free to read the full guide
8 more sections (about 5,770 words) continue below. The first domain's study guide is free for every AI-103 learner with a free account.