Azure AI Foundry & Azure OpenAI
This domain accounts for 22% of the ai-intermediate exam — the single largest domain on the test. It covers the full lifecycle of building, deploying, and monitoring AI applications on Azure, including the organizational model of Azure AI Foundry, Azure OpenAI Service endpoints and quotas, Prompt Flow orchestration, content safety, managed endpoints, and RAG architectures using Azure AI Search. Expect scenario-based questions that require you to select the right service tier, endpoint type, or deployment configuration for a given requirement.
Azure AI Foundry: Hubs, Projects, Connections, Model Catalog
Core Organizational Model
Azure AI Foundry (formerly Azure AI Studio) is the unified portal and platform for building enterprise AI applications. It uses a two-level hierarchy:
| Level | Resource Type | Purpose | Scope |
|---|---|---|---|
| Hub | Microsoft.MachineLearning/workspaces (kind: Hub) | Shared infrastructure, governance, connections | Multiple teams/projects |
| Project | Microsoft.MachineLearning/workspaces (kind: Project) | Isolated dev/test workspace | Single team or use case |
- A hub owns the shared resources: Azure Storage, Azure Key Vault, Azure Container Registry, and connections to external services.
- A project inherits the hub's connections but has its own compute, experiments, and deployments.
- One hub can host many projects; a project belongs to exactly one hub.
- Hub connections are visible to all child projects. Project-level connections are private to that project.
Connections
Connections store credentials and endpoints for external services so they are not embedded in code. Common connection types:
| Connection Type | Typical Use |
|---|---|
| Azure OpenAI | Chat, completions, embeddings nodes in Prompt Flow |
| Azure AI Search | RAG pipelines, vectorization |
| Azure Blob Storage | Data assets, grounding data |
| Custom API key | Third-party models or services |
Connections can be set at hub level (shared) or project level (isolated). Credentials are stored in the hub's linked Azure Key Vault.
Model Catalog
The model catalog inside Azure AI Foundry aggregates models from multiple sources:
| Source | Examples | Deployment target |
|---|---|---|
| Azure OpenAI | GPT-4o, o1, DALL-E 3, text-embedding-3-large | Azure OpenAI Service |
| Hugging Face / open-source | Llama 3, Mistral, Phi-3 | Managed online endpoints |
| Azure AI model inference | Cohere, AI21 | Serverless API (pay-per-token) |
- Serverless API deployments require no quota; billing is per token.
- Managed compute deployments consume dedicated compute quota.
Common traps:
- The exam may say "Azure AI Studio" — this is now Azure AI Foundry (rebranded November 2023). Treat them as synonymous.
- Connections belong to the hub by default; you must explicitly scope them to a project to restrict access.
- Not all catalog models can be deployed to Azure OpenAI endpoints — open-source models go to managed online endpoints.
Azure OpenAI Service: Deployments, Quota, Model Versions
Deployments
In Azure OpenAI, a deployment is a named instance of a model that exposes an endpoint. You call the deployment name (not the model name) in API calls.
| Deployment Type | Description |
|---|---|
| Standard | Shared capacity, pay-per-token, subject to TPM/RPM quota |
| Provisioned Throughput Unit (PTU) | Reserved capacity, predictable latency, billed hourly per PTU |
| Global Standard | Routes to optimal Azure datacenter globally; lower cost than regional Standard |
| Global Provisioned | PTU-style reservation with global routing |
Quota
Quota is set per subscription + region and measured in:
- TPM — Tokens per minute (controls token throughput)
- RPM — Requests per minute (derived: 1000 TPM ≈ 6 RPM by default)
Key quota behaviors:
- Quota is allocated when you create a deployment, not when you call the API.
- Exceeding quota returns HTTP 429 (TooManyRequests).
- Quota can be requested to be increased via the Azure portal.
- PTU quota is separate from Standard TPM quota.
Key Model Versions (exam-relevant)
| Model | Key Capability | Notes |
|---|---|---|
| GPT-4o | Multimodal (text + image input), fast, cost-efficient | Default choice for most chat scenarios |
| GPT-4o mini | Lighter, cheaper version of GPT-4o | Use when cost matters and task is simpler |
| o1 / o1-mini | Chain-of-thought reasoning, complex math/code | No streaming support in initial release; higher latency |
| text-embedding-3-large | 3072-dimension embeddings | Highest quality; also available in 256/1024/3072 dim |
| text-embedding-3-small | 1536-dimension embeddings | Lower cost alternative |
| DALL-E 3 | Image generation | Requires dalle deployment type |
| Whisper | Speech-to-text transcription | Audio API endpoint |
Common traps:
- o1 models do not support system messages in the initial API version — use user-turn prompts only.
- Deployment name and model name are different; the API call uses the deployment name.
- PTU billing starts immediately on reservation regardless of usage.
GPT-4osupports image inputs;GPT-4(non-o) does not without the vision variant.
Sign in free to read the full guide
11 more sections (about 2,657 words) continue below. The first domain's study guide is free for every AI Intermediate learner with a free account.