Design and Implement a GenAIOps Infrastructure
This domain accounts for 24% of the AI-300 exam — the second-largest weighted domain, after Implement machine learning model lifecycle and operations. It tests your ability to architect, deploy, secure, and operate production-grade generative AI workloads on Azure using Azure AI Foundry (formerly Azure AI Studio). Questions focus on infrastructure decisions: which resource type to use, how to secure it, how to deploy models cost-effectively, and how to manage prompts and throughput at scale. Expect scenario-based questions where you must choose between confusable options such as serverless vs. managed compute, or system-assigned vs. user-assigned managed identities.
Foundry Resources and Project Environments
Core Concepts
Azure AI Foundry is organized around two hierarchical resource types:
| Resource | Azure Resource Type | Purpose | Scope |
|---|---|---|---|
| AI Foundry Hub | Microsoft.MachineLearningServices/workspaces (kind: Hub) | Shared governance layer; hosts connections, security config, shared compute | Organization / department |
| AI Foundry Project | Microsoft.MachineLearningServices/workspaces (kind: Project) | Workload isolation unit; developers deploy models and run evaluations here | Team / application |
A Hub is always required before creating a Project. Projects inherit the Hub's network settings, managed identity configuration, and shared connections (such as Azure OpenAI, Azure AI Search, and storage). A single Hub can host multiple Projects, but a Project belongs to exactly one Hub.
Associated resources automatically provisioned with a Hub include:
- Azure Storage Account — artifact and data storage
- Azure Key Vault — secret management
- Azure Container Registry (optional, provisioned on demand) — for custom environments
- Application Insights (optional) — for monitoring
Tiers and Quotas
- Foundry Hubs and Projects are free resources (no per-resource charge); you pay for the compute, models, and services they orchestrate.
- There is no hard limit on the number of Projects per Hub enforced by the platform, but governance best practice is to align Projects to application boundaries.
- Connections defined at Hub scope are visible and usable by all child Projects. Connections defined at Project scope are private to that Project.
Common Traps
- The exam will offer Azure Machine Learning workspace as a distractor. AML workspaces are not AI Foundry Hubs — they share the same ARM resource type but differ in
kind. AI-300 questions about GenAI infrastructure expect Foundry Hub/Project answers. - Deleting a Hub deletes all child Projects and their deployments. The exam tests awareness that Hub is the parent lifecycle owner.
- Shared connections are configured on the Hub, not replicated to each Project individually.
Worked Scenario
Requirement: A company has three development teams building different copilot applications. They want shared governance and network policy, but isolated model deployments per team.
Answer: Create one AI Foundry Hub with the shared network and identity configuration. Create three AI Foundry Projects (one per team) under that Hub. Each Project gets its own deployment namespace while inheriting Hub security settings.
Managed Identities and RBAC for GenAI Workloads
Managed Identity Types
| Type | Identity lifecycle | Use case |
|---|---|---|
| System-assigned managed identity | Tied to the resource; deleted with the resource | Single-resource access; simpler lifecycle |
| User-assigned managed identity | Independent lifecycle; reusable | Cross-resource sharing; pre-provisioning access before resource creation |
AI Foundry Hubs support both identity types. For GenAIOps, user-assigned managed identities (UAMI) are preferred in production because infrastructure-as-code pipelines can grant permissions to the UAMI before the Hub exists.
Key RBAC Roles for AI Foundry
| Role | Scope typically assigned | Permissions |
|---|---|---|
| Azure AI Developer | Project | Create deployments, run prompts, manage evaluations |
| Azure AI Inference Deployment Operator | Project | Deploy models to endpoints only |
| Azure AI Administrator | Hub | Full Hub management including connections |
| Cognitive Services OpenAI User | Azure OpenAI resource | Call inference endpoints |
| Cognitive Services OpenAI Contributor | Azure OpenAI resource | Manage fine-tuned models and deployments |
| Storage Blob Data Contributor | Storage Account | Read/write artifacts; required for evaluation jobs |
Decision rule: Assign Azure AI Developer to human developers working within a Project. Assign Cognitive Services OpenAI User to the Project's managed identity so the Project can call Azure OpenAI endpoints without storing API keys.
Common Traps
- Owner or Contributor roles on the Hub do not grant inference access on the underlying Azure OpenAI resource — you must separately assign Cognitive Services OpenAI User on that resource.
- The exam distinguishes between Cognitive Services OpenAI User (inference only) and Cognitive Services OpenAI Contributor (management plane, including fine-tuning). Using Contributor for a runtime service principal violates least privilege.
- System-assigned identity is deleted if you delete and recreate the resource; any RBAC role assignments must be redone. The exam uses this as a trap in disaster recovery scenarios.
Unlock with a paid plan
9 more sections (about 2,698 words) continue below. Every paid plan unlocks all AI-300 study guides, the full question bank and mock exams, from $4.97/month.