Microsoft AI-300: Design and implement a GenAIOps infrastructure Practice Questions

A serverless API endpoint billed per token is the most cost-effective option for a low-frequency, short-duration batch workload because you pay only for the tokens processed during the 30-minute window each day, with no idle compute charges. A dedicated GPU VM running continuously incurs 24/7 compute costs regardless of usage, which is wasteful for a 30-minute daily job. While managed compute with scale-to-zero could reduce costs, Azure AI Foundry managed compute does not natively support scale-to-zero for all deployment types, and cold-start latency makes it unsuitable; serverless endpoints avoid this entirely. A serverless endpoint with a reserved capacity commitment adds a fixed upfront cost that is only economical for sustained high-volume workloads, not intermittent daily batch jobs.

Provisioned throughput units (PTUs) reserve dedicated model capacity for the deployment, which eliminates the throttling caused by shared capacity limits and reduces the effective per-token cost for high-volume sustained workloads compared to pay-as-you-go serverless pricing. Switching to a serverless API endpoint does not resolve throttling because serverless endpoints are subject to tokens-per-minute quota limits and pay-as-you-go pricing, which is more expensive than PTUs at high consistent volumes. Deploying to managed compute with a larger VM increases infrastructure cost without directly addressing token throughput limits imposed by the model serving layer. Implementing retry logic handles failures gracefully but does not prevent throttling or reduce costs; it only masks the underlying capacity problem.

Running evaluation SDK tests against the new prompt on a fixed test dataset provides measurable quality signal before any deployment occurs. Deploying to staging and executing evaluation runs validates the prompt in a realistic environment that mirrors production. Blocking production deployment when metrics fall below thresholds is the safety gate that prevents regressions from reaching users. These three elements together form a complete validate-then-deploy pipeline. Auto-approving PRs based on line count bypasses human and automated quality review, which defeats the purpose of the evaluation pipeline. Deleting previous prompt versions from Git destroys version history needed for rollback and audit, violating version control best practices. Updating the production prompt reference via Azure CLI is a deployment step, not a validation step; while necessary, it is the final action after all gates pass and does not itself constitute a validation element.

Azure RBAC is additive: when a principal has multiple role assignments, their effective permissions are the union of all permissions granted across all assigned roles. The engineer therefore has read, computes/read, write, and delete permissions on the hub. It is incorrect that higher-privilege roles supersede lower ones; Azure RBAC does not have a precedence ordering between allow roles — all are combined. Similarly, write and delete do not replace or override read; they are simply added. The mention of a Deny assignment is a distractor; no deny assignment exists in this scenario, and even if one did, it would subtract permissions rather than require manual review.

Key Vault Secrets User grants only the get and list permissions needed to read secret values, satisfying least-privilege for a read-only runtime scenario. Key Vault Administrator grants full control including the ability to manage access policies and delete the vault, far exceeding what is needed. Key Vault Secrets Officer adds the ability to create, update, and delete secrets, which is more than read-only access requires. Key Vault Reader is an Azure Resource Manager role that allows viewing Key Vault metadata but does not grant data-plane access to read secret values.