Developing AI Apps and Agents on Azure · 25% of the exam

Optimize and monitor AI solutions: free practice questions

5 sample questions from our 63-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Microsoft exam.

1. A developer is instrumenting a multi-step Azure AI application that calls Azure OpenAI, performs a vector search, and then calls a secondary classification model. They need every step to appear as a linked span in a single distributed trace visible in Application Insights. Which SDK feature should they use?

A. Azure Monitor Diagnostic Settings with the 'AllLogs' category enabled
B. The OpenTelemetry-based tracing support in the Azure AI Inference SDK, configured with an OTLP exporter pointing to the Application Insights connection string✓ Correct
C. Azure AI Foundry's built-in evaluation runner, configured in 'trace' mode
D. The Azure OpenAI content filter audit log, exported to a Log Analytics workspace

Explanation

The Azure AI Inference SDK exposes OpenTelemetry-compatible tracing that creates spans for each AI call. By configuring an OTLP exporter (or the Azure Monitor OpenTelemetry distro) with the Application Insights connection string, all spans from all steps are correlated into a single distributed trace viewable in Application Insights. Option A (Diagnostic Settings) captures platform-level logs, not application-level spans for multi-step traces. Option C (evaluation runner) is for offline quality measurement, not real-time distributed tracing. Option D (audit log) records content filter decisions, not application span data.

2. An enterprise AI team has a strict latency SLA requiring Azure OpenAI responses to complete within 3 seconds for 95% of requests. Their current P95 latency is 7 seconds. The model deployment is gpt-4o with a max_tokens of 2,048. Analysis shows that time-to-first-token (TTFT) is 500ms but time-to-last-token (TTLT) is 6.8 seconds. What is the MOST effective single change to meet the SLA?

A. Enable response streaming and redefine the SLA measurement point as time-to-first-token (TTFT), presenting partial responses to users as they arrive.
B. Reduce max_tokens from 2,048 to 256 to significantly decrease the number of tokens generated per response.✓ Correct
C. Switch to a provisioned throughput (PTU) deployment to reduce queuing latency under load.
D. Move the Azure OpenAI deployment to a region geographically closer to the application servers to reduce network round-trip time.

Explanation

The data shows TTFT is only 500ms but TTLT is 6.8 seconds — indicating the bottleneck is token generation time, not queuing or network latency. Since generation time scales linearly with tokens produced, reducing max_tokens from 2,048 to 256 (an 8x reduction) would dramatically reduce TTLT. If the application genuinely only needs concise answers, this is the most impactful single change. Option A (streaming) changes user perception of latency but does not change TTLT — the total time to generate all tokens remains 6.8 seconds; redefining the SLA measurement point is an accounting trick, not a true SLA improvement. Option C (PTU) primarily addresses queuing/throttling latency (which manifests as high TTFT), but TTFT is already only 500ms here, so PTU would have minimal impact. Option D (region proximity) reduces network RTT by tens of milliseconds at most — far too small to close a 4-second gap.

3. An AI engineer is configuring Azure AI Content Safety for a children's educational platform. They need the strictest possible filtering across all harm categories. After setting all content filter categories to 'High' severity blocking, a content review finds that some borderline outputs still pass through. What is the MOST likely reason?

A. Content filters only apply to completion models, not chat models
B. The 'High' severity setting blocks only the most extreme content; lower-severity harmful content may still pass at that threshold configuration✓ Correct
C. Azure AI Content Safety requires a separate API call and is not integrated with Azure OpenAI by default
D. Content filters are disabled for custom fine-tuned models and must be re-enabled manually

Explanation

In Azure AI Content Safety, severity thresholds (Low, Medium, High) define the minimum severity level at which content is blocked. Setting filters to block at 'High' severity means only content scored as high severity is blocked—content scored as low or medium severity will still pass through. To catch borderline content, filters should be set to block at 'Low' severity. Content filters apply to both completion and chat models. Azure AI Content Safety is natively integrated with Azure OpenAI deployments. Content filters are applied to fine-tuned models as well, though some configurations may differ; they are not automatically disabled.

4. A development team wants to instrument their Azure AI application so that every LLM call, tool invocation, and retrieval step is captured as a structured trace for debugging slow responses. Which Azure service and feature combination should they enable?

A. Azure Monitor Metrics + diagnostic settings on the Azure OpenAI resource
B. Azure AI Foundry tracing + Application Insights integration✓ Correct
C. Azure Log Analytics + custom event tables
D. Microsoft Defender for Cloud + workload protection alerts

Explanation

Azure AI Foundry's built-in tracing captures span-level details for each step of an AI application (LLM calls, retrievals, tool calls) and forwards them to Application Insights for querying, visualization, and alerting — exactly matching this requirement. Azure Monitor Metrics provides aggregate resource metrics but does not capture per-step application traces. Azure Log Analytics can store logs but requires custom instrumentation and does not natively understand AI application spans. Microsoft Defender for Cloud is a security posture tool and does not provide application-level tracing.

5. A content moderation team is configuring Azure AI Content Safety filters for a public-facing generative AI application. They need to ensure that user-submitted content referencing self-harm is blocked at the lowest detectable severity, while allowing mature (but non-explicit) violence references for a gaming context. Which filter configuration BEST meets these requirements?

A. Set 'Self-harm' filter threshold to 'Low' (block at severity 2+) and set 'Violence' filter threshold to 'Medium' (block at severity 4+).✓ Correct
B. Set 'Self-harm' filter threshold to 'Medium' and set 'Violence' filter threshold to 'Low' to balance safety with gaming context allowance.
C. Set 'Self-harm' filter threshold to 'High' to only block severe self-harm content and set 'Violence' to 'High' to allow most gaming violence.
D. Disable the 'Violence' filter entirely and set 'Self-harm' to 'Low', as disabling is the only way to allow mature violence content.

Explanation

Option A correctly maps the requirements: 'Low' threshold for Self-harm means the filter blocks at the lowest detectable severity level (severity 2 and above), providing maximum protection. 'Medium' threshold for Violence allows low-severity violent content (consistent with gaming references) while still blocking high-severity violent content. Option B inverts the settings — 'Medium' on self-harm would allow low-severity self-harm references through, violating the requirement, and 'Low' on violence would block gaming-appropriate content. Option C sets both to 'High', which would allow medium-severity self-harm through — directly violating the requirement for maximum self-harm protection. Option D is incorrect because disabling a filter entirely removes all protection for violence, which is unnecessary; the 'Medium' threshold already allows moderate violence while maintaining guardrails.

58 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free