Azure AI Apps and Agents Developer Associate · 28% of the exam

Plan and manage an Azure AI solution: free practice questions

5 sample questions from our 90-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Microsoft exam.

1. A startup is building a conversational AI assistant using Azure AI Foundry. The team wants to quickly prototype using a pre-trained large language model without any custom training data. The assistant must answer general business questions and summarize documents. Which approach best fits this scenario?

  • A. Fine-tune an Azure OpenAI GPT model on a proprietary dataset to improve domain specificity.
  • B. Use prompt engineering with a pre-trained Azure OpenAI model, crafting system prompts to guide behavior.✓ Correct
  • C. Train a custom neural network from scratch using Azure Machine Learning to avoid licensing costs.
  • D. Use Retrieval-Augmented Generation (RAG) with a vector index before any other approach to ensure accuracy.
Explanation

Prompt engineering with a pre-trained model is the fastest, lowest-cost starting point when no custom training data exists and the task is general-purpose. It requires no training pipeline or labeled data. Fine-tuning (A) requires labeled examples and additional cost, which is unnecessary here. Training from scratch (C) is impractical for a startup with no training infrastructure or data. RAG (D) adds architectural complexity and requires a document index; it is valuable but not the simplest first choice when the task is already general and no specific knowledge base is mandated.

2. An AI platform engineer wants to monitor token usage across all Azure OpenAI deployments in a subscription to attribute costs to individual product teams. They need a near-real-time dashboard broken down by deployment name and model. What is the BEST approach?

  • A. Query the Azure OpenAI resource's activity log for 'Microsoft.CognitiveServices/accounts/deployments/write' events and sum the payload sizes
  • B. Enable the 'Azure OpenAI Requests' diagnostic metric category in Diagnostic Settings, stream it to a Log Analytics workspace, and build a workbook that aggregates PromptTokens and CompletionTokens by deployment✓ Correct
  • C. Export the Azure subscription's cost analysis CSV monthly and filter by resource type 'Microsoft.CognitiveServices/accounts'
  • D. Use the Azure AI Foundry Evaluations dashboard, which automatically aggregates token usage across deployments
Explanation

Enabling diagnostic metrics for Azure OpenAI and streaming them to Log Analytics provides granular, near-real-time data including prompt and completion token counts per deployment. A Log Analytics workbook can aggregate and visualize this by deployment name. Option A (activity log) records management-plane operations, not data-plane token consumption. Option C (monthly cost CSV) is retrospective and not near-real-time or granular by deployment. Option D (Evaluations dashboard) is for quality metrics, not token usage attribution.

3. An AI engineer is configuring Azure AI Content Safety for a children's educational platform powered by Azure OpenAI. The platform must block all sexual content at any severity and block violent content only when it is graphically detailed (high severity). Which configuration correctly reflects these requirements?

  • A. Set Sexual content filter threshold to Low; set Violence content filter threshold to High.✓ Correct
  • B. Set Sexual content filter threshold to High; set Violence content filter threshold to Low.
  • C. Disable the Sexual content filter and enable only the Violence filter at Medium threshold.
  • D. Set both Sexual and Violence content filter thresholds to Low to maximize safety across all categories.
Explanation

In Azure AI Content Safety, a lower severity threshold means the filter triggers (blocks) content at lower severity levels, providing stricter filtering. Setting Sexual to Low blocks all sexual content (even mild). Setting Violence to High means only high-severity (graphic) violent content is blocked, allowing lower-severity content to pass—matching the requirement. Option B reverses the thresholds, which would allow low-severity sexual content (incorrect for a children's platform) and block even low-severity violence. Option C disables sexual filtering entirely, which is unacceptable. Option D sets both to Low, which would block even mild violence references—more restrictive than the stated requirement for violence.

4. A team running an Azure AI Foundry project wants to evaluate the relevance of responses from their RAG-based travel assistant. They have a test dataset of 200 question-answer pairs. The Azure AI Foundry relevance evaluator returns a score for each sample. Which description BEST matches what the relevance metric measures in Azure AI Foundry?

  • A. How well the generated response addresses the user's question, regardless of whether the answer is factually grounded in the retrieved documents.✓ Correct
  • B. The degree to which the retrieved context chunks are topically related to the user's query before generation.
  • C. The similarity between the model-generated answer and a human-authored reference answer using BLEU or ROUGE scoring.
  • D. Whether the response contains any harmful or policy-violating content as judged by the content safety classifier.
Explanation

In Azure AI Foundry's built-in evaluators, relevance measures how well the model's generated response answers the user's question — it is an answer-to-question relevance score, not a retrieval-to-query relevance score. Option B describes retrieval relevance (how well retrieved chunks match the query), which is a separate retrieval-quality concern and not what the Azure AI Foundry relevance evaluator scores. Option C describes reference-based metrics like BLEU/ROUGE, which Azure AI Foundry does offer separately but are distinct from the LLM-as-judge relevance metric. Option D describes content safety evaluation, which is a completely separate evaluator category.

5. An AI engineer working on an Azure AI Foundry project notices that the groundedness score for a RAG pipeline drops significantly when user queries are longer than 50 tokens. The retrieval component returns the top-3 chunks by cosine similarity. What is the MOST likely cause of the low groundedness?

  • A. The Azure OpenAI model version does not support inputs longer than 50 tokens
  • B. Long queries dilute the embedding vector, causing the retrieval step to return chunks that are less relevant to the core question, leading the model to hallucinate✓ Correct
  • C. The fluency metric is interfering with the groundedness calculation for long inputs
  • D. Application Insights is not capturing traces for queries over 50 tokens, skewing the evaluation dataset
Explanation

With longer, multi-intent queries, the single embedding vector becomes a blended representation, which reduces retrieval precision. The returned chunks may not cover the specific sub-topic the model needs, prompting it to generate content not grounded in the retrieved context. Option A is false; current Azure OpenAI models support far more than 50 tokens. Option C is incorrect because fluency and groundedness are independent metrics evaluated separately. Option D conflates observability tooling with evaluation scoring.

85 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free