Developing AI Apps and Agents on Azure · 30% of the exam

Implement generative AI solutions: free practice questions

5 sample questions from our 75-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Microsoft exam.

1. A company fine-tuned a GPT-4o-mini model on 3,000 labeled examples to perform sentiment classification. After deployment, the model's accuracy on new production data drops significantly compared to validation accuracy. They have not changed the prompt or fine-tuning configuration. Which THREE factors are the most likely contributors to this degradation? (Choose THREE)

  • A. Distribution shift: production data has different vocabulary or topics than the fine-tuning dataset✓ Correct
  • B. The fine-tuning learning rate was too low, causing the model to underfit the training data
  • C. Overfitting to the fine-tuning dataset due to insufficient training data diversity or too many epochs✓ Correct
  • D. The base model was updated by Microsoft after fine-tuning, and the fine-tuned adapter is no longer compatible
  • E. The production prompts differ from the prompt format used during fine-tuning, breaking the learned pattern✓ Correct
  • F. The `max_tokens` parameter in the deployment is set too low, truncating output
Explanation

Distribution shift (A) is a primary cause of post-deployment accuracy drops when production data differs from training data. Overfitting (C) on small datasets causes the model to memorize training examples and generalize poorly. Prompt format mismatch (E) is a common fine-tuning pitfall—the model learns to respond to the exact prompt structure used during training, and deviations at inference time degrade performance. A low learning rate causing underfitting (B) would have shown poor validation accuracy during fine-tuning, which is not described here. Azure OpenAI fine-tuned model adapters are tied to a specific base model snapshot and are not silently invalidated by base model updates (D)—this is managed separately. A low `max_tokens` (F) would truncate completions but for a classification task returning a short label, this is unlikely to cause the observed accuracy drop.

2. An AI engineer is evaluating whether to use fine-tuning or RAG for a new internal knowledge base assistant at a financial services firm. The knowledge base is updated daily with new regulatory guidance. The firm also requires that source citations be provided for every answer. Which recommendation is MOST appropriate?

  • A. Use fine-tuning because it bakes knowledge directly into the model weights, eliminating the need for a retrieval layer and reducing latency.
  • B. Use RAG because it allows the model to retrieve up-to-date documents at inference time and can surface source metadata (e.g., document title, section) for citations without retraining.✓ Correct
  • C. Use fine-tuning because it is the only way to prevent the model from hallucinating facts not in the training set.
  • D. Use RAG only if the knowledge base contains fewer than 1,000 documents; otherwise, fine-tuning is required for scalability.
Explanation

RAG is the correct choice for daily-updated knowledge with citation requirements. RAG retrieves documents at inference time, so updates are reflected immediately without retraining. Retrieved chunk metadata (document name, page, section) can be passed through to the response for citations. Option A is wrong — fine-tuned knowledge is static and becomes outdated as the knowledge base changes; retraining is expensive and cannot happen daily. Fine-tuning also does not inherently produce citations. Option C is a misconception — fine-tuning does not eliminate hallucination; models can still hallucinate outside their training distribution. Option D is entirely fabricated; there is no such document count threshold governing the choice between RAG and fine-tuning.

3. A solutions architect is designing a RAG system for a legal firm. Documents are very long (up to 50 pages), and attorneys frequently ask questions that require synthesizing information from multiple distant sections of the same document. The team is considering three retrieval strategies: (A) dense vector search only, (B) BM25 keyword search only, (C) hybrid search with semantic re-ranking. Which strategy BEST addresses multi-section synthesis queries over long legal documents?

  • A. Strategy A — dense vector search captures semantic intent better than keywords for complex legal language.
  • B. Strategy B — BM25 is superior for legal documents because legal terminology is highly specific and keyword-based.
  • C. Strategy C — hybrid search retrieves candidates via both BM25 and vector similarity, and the semantic ranker re-orders by contextual relevance, maximizing the chance that all relevant sections surface across the top-K results.✓ Correct
  • D. None of the above; the only correct approach is to feed the entire document to the model's context window.
Explanation

Strategy C (hybrid + semantic ranker) is the most robust for complex, multi-section queries. Hybrid search ensures that both exact legal terminology (BM25) and semantic meaning (vector) are captured, while the semantic ranker re-orders the merged candidate set by contextual relevance. Option A (vector only) may miss exact legal terms or citations. Option B (BM25 only) fails on paraphrased or conceptual queries. Option D is impractical — 50-page documents can easily exceed even the largest context windows, and even when they fit, needle-in-a-haystack performance degrades with long contexts.

4. A developer is integrating Azure OpenAI tool calling into a travel booking assistant. The model returns a response with finish_reason equal to 'tool_calls' and a tool_calls array containing one entry. What is the correct sequence of steps the developer must follow next to produce a final natural-language response for the user?

  • A. Parse the function name and arguments from tool_calls, execute the function locally, append a new message with role 'tool' containing the function result, then call the chat completions API again.✓ Correct
  • B. Parse the function name and arguments from tool_calls, execute the function locally, append the result as a new user message, then call the chat completions API again.
  • C. Extract the content field from the assistant message—it already contains the final answer formatted using the tool schema.
  • D. Call the chat completions API again with the same messages array; the model will automatically execute the tool and return a final answer.
Explanation

When finish_reason is 'tool_calls', the model has decided to invoke a function but has not yet produced a final answer. The developer must: (1) extract the function name and JSON arguments from the tool_calls array; (2) execute the actual function locally or call an external service; (3) append a message with role='tool' (and the corresponding tool_call_id) containing the function's return value; (4) call the chat completions API again so the model can synthesize a final response. Appending the result as a user message is incorrect because the API requires the 'tool' role for function results to maintain proper conversation structure. The content field is typically null when finish_reason is 'tool_calls', so there is no final answer there. The API does not execute functions autonomously—the developer's code must perform this step.

5. A product team is deciding between prompt engineering with RAG versus fine-tuning to adapt an Azure OpenAI model for a new internal HR chatbot. The HR chatbot needs to answer questions using a 500-page employee handbook that is updated quarterly. Which approach is most appropriate, and why?

  • A. Fine-tuning, because the model needs to memorize the employee handbook content and fine-tuning embeds knowledge into model weights for faster retrieval
  • B. RAG with Azure AI Search, because the handbook is a discrete, updatable document corpus that can be re-indexed quarterly without retraining the model✓ Correct
  • C. Fine-tuning, because RAG cannot handle documents longer than the model's context window
  • D. Prompt engineering only (no retrieval), because the entire handbook can be placed in a single system prompt using a large context window model
Explanation

RAG is the correct choice here. The handbook is a discrete, authoritative document corpus that changes quarterly—RAG allows the index to be refreshed without retraining the model, keeping answers current at low cost and effort. Fine-tuning embeds knowledge into model weights during training; when the handbook is updated quarterly, the model would need to be retrained each time, which is expensive and slow. Fine-tuning is better for teaching style, format, or behavior—not for injecting frequently changing factual knowledge. RAG can handle large corpora far exceeding the context window via chunking and retrieval—it is not limited to context-window-sized documents. While large context models can accept long documents, placing a 500-page handbook in every system prompt is prohibitively expensive in tokens, slow, and impractical for production use.

70 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free