Microsoft Designing and Implementing Multi-Agent AI Solutions (AI-500) · 24% practice allocation

Microsoft AI-500: Evaluate, optimize, and monitor multi-agent solutions Practice Questions

5 free sample questions from the Evaluate, optimize, and monitor multi-agent solutions domain, with answers and explanations. The full AI-500 bank has 52 questions across 4 domains. Questions are scenario-based and written from the official Microsoft exam guide.

1.

During an iterative multi-turn research session, an agent summarizes previous conversation turns repeatedly. By turn 20, the agent claims an event happened in 2026 that was never mentioned in the original documents, and it begins citing its own earlier hallucinations as factual history. What term describes this degradation?

  • A. Summary drift✓ Correct
  • B. Quantization error
  • C. Adversarial perturbation
  • D. Gradient clipping
Explanation

Option A is correct because 'summary drift' occurs when successive rounds of summarization compound subtle omissions, embellishments, or hallucinations over time, drifting further from the original source facts. Option B relates to weight precision reduction during model inference. Option C relates to intentional adversarial attacks. Option D is an optimization technique during model pre-training.

2.

An agent that generates multi-document executive summaries over large PDF disclosures (150,000 tokens) consistently overlooks critical financial footnotes located in the middle of the uploaded document, despite acknowledging points at the beginning and conclusion. Which known LLM characteristic is responsible?

  • A. Lost in the Middle phenomenon✓ Correct
  • B. Token bucket exhaustion
  • C. Prompt injection vulnerability
  • D. Reciprocal Rank Fusion degradation
Explanation

Option A is correct because the 'Lost in the Middle' phenomenon describes the well-documented tendency of attention mechanisms in large language models to prioritize tokens placed at the start (primacy) and end (recency) of long prompts, while suffering decreased recall for information buried in the middle. Option B relates to API rate limiting. Option C involves adversarial user manipulation. Option D is a search ranking fusion algorithm, not an LLM attention effect.

3.

An architect is evaluating an agent that assists software engineers in refactoring code. The evaluation team needs to assess whether the generated code compiles, passes unit tests, and satisfies all specified acceptance criteria. What evaluation approach should be used?

  • A. Deterministic sandbox execution testing (running compilers and test suites against the agent's code) alongside LLM-as-a-judge review of code maintainability.✓ Correct
  • B. Comparing the character count of the generated code against the original code.
  • C. Checking whether the code contains any vowels.
  • D. Relying solely on whether the user said 'thank you' at the end of the chat.
Explanation

Option A is correct because functional code generation requires deterministic execution verification (compilation, automated unit tests, linting) in an isolated sandbox, supplemented by qualitative LLM-as-a-judge rubrics for maintainability, documentation, and architectural best practices. Option B, C, and D are irrelevant metrics that do not measure software correctness.

4.

Users interacting with an autonomous multi-turn itinerary planner notice that after 15 conversation turns, the agent forgets dietary restrictions and hotel budget preferences specified in the very first turn. The agent continues operating normally otherwise. What is the root cause of this failure?

  • A. Sliding-window amnesia caused by truncating earlier messages from the prompt context.✓ Correct
  • B. The model's temperature parameter was set to 0.0.
  • C. The user's network connection dropped.
  • D. The agent's vector database index was deleted.
Explanation

Option A is correct because sliding-window amnesia occurs when context management discards older conversation turns to stay within token limits, purging initial user goals and constraints from working memory. Option B causes deterministic output, not amnesia. Option C would prevent requests from reaching the server entirely. Option D would fail RAG lookups across all turns, not just historical conversational constraints.

5.

An agent that aggregates quarterly sales reports periodically enters an infinite loop, attempting to re-query database views with slightly altered SQL queries whenever zero rows are returned. How should the system engineer configure the agent runtime in Azure AI Foundry to prevent cost overruns?

  • A. Enforce a maximum iteration step limit (max_iterations = 8) in the agent runtime configuration that terminates the loop and issues an informative user error if reached.✓ Correct
  • B. Remove all SQL queries from the database views.
  • C. Double the token price in Azure OpenAI settings.
  • D. Increase the HTTP connection timeout to 2 hours.
Explanation

Option A is correct because setting an explicit maximum step/iteration limit in the agent runtime provides a deterministic circuit-breaker that prevents runaway loops and token spend when the agent fails to find data. Option B destroys database functionality. Option C increases costs without solving the loop. Option D exacerbates resource lockups.

Test yourself across every AI-500 domain

The free AI-500 practice test mixes about 20 questions weighted like the exam. The full 52-question bank adds instant grading, flashcards and a timed mock exam.

Take the free AI-500 practice testPractice the full bank

Study this domain

Evaluate, optimize, and monitor multi-agent solutions study guide (AI-500) →

Other AI-500 domains

← All Microsoft Designing and Implementing Multi-Agent AI Solutions study resources