Microsoft AI-300: Optimize generative AI systems and model performance Practice Questions

Monitoring production outputs (E) is essential for detecting drift as user behavior, language, and HR policies evolve over time. Periodic retraining with new data (B) ensures the model stays current with policy changes and new examples. Redeploying the same model indefinitely (D) ignores drift and policy changes, degrading quality over time. Deleting the fine-tuning dataset (A) prevents future retraining and auditing — the dataset should be archived securely, not deleted. Increasing temperature (C) makes outputs less consistent and is unrelated to maintaining fine-tuned model quality.

Option A (grounding verification) is critical because LLM-generated answers can hallucinate policy terms, which would teach the fine-tuned model incorrect facts. Option E (domain expert review) catches domain-specific inaccuracies an automated check might miss. Together, these ensure factual accuracy. Option D is not a general requirement — most QA datasets do not need 50% refusals, and forcing this ratio would distort the model's behavior. Option B (padding to uniform length) is unnecessary and can actually harm training by introducing meaningless tokens. Option C (deduplication) is a good practice but is secondary to factual accuracy for preventing model degradation from wrong information.

Conditioning generation on approved source excerpts (A) ensures outputs are grounded in approved content. Metadata linking to source documents (B) provides traceability required by compliance. Factual consistency checking (C) validates that no fabricated details slipped through generation. Using parametric knowledge without grounding (D) is exactly what compliance wants to prevent — fabricated policy details would appear. Shuffling to remove source links (E) would destroy traceability, violating the requirement. Human review alone (F) is valuable but is not listed among the three most systematic, scalable practices; the three selected together provide automated + structural compliance.

Option F (held-out evaluation against accuracy thresholds) satisfies the accuracy requirement with a controlled pre-deployment gate. Option C (content safety evaluation with adversarial prompts) satisfies the safety/content policy requirement by stress-testing the model before it serves real users. Option E (documented rollback plan) satisfies the third requirement by ensuring the team can revert quickly if production issues arise. Option B (relying on production user feedback) is insufficient for a regulated application — accuracy must be validated before deployment, not after. Option D (10x dataset retraining before every deployment) is not a deployment lifecycle practice; it is a training decision and would be prohibitively expensive. Option A (sole endpoint immediately) eliminates the ability to perform safe staged rollouts and removes the option for easy rollback.

Groundedness (also called faithfulness) specifically measures whether each claim in the generated answer is supported by the retrieved passages—directly capturing the relationship between the retrieved context and the output. Option A (answer relevance) measures alignment between the answer and the query, not between the answer and retrieved chunks. Option C (context recall) measures coverage of the retrieved chunks relative to a ground-truth answer, not whether the model used them faithfully. Option D (BLEU) is a surface-form n-gram metric that does not assess source attribution or factual grounding at all.