Machine Learning Operations Engineer Associate · 29% of the exam

Implement machine learning model lifecycle and operations: free practice questions

5 sample questions from our 68-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Microsoft exam.

1. A team runs an Azure Machine Learning training pipeline nightly. Recently, a pipeline step that reads raw data from Azure Data Lake fails intermittently with transient network errors, causing the entire pipeline to restart from scratch. The team wants the pipeline to retry only the failed step without re-executing successful upstream steps. Which pipeline configuration option addresses this?

  • A. Enable the 'reuse' (allow_reuse=True) setting on all steps so Azure ML caches completed step outputs and skips them on retry.✓ Correct
  • B. Wrap the data-loading step in a try-except block and log the error to MLflow so the pipeline continues without that step's output.
  • C. Configure the pipeline to run in 'debug' mode, which automatically retries failed steps up to three times.
  • D. Split the pipeline into two separate pipelines: one for data preparation and one for training, and trigger the training pipeline only after the data pipeline succeeds.
Explanation

Setting allow_reuse=True on pipeline steps causes Azure ML to cache the output of each step keyed to its inputs and code. On a re-run (after fixing or retrying the failed step), all steps whose inputs and code are unchanged will be skipped and their cached outputs reused, so only the failed step re-executes. Option B would suppress the error and propagate incomplete data downstream, corrupting results. Option C is incorrect: Azure ML pipelines do not have a 'debug' mode that provides automatic step-level retry. Option D avoids the problem partially but introduces operational complexity and does not solve transient failures within a single pipeline run.

2. A financial services company uses Azure Machine Learning Automated ML to train a credit risk model. The target column is highly imbalanced: 95% of records are non-default and 5% are default. The team notices that Automated ML selects accuracy as the primary metric by default and the winning model achieves 95% accuracy by predicting non-default for every record. Which configuration change should the data scientist make to ensure Automated ML selects a model that meaningfully distinguishes defaulters from non-defaulters?

  • A. Change the primary metric to AUC_weighted or average_precision_score_weighted✓ Correct
  • B. Enable the class_weight='balanced' flag in the AutoML featurization configuration
  • C. Increase the experiment_timeout_minutes to allow more model iterations
  • D. Set enable_early_stopping=False so all models complete full training
Explanation

Changing the primary metric to AUC_weighted or average_precision_score_weighted ensures that Automated ML optimizes for a metric that is sensitive to the minority class performance, unlike accuracy which is dominated by the majority class. A model that predicts only non-default will score poorly on AUC or average precision. Enabling class_weight='balanced' in featurization affects how some learners handle class imbalance, but it does not fix the metric selection problem—accuracy will still reward the trivial classifier. Increasing experiment_timeout_minutes allows more iterations but does not change the optimization objective, so Automated ML may still select the trivially accurate model. Setting enable_early_stopping=False prevents early termination of individual model training but does not address the flawed metric choice.

3. A data scientist is designing an Azure Machine Learning pipeline that includes a Responsible AI component to evaluate a text classification model. The team needs to assess three aspects: (1) model fairness across language subgroups, (2) feature importance for individual predictions, and (3) systematic identification of input segments where the model fails most often. Which THREE Responsible AI dashboard components should be added to address all three requirements?

  • A. Fairness assessment✓ Correct
  • B. Interpretability (feature importance)✓ Correct
  • C. Error analysis✓ Correct
  • D. Counterfactual analysis
  • E. Data explorer
  • F. Causal analysis
Explanation

Fairness assessment measures performance disparities across demographic or categorical subgroups (here, language subgroups), satisfying requirement 1. Interpretability (feature importance) surfaces global and local feature attribution scores, explaining which input features drive individual predictions, satisfying requirement 2. Error analysis identifies cohorts and input segments with the highest error rates, helping engineers find systematic failure modes, satisfying requirement 3. Counterfactual analysis generates what-if perturbations for individual instances to explain prediction boundaries but does not address group fairness or systematic error patterns. Data explorer provides statistical summaries and distributions of the dataset but is not an evaluation component for the three stated requirements. Causal analysis estimates causal treatment effects of features on outcomes, which is relevant for intervention planning but does not address fairness, feature attribution for predictions, or error cohort identification.

4. A company stores features for a customer churn model in an Azure Machine Learning feature store. A training pipeline retrieves a feature set using a point-in-time join to avoid data leakage. After retraining, the deployed model begins returning unexpectedly low confidence scores. Investigation reveals that the online feature retrieval spec used at inference time references a different feature version than the one used during training. What is the most direct remediation?

  • A. Retrain the model using the same feature version that is served at inference time, and update the model's metadata to pin the feature retrieval spec to that version.✓ Correct
  • B. Delete the older feature version from the feature store so that all consumers are forced to use the latest version.
  • C. Redeploy the model without a feature retrieval spec and compute all features inline in the scoring script.
  • D. Enable feature backfill for the production feature set so both versions produce identical values.
Explanation

The root cause is a training–serving skew caused by mismatched feature versions. The correct fix is to align the training and serving environments by either retraining with the inference version or updating the inference spec to match the training version, and then pinning the spec in the model's metadata so they remain consistent. Option B is destructive and may break other consumers that depend on the older version. Option C removes the feature store entirely, re-introduces manual feature engineering in the scoring script, and recreates the risk of skew. Option D (backfill) addresses data availability, not version mismatches in feature definitions or transformations.

5. An organization is configuring retraining triggers for a product recommendation model in Azure Machine Learning. The ML team has defined three potential trigger conditions. Which TWO trigger types are natively supported as automated retraining triggers in Azure Machine Learning model monitoring?

  • A. A data drift metric exceeding a configured threshold✓ Correct
  • B. A prediction drift metric exceeding a configured threshold✓ Correct
  • C. A scheduled calendar interval (e.g., every Monday at 02:00 UTC)
  • D. A manual approval gate from a human reviewer in Azure DevOps
  • E. The number of inference requests per day exceeding a defined volume limit
Explanation

Azure Machine Learning model monitoring natively supports alert-based retraining triggers when monitored signals—such as data drift or prediction drift—exceed configured thresholds. Both data drift and prediction drift signals can fire Azure Monitor alerts that can be connected to retraining pipelines via webhooks or event-based triggers. A scheduled calendar interval is a valid retraining trigger, but it is implemented through Azure ML pipeline schedules or Azure Data Factory, not through the model monitoring system itself. A manual approval gate in Azure DevOps is a pipeline release management feature, not a monitoring-native retraining trigger. Request volume limits are not a built-in monitoring signal in Azure ML model monitors.

63 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free