Implement machine learning model lifecycle and operations Study Guide — AI-300

Implement ML Model Lifecycle and Operations

This domain accounts for 29% of the AI-300 exam — the single largest domain. It tests your ability to orchestrate the full machine learning lifecycle inside Azure Machine Learning (Azure ML): from experiment tracking and pipeline authoring, through model registration and responsible AI checks, to deployment, monitoring, drift detection, and automated retraining. Expect scenario-based questions where you must choose the right SDK v2 construct, CLI v2 command, or Azure ML Studio feature for a given operational requirement.

MLflow Experiment Tracking, Automated ML, and Notebooks

MLflow in Azure ML

MLflow is the open-source tracking library natively integrated into Azure ML. Every Azure ML workspace exposes a tracking URI so MLflow calls are automatically associated with workspace experiments.

Key facts the exam tests:

• mlflow.set_experiment("name") creates or reuses an experiment by name.
• mlflow.start_run() opens a run context; all mlflow.log_param(), mlflow.log_metric(), and mlflow.log_artifact() calls inside it write to Azure ML.
• Autologging (mlflow.autolog()) captures params, metrics, and model artifacts without explicit calls. Supported frameworks: scikit-learn, PyTorch, TensorFlow/Keras, XGBoost, LightGBM, Spark MLlib.
• Azure ML stores the MLflow model format under the run's outputs/mlflow-model/ path, making it directly deployable without conversion.
• The tracking URI is retrieved with mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri()) in SDK v1, or is set automatically when a job script runs inside an Azure ML compute context.

Automated ML (AutoML)

AutoML automates algorithm selection and hyperparameter search for classification, regression, time-series forecasting, NLP (text classification, NER), and computer vision tasks.

Key AutoML limits:

• experiment_timeout_hours: max wall-clock time; default 6 hours.
• max_concurrent_iterations: must not exceed the number of nodes in the compute cluster.
• enable_early_stopping: stops a child run if metric improvement stalls; enabled by default.
• featurization: "auto" (default), "off", or a FeaturizationConfig object.

Notebooks

Azure ML Studio provides managed Jupyter notebooks and JupyterLab on compute instances. Notebooks are stored in the workspace file share (/home/azureuser/cloudfiles/code). The exam distinguishes:

• Compute instance: single-node, always-on or scheduled, for interactive development.
• Compute cluster: multi-node, scales to zero, for training jobs — not for interactive notebooks.

Common traps:

• Assuming MLflow autolog captures every framework — Spark MLlib is supported but requires mlflow.pyspark.ml.autolog() explicitly.
• Confusing mlflow.log_model() (logs a model artifact within a run) with mlflow.register_model() (registers to the model registry).
• Believing AutoML requires a labeled dataset for forecasting — it also requires a time_column_name and a defined forecast_horizon.
• Setting max_concurrent_iterations higher than cluster node count — this silently clamps or causes failures.

Hyperparameter Tuning Automation and Distributed Training

Hyperparameter Tuning with Azure ML Sweep Jobs

A sweep job (SDK v2) / HyperDriveConfig (SDK v1) automates hyperparameter search. The exam uses SDK v2 terminology.

Early termination policies stop unpromising runs early:

BanditPolicy with slack_factor=0.1 is the most exam-tested policy. It cancels a run if its primary metric is not within 10% of the best run's metric at that evaluation interval.

Distributed Training

In SDK v2, set distribution = MpiDistribution(process_count_per_instance=N) or TorchDistribution(process_count_per_instance=N). The instance_count in ResourceConfiguration sets the number of nodes.

Common traps:

• Using GridParameterSampling with BanditPolicy — Grid sampling does not support early termination; the exam may present this as a valid combo.
• Confusing max_total_trials (total runs) with max_concurrent_trials (parallel runs).
• Believing Bayesian sampling supports early termination — it does not (runs must complete to inform the surrogate model).
• Setting process_count_per_instance equal to the number of GPUs per node (correct) versus the total GPU count (wrong).

Worked scenario: A team has a 4-node GPU cluster (4 GPUs each) and wants to maximize metric efficiency with minimal runs. → Use BayesianParameterSampling, no early termination policy, instance_count=4, process_count_per_instance=4.