Machine Learning Implementation and Operations: free practice questions

Target tracking policy targeting InvocationsPerInstance is correct because SageMaker exposes InvocationsPerInstance as a CloudWatch metric specifically for auto scaling, and target tracking automatically adjusts capacity to keep the metric at a defined target value, which is the recommended approach for SageMaker endpoints. Step scaling policy targeting CPUUtilization is wrong because while step scaling is valid for some AWS services, SageMaker endpoint auto scaling best practice uses target tracking, and CPUUtilization is not a published SageMaker endpoint metric. Target tracking targeting MemoryUtilization is wrong because MemoryUtilization is not a built-in SageMaker endpoint CloudWatch metric and cannot be used directly for endpoint auto scaling without custom metrics. Scheduled scaling is wrong because it cannot react dynamically to sudden, unpredictable spikes — it only scales at predefined times, which is insufficient for unexpected traffic surges.

Retraining on only recent post-pandemic data is correct because when the underlying data distribution has permanently shifted (concept drift), training on historical pre-pandemic data would introduce noise and bias — the model needs to learn from data representative of the new regime. Re-evaluating and re-engineering features is correct because a fundamental change in consumer behavior may mean existing features no longer have predictive power; feature engineering aligned with new patterns is a necessary part of adapting to concept drift. Establishing a new Model Monitor baseline is correct because the original baseline reflects the old data distribution — monitoring against a stale baseline will generate irrelevant alerts or miss new issues; a fresh baseline from recent data is required for meaningful ongoing monitoring. Increasing auto scaling capacity is wrong because the problem is model accuracy (concept drift), not insufficient compute capacity — adding more instances will not improve predictions. Scheduling retraining every 24 hours on all historical data is wrong because including pre-pandemic data dilutes the training signal from the new distribution and does not address concept drift; it would perpetuate the problem. Switching to a multi-model endpoint for both old and new versions is wrong because the old model is known to be inaccurate under the new distribution — hosting it alongside the new model adds no value and could confuse A/B testing results.

Data Quality Monitor is correct because it compares the statistical properties (distributions, missing values, data types) of live inference input data against a baseline computed from training data, which is exactly the scenario described. Model Quality Monitor is wrong because it measures prediction quality metrics (e.g., accuracy, F1) against ground-truth labels — it requires actual labels to be available after inference. Bias Drift Monitor is wrong because it tracks changes in bias metrics (e.g., disparate impact across demographic groups), not general feature distribution shift. Feature Attribution Drift Monitor is wrong because it uses SHAP values to detect changes in feature importance over time, not raw feature distribution statistics.

Enable VPC configuration (SubnetIds and SecurityGroupIds) is correct because placing the training job inside a VPC with no internet gateway routes ensures training instances cannot reach the internet, and when combined with an S3 VPC endpoint, S3 traffic stays within the AWS network and is encrypted in transit via HTTPS. EnableInterContainerTrafficEncryption is correct because it explicitly encrypts the inter-node communication channel between distributed training instances, satisfying the third requirement. EnableNetworkIsolation is wrong because while it prevents outbound network access, it also blocks the training container from reaching S3, which means training data cannot be fetched — VPC configuration is the correct mechanism for internet isolation while retaining S3 access via VPC endpoints. Configuring an S3 VPC endpoint and enabling SSE on S3 alone is wrong because it addresses data-at-rest encryption and S3 routing but does not prevent internet access from the training instances by itself — VPC configuration is still required. Attaching an IAM policy that denies outbound internet access is wrong because IAM controls API-level permissions, not network-level traffic; it cannot enforce network isolation for training instance traffic.

SageMaker Pipelines is correct because it is a purpose-built CI/CD service for ML workflows that natively integrates with SageMaker training, processing, evaluation, Model Registry, and deployment steps. It automatically tracks lineage, supports conditional steps, and is designed for reproducible, auditable ML pipelines with built-in SageMaker integration. AWS Glue Workflows is wrong because Glue Workflows orchestrate ETL jobs (Glue crawlers and jobs), not SageMaker training and evaluation steps, and lacks native ML pipeline primitives. Amazon MWAA is wrong because while Airflow can orchestrate ML workflows, it requires significant custom setup to integrate with SageMaker and does not provide native lineage tracking or Model Registry integration out of the box — SageMaker Pipelines is more appropriate for SageMaker-centric ML pipelines. AWS Step Functions is wrong because while it can orchestrate SageMaker jobs via the SageMaker SDK integration, it lacks the native ML-specific features of SageMaker Pipelines such as automatic experiment tracking, lineage, and direct Model Registry integration.