ML Solution Monitoring, Maintenance, and Security Study Guide — MLA-C01

ML Solution Monitoring, Maintenance, and Security – MLA-C01 Study Guide

This domain accounts for 24% of the exam and covers keeping a deployed model healthy and secure: detecting drift, monitoring operations, retraining, and applying AWS security controls to ML systems. The highest-value topics are drift detection with Model Monitor and securing ML with IAM/encryption/VPC. Much of the security content is standard AWS security applied to ML — know the shared responsibility split and which service does audit vs monitoring vs sensitive-data discovery.

Monitoring Model Quality: Drift

A model that was accurate at launch can silently degrade:

• Data drift — the input feature distribution changes over time (new user behavior, seasonality).
• Concept (model quality) drift — the relationship between inputs and the target changes, so predictions degrade even if inputs look similar.

Amazon SageMaker Model Monitor runs scheduled checks against a baseline to detect data-quality and model-quality drift on live endpoints (and, integrated with SageMaker Clarify, bias drift and feature-attribution/explainability drift).

Maintenance response: when significant drift is detected, trigger retraining on fresh data and redeploy an updated model — often automated via a pipeline. Ignoring drift is not an option; it degrades accuracy.

Monitoring Operations and Auditing

Discriminators: "endpoint latency/error alarms" → CloudWatch; "who called which API" → CloudTrail; "find PII before using data" → Macie; "download SOC/ISO reports" → Artifact.

Under load with rising latency/errors, the right first response is: set CloudWatch alarms, enable/adjust auto scaling to add capacity, then investigate the root cause — never disable monitoring to silence alarms.