Machine Learning Implementation and Operations Study Guide — MLS-C01

Machine Learning Implementation and Operations (MLS-C01)

This domain tests your ability to take trained models into production reliably, cheaply, and securely. Expect scenario questions where you must choose the right deployment pattern, identify what triggered a monitoring alert, or design a retraining pipeline. The exam heavily favors SageMaker-native solutions over custom code—when in doubt, choose the managed service.

SageMaker Deployment Patterns

Choose the inference mode based on latency tolerance, payload size, and traffic shape:

Trap: Serverless inference has cold starts—it is not appropriate for latency-sensitive workloads. Async inference is not real-time; it queues requests and returns an S3 location, not an immediate response.

Multi-Model and Multi-Container Endpoints

• Multi-model endpoints (MME): Host thousands of models on a single endpoint. Models are loaded lazily into memory on first invocation and can be evicted. Use when models share the same framework and you need cost efficiency. Requires a TargetModel header in each request.
• Multi-container endpoints (MCE): Run up to 15 different containers on one endpoint, each with a different framework. Invoke a specific container directly or chain them in a pipeline mode for serial inference.

Trap: MME is for many models of the same type (cost savings). MCE is for different frameworks or preprocessing + inference chains (flexibility). They solve different problems.

SageMaker Pipelines and Model Registry

Pipelines provide a native CI/CD DAG for ML workflows. Steps include ProcessingStep, TrainingStep, EvaluationStep, RegisterModel, ConditionStep, and TransformStep. Pipelines integrate with EventBridge for triggering and store execution metadata in SageMaker Studio.

Model Registry handles versioning and promotion:

• Models are registered in model groups with a version status of PendingManualApproval, Approved, or Rejected.
• Deployment to staging/production is gated by approval status—this is the exam's canonical human-in-the-loop gate.
• Integrates with CodePipeline for automated deployment on approval.

Trap: A ConditionStep in a Pipeline is not the approval gate—it branches on a metric (e.g., accuracy threshold). The Model Registry approval is the human gate before production deployment.

SageMaker Model Monitor

Four monitor types, each with distinct inputs and alerts:

• Data Quality Monitor: Detects input data drift against a baseline captured at training time. Alerts on feature distribution shifts.
• Model Quality Monitor: Monitors prediction quality (accuracy, F1) by comparing predictions to ground-truth labels ingested via mergeTransform. Requires label capture.
• Bias Drift Monitor (Clarify): Detects changes in bias metrics (e.g., DPPL) using Clarify-based analysis.
• Feature Attribution Drift Monitor (Clarify): Monitors SHAP value drift to detect changes in feature importance.

All monitors generate violations as CloudWatch metrics and can trigger alerts. Baselines are computed once and stored in S3.