AWS MLS-C01: Modeling Practice Questions

Linear Learner is correct because it natively supports binary classification, automatically normalizes input features during preprocessing, provides a 'positive_example_weight_mult' hyperparameter for class imbalance, and is highly optimized for fast training on large tabular datasets. XGBoost is a strong classifier but does not perform automatic feature normalization and requires manual handling of class imbalance (e.g., via 'scale_pos_weight'). K-Means is an unsupervised clustering algorithm and cannot perform supervised binary classification. Random Cut Forest is an unsupervised anomaly detection algorithm, not a binary classifier.

Recall (also called sensitivity or true positive rate) measures the fraction of actual positives (churners) that the model correctly identifies. Since the business priority is minimizing missed churners (false negatives), recall is the correct primary metric. Accuracy is misleading when the class distribution is imbalanced (churn is typically rare) and does not specifically reward finding all churners. Precision measures the fraction of predicted churners that are actual churners; optimizing precision reduces false alarms but allows more churners to go undetected, which conflicts with the stated business priority. Specificity measures the true negative rate (correctly identifying non-churners), which is the opposite of what is needed here.

Disparate Impact (DI) measures the ratio of the positive prediction rate (favorable outcome) for an unprivileged group to that of a privileged group. A DI value significantly below 1.0 indicates the model is less likely to grant positive outcomes to the disadvantaged group. — Class Imbalance (CI) is a pre-training bias metric measuring imbalance in the label distribution or group representation in training data, not the model's prediction rates. — SHAP values measure feature importance and contribution to individual predictions but are not a bias fairness metric between groups. — Label Distribution Skew is not a standard SageMaker Clarify bias metric name.

Setting 'scale_pos_weight' to ~124 (ratio of negatives to positives: ~99.2/0.8 ≈ 124) directly tells XGBoost to penalize missed defaults much more heavily than missed non-defaults, correcting for class imbalance at the algorithm level. Switching the evaluation metric to 'auc' (Area Under the ROC Curve) is critical because accuracy is misleading on severely imbalanced datasets; AUC measures the model's ability to discriminate between classes regardless of the prediction threshold. Oversampling the minority class with SMOTE or random oversampling before training increases the effective representation of defaulters in the training data, helping the model learn default patterns. Increasing 'num_round' would allow the model to better fit the majority class but does nothing to address imbalance and could worsen the minority-class problem. Reducing 'max_depth' to 2 severely underfits the model; it does not improve minority-class detection and would produce a model that defaults to the majority class prediction. Adding L1 regularization to prevent learning any features is counterproductive and would result in a useless model.

K-Means is the correct choice because it is an unsupervised clustering algorithm purpose-built for exactly this scenario: partitioning unlabeled data points into k distinct groups based on feature similarity. SageMaker's K-Means implementation scales efficiently to millions of records. Linear Learner requires labeled data for supervised classification and cannot perform unsupervised segmentation. XGBoost does not have an unsupervised training objective; it is a supervised gradient-boosted tree algorithm. BlazingText in supervised mode is a text classification algorithm that requires labeled text data and is not applicable to tabular customer behavior data.