1. A data scientist fits a LASSO regression model (L1 regularization) for feature selection on a dataset with 200 features. After tuning the regularization strength `alpha`, they observe that 170 features have coefficients exactly equal to zero. Which of the following CORRECTLY describes WHY L1 regularization produces sparse solutions, and what the 30 remaining non-zero features represent? (Select TWO)
- A. L1 regularization adds the sum of absolute values of coefficients to the loss, which creates a geometry that encourages solutions at corners of the constraint space where many coefficients are exactly zero.✓ Correct
- B. L1 regularization penalizes the sum of squared coefficients, which shrinks all coefficients proportionally toward zero but never sets them exactly to zero.
- C. The 30 non-zero features are those the model has identified as having the strongest linear relationship with the target, given the chosen regularization strength.
- D. L1 regularization performs automatic feature selection because its non-differentiable penalty at zero creates subgradients that pull small coefficients to exactly zero during optimization.✓ Correct
- E. Increasing `alpha` further would increase the number of non-zero features because a higher penalty forces the model to distribute weight more evenly.
Explanation
Option A is correct: the L1 constraint region is a hyperdiamond (L1 ball) with corners aligned to the axes. The loss function's minimum tends to land on a corner where many coordinates are zero, producing sparsity — this is the geometric intuition for why L1 yields exact zeros. Option D is correct: analytically, the subdifferential of |w| at w=0 includes the interval [-1, 1], and the optimization update can push small coefficients precisely to zero during coordinate descent or subgradient methods. Together, these two options explain the mechanism from both geometric and optimization perspectives. Option B is incorrect: it describes L2 (Ridge) regularization, which penalizes squared coefficients and shrinks them toward zero but rarely to exactly zero. Option C is partially true in spirit but is imprecise — LASSO selects features based on the interplay between the penalty and the data; 'strongest linear relationship' is an oversimplification that ignores multicollinearity effects and does not describe the mechanism. Option E is incorrect: *increasing* `alpha` makes the penalty stronger, forcing *more* coefficients to zero (sparser solution), not fewer.