Machine Learning Associate · 31% of the exam

Model Development: free practice questions

5 sample questions from our 95-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Databricks exam.

1. A senior ML engineer is designing a distributed training strategy for a large transformer model on Databricks. The model has 7 billion parameters and does not fit in the memory of a single GPU. Which THREE of the following techniques or frameworks are relevant solutions for training models that exceed single-GPU memory capacity on Databricks?

  • A. DeepSpeed ZeRO optimization, which partitions optimizer states, gradients, and model parameters across GPUs/nodes to reduce per-device memory requirements.✓ Correct
  • B. Horovod with AllReduce, which replicates the full model on each GPU and averages gradients — suitable when the model fits on a single GPU.
  • C. PyTorch Fully Sharded Data Parallel (FSDP) via TorchDistributor, which shards model parameters, gradients, and optimizer states across all devices.✓ Correct
  • D. Gradient accumulation with a single GPU, which allows training with effectively larger batch sizes but does not address model parameter memory limits.
  • E. Model parallelism (pipeline or tensor parallelism), which splits different layers or tensor partitions across multiple GPUs so each holds only a portion of the model.✓ Correct
  • F. Increasing the Spark driver memory to 128 GB to cache the full model on the driver node before distributing.
Explanation

Options A, C, and E are correct. DeepSpeed ZeRO (stages 1–3) partitions optimizer states, gradients, and/or parameters across devices, directly reducing per-GPU memory usage — stage 3 can train models far larger than single-GPU memory. PyTorch FSDP (accessible via TorchDistributor on Databricks) similarly shards parameters, gradients, and optimizer states across all GPUs. Model parallelism (pipeline or tensor) distributes model layers or tensor partitions across GPUs so no single device holds the full model. Option B (Horovod AllReduce) is wrong for this scenario because AllReduce is data-parallel and requires each GPU to hold a full model replica — it does not address models that exceed single-GPU memory. Option D (gradient accumulation) addresses batch size, not model size; the model parameters must still fit in GPU memory. Option F is wrong; the Spark driver is a CPU-based JVM process and is not used for GPU-based deep learning training — increasing driver memory has no effect on GPU memory for model parameters.

2. A data scientist is training a PyTorch model using `TorchDistributor` on a multi-node Databricks cluster. Inside the training function passed to `TorchDistributor.run()`, they need to initialize the distributed process group. Which PyTorch call is responsible for this initialization?

  • A. torch.distributed.init_process_group(backend='nccl')✓ Correct
  • B. torch.nn.parallel.DistributedDataParallel(model)
  • C. torch.multiprocessing.spawn(fn, nprocs=num_gpus)
  • D. torch.distributed.barrier()
Explanation

torch.distributed.init_process_group() initializes the distributed communication backend before any distributed training can occur. For GPU training, the 'nccl' backend is the standard choice as it is optimized for NVIDIA GPU-to-GPU communication. Option B (DistributedDataParallel) wraps the model for distributed training but requires init_process_group to have been called first — it cannot substitute for initialization. Option C (multiprocessing.spawn) is a way to launch multiple processes locally but is not used inside a TorchDistributor training function, where processes are already managed by Spark. Option D (torch.distributed.barrier) is a synchronization primitive that blocks all processes until they reach the barrier — it does not initialize the process group.

3. A data scientist is tuning a LightGBM classifier with Hyperopt and `SparkTrials`. They notice that each trial takes a long time because the LightGBM training itself uses multiple CPU threads internally. They set `SparkTrials(parallelism=16)` on a cluster with 16 workers (each with 4 cores). Which of the following best describes the potential problem and the correct fix? (Select TWO)

  • A. The problem is that SparkTrials dispatches one trial per Spark task, and if LightGBM uses multiple threads per trial, each task may request more cores than a single Spark executor slot provides, causing resource contention or underutilization✓ Correct
  • B. The fix is to set `num_threads=1` (or the equivalent LightGBM parameter) in the LightGBM training call inside the objective function so each trial uses a single core, matching the single-slot-per-task Spark model✓ Correct
  • C. The fix is to increase parallelism to 64 (16 workers × 4 cores) so each core runs a separate trial simultaneously
  • D. The problem is that SparkTrials is incompatible with LightGBM and only works with XGBoost models
  • E. The fix is to replace SparkTrials with Trials() and run 16 trials sequentially to avoid thread contention
Explanation

Options A and B together correctly describe the problem and fix. SparkTrials executes each Hyperopt trial as a single Spark task. Each Spark task is typically allocated one CPU slot. If LightGBM internally uses multiple threads (e.g., num_threads=4 by default), it will try to claim 4 cores but only 1 is allocated per Spark task, causing resource contention and degraded performance across concurrent trials. The fix is to set num_threads=1 (LightGBM's parameter for thread count) inside the objective function, aligning resource usage with Spark's single-slot-per-task model. Option C is wrong because increasing parallelism to 64 does not solve the thread contention issue and would over-subscribe cores. Option D is false — SparkTrials works with any Python-based model including LightGBM; it is framework-agnostic. Option E is counterproductive — switching to sequential Trials() defeats the purpose of distributed tuning.

4. Why might accuracy be a misleading metric on a heavily imbalanced dataset?

  • A. A model predicting only the majority class scores high accuracy while missing the minority class entirely✓ Correct
  • B. Accuracy always equals recall
  • C. Accuracy cannot be computed on imbalanced data
  • D. Accuracy is only for regression
Explanation

With imbalance, always predicting the majority class yields high accuracy but catches none of the important minority class — so precision/recall/F1 are more informative. The other statements are false.

5. A data scientist is building a churn prediction model and uses SHAP values to explain the model's predictions. After logging SHAP values with MLflow, she observes that a feature called `days_since_last_purchase` has consistently large absolute SHAP values across nearly all predictions. Which of the following is the MOST accurate interpretation?

  • A. `days_since_last_purchase` has a large coefficient in the model's weight matrix, which means it was heavily regularized during training.
  • B. `days_since_last_purchase` contributes significantly to the model's output (pushes predictions away from the base value) for most customers, indicating it is one of the most globally important features for churn prediction.✓ Correct
  • C. Large SHAP values for `days_since_last_purchase` indicate that the feature has a high variance in the dataset, which may cause instability in the model.
  • D. SHAP values measure prediction error, so a large SHAP value for this feature means it is causing the model to make incorrect predictions.
Explanation

Correct: SHAP (SHapley Additive exPlanations) values measure each feature's contribution to shifting a specific prediction away from the average model output (base value). When a feature has consistently large absolute SHAP values across most predictions, it means that feature is highly influential in driving the model's output — it is a globally important feature. This is the correct interpretation. Wrong - A: SHAP values are model-agnostic explanations of output contributions and do not correspond to weight matrix coefficients or regularization. Many models explained by SHAP (e.g., tree ensembles) do not even have a traditional weight matrix. Wrong - C: SHAP values measure feature contribution to predictions, not feature variance in the dataset. High variance in a feature does not directly produce large SHAP values — an uninformative high-variance feature can have near-zero SHAP values. Wrong - D: SHAP values are NOT error metrics. A large SHAP value means the feature strongly influences the prediction, whether that prediction is correct or incorrect.

90 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free
Model Development — Free Machine Learning Associate Practice Questions | DataCertPrep — Certification Prep