1. A senior ML engineer is designing a distributed training strategy for a large transformer model on Databricks. The model has 7 billion parameters and does not fit in the memory of a single GPU. Which THREE of the following techniques or frameworks are relevant solutions for training models that exceed single-GPU memory capacity on Databricks?
- A. DeepSpeed ZeRO optimization, which partitions optimizer states, gradients, and model parameters across GPUs/nodes to reduce per-device memory requirements.✓ Correct
- B. Horovod with AllReduce, which replicates the full model on each GPU and averages gradients — suitable when the model fits on a single GPU.
- C. PyTorch Fully Sharded Data Parallel (FSDP) via TorchDistributor, which shards model parameters, gradients, and optimizer states across all devices.✓ Correct
- D. Gradient accumulation with a single GPU, which allows training with effectively larger batch sizes but does not address model parameter memory limits.
- E. Model parallelism (pipeline or tensor parallelism), which splits different layers or tensor partitions across multiple GPUs so each holds only a portion of the model.✓ Correct
- F. Increasing the Spark driver memory to 128 GB to cache the full model on the driver node before distributing.
Explanation
Options A, C, and E are correct. DeepSpeed ZeRO (stages 1–3) partitions optimizer states, gradients, and/or parameters across devices, directly reducing per-GPU memory usage — stage 3 can train models far larger than single-GPU memory. PyTorch FSDP (accessible via TorchDistributor on Databricks) similarly shards parameters, gradients, and optimizer states across all GPUs. Model parallelism (pipeline or tensor) distributes model layers or tensor partitions across GPUs so no single device holds the full model. Option B (Horovod AllReduce) is wrong for this scenario because AllReduce is data-parallel and requires each GPU to hold a full model replica — it does not address models that exceed single-GPU memory. Option D (gradient accumulation) addresses batch size, not model size; the model parameters must still fit in GPU memory. Option F is wrong; the Spark driver is a CPU-based JVM process and is not used for GPU-based deep learning training — increasing driver memory has no effect on GPU memory for model parameters.