Model Development Study Guide — Machine Learning Associate

Model Development – ML Associate Study Guide

This domain accounts for 31% of the exam and covers training and tuning models on Databricks: Spark ML pipelines, hyperparameter tuning, evaluation metrics, and controlling overfitting.

Problem Types

• Regression predicts a continuous value (RMSE, R²).
• Classification predicts a category (precision, recall, F1, AUC).
• Clustering groups unlabeled data (unsupervised).

Spark ML Pipelines

A Spark ML Pipeline chains stages of Transformers and an Estimator:

• A Transformer implements transform() to produce a new DataFrame.
• An Estimator implements fit() to learn and return a Transformer (the model).

The fitted PipelineModel applies the identical sequence at inference, preventing train/serve skew. Spark ML distributes computation across the cluster, scaling to data larger than one node's memory.