Databricks Machine Learning Associate: ML Workflows Practice Questions

sensor_df.describe() returns count, mean, stddev, min, and max for all numeric (and string) columns, which is exactly what is requested. sensor_df.summary() also works but additionally includes percentile statistics (25%, 50%, 75%) that go beyond the basic set described — describe() is the more direct match to the stated requirements. dbutils.data.summarize(sensor_df) produces a richer, notebook-rendered profile (including histograms and null counts), which is more than what was asked and is a Databricks-specific utility rather than a standard API call. printSchema() only shows column names and data types, not any statistical summaries.

Cramér's V is a measure of association between two categorical variables derived from the chi-squared statistic. It is appropriate when both variables are categorical (nominal or ordinal), as is the case here with churned (binary categorical) and subscription_plan (nominal categorical). Pearson correlation measures the linear relationship between two continuous numeric variables and is not valid for nominal categorical data. Spearman rank correlation is appropriate for ordinal data or monotonic relationships between continuous variables, but subscription_plan has no meaningful ordinal ranking. Point-biserial correlation measures the relationship between one binary variable and one truly continuous variable; while churned is binary, subscription_plan is nominal with multiple non-numeric levels, making point-biserial inapplicable.

The correct order for TF-IDF in PySpark ML is: (1) Tokenizer splits the raw text string into a list of tokens, (2) HashingTF maps those tokens to term-frequency vectors via feature hashing, and (3) IDF (an Estimator) is fit on the corpus and transforms the TF vectors into TF-IDF weighted vectors. Placing HashingTF before Tokenizer fails because HashingTF expects a sequence of tokens, not a raw string. Starting with IDF is impossible since IDF requires TF vectors as input. Placing IDF before HashingTF inverts the dependency — IDF rescales TF vectors and cannot produce them.

A histogram bins a continuous variable into intervals and shows the frequency or density of each bin, making it the standard chart type for assessing the shape of a distribution (e.g., normality, skewness, modality). Databricks built-in visualizations include histograms as a chart type accessible from the display() chart editor. A bar chart is used for comparing discrete categories, not for visualizing the distribution of a continuous variable. A scatter plot shows the relationship between two variables and is not suitable for displaying the distribution of a single variable. A pie chart shows proportions of a whole for categorical data and is entirely inappropriate for a continuous distribution.

MICE (Multiple Imputation by Chained Equations) is specifically designed for Missing At Random (MAR) data, where missingness depends on other observed variables. MICE iteratively models each feature with missing values as a function of other features, producing statistically sound imputations that preserve relationships between variables — critical for a medical dataset. Forward-fill is appropriate for time-series data where the previous value is a reasonable proxy for a missing value; it is not appropriate here because row order in a medical dataset is not temporally meaningful for blood pressure. Mean imputation ignores the MAR structure (the fact that missingness relates to age and diagnosis) and reduces variance, leading to biased model inputs. Dropping rows with 12% missingness wastes data and can introduce bias if the missingness is not completely at random (MCAR).