Data Engineer Professional · 10% of the exam

Monitoring and Alerting: free practice questions

5 sample questions from our 31-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Databricks exam.

1. A data engineering team runs 50 Delta Live Tables pipelines across multiple business units. The finance team needs a weekly report showing the DBU consumption broken down by individual pipeline. Which approach is MOST appropriate for obtaining this data?

  • A. Query the Databricks Billing API (or the system.billing.usage table if available) filtering on usage_metadata.dlt_pipeline_id to aggregate DBUs per pipeline per week.✓ Correct
  • B. Sum the number of cluster events in each pipeline's cluster event log and multiply by a fixed DBU rate to estimate consumption.
  • C. Use the DLT event log and sum the processing_time field across all flow_progress events to derive a proxy for DBU usage.
  • D. Navigate to the Databricks Workflows UI for each pipeline and manually read the cluster runtime from each run's summary page.
Explanation

The system.billing.usage table (or the Databricks Billing API) contains per-workload DBU consumption records with metadata including pipeline identifiers (usage_metadata.dlt_pipeline_id), making it the correct and scalable source for pipeline-level cost attribution (option A). Option B is incorrect because cluster events are lifecycle records (e.g., STARTED, RESIZED) and not time-series utilization metrics; multiplying event counts by a DBU rate is not a valid computation. Option C is incorrect because the DLT event log's processing_time tracks execution duration of individual flow runs, not DBU consumption — the conversion would require cluster size and type information not present in the event log. Option D is operationally impractical for 50 pipelines and produces only rough wall-clock time, not DBU figures.

2. After deploying a new version of a Delta Live Tables pipeline, a data engineer wants to confirm that the expectations (data quality rules) for the `silver_orders` dataset are behaving as expected. Which query against the DLT event log would correctly return the pass and fail counts for expectations on the `silver_orders` dataset from the most recent pipeline update?

  • A. SELECT details:flow_progress:data_quality:dropped_records FROM event_log('silver_orders') WHERE event_type = 'flow_progress' ORDER BY timestamp DESC LIMIT 1
  • B. SELECT details:flow_progress:data_quality:expectations FROM event_log('my_pipeline') WHERE event_type = 'flow_progress' AND details:flow_progress:name = 'silver_orders' ORDER BY timestamp DESC LIMIT 1✓ Correct
  • C. SELECT * FROM delta.`/pipelines/silver_orders/_quality_log` ORDER BY timestamp DESC LIMIT 10
  • D. SELECT details:expectations FROM event_log('my_pipeline') WHERE event_type = 'dataset_definition' AND details:dataset_name = 'silver_orders'
Explanation

In the DLT event log, data quality metrics (pass counts, fail counts, and per-expectation results) are nested inside `details:flow_progress:data_quality:expectations` for events of type `flow_progress`. Filtering on `details:flow_progress:name = 'silver_orders'` scopes the results to the correct dataset, and ordering by timestamp descending with LIMIT 1 retrieves the most recent update. Option A uses `event_log('silver_orders')` which is incorrect syntax—the `event_log()` function takes a pipeline ID or name, not a dataset name. Option C references a fabricated path (`_quality_log` directory) that does not exist in DLT. Option D uses `event_type = 'dataset_definition'`, which records schema and metadata at startup, not runtime data quality metrics.

3. A data engineer is integrating Databricks cluster metrics with Prometheus for a centralized monitoring stack. Which of the following correctly describes how Databricks supports Prometheus metric collection from clusters?

  • A. The Databricks Metrics API (spark.metrics.conf) can be configured to push cluster and Spark metrics to a Prometheus Pushgateway by specifying the sink class and endpoint in a cluster init script.✓ Correct
  • B. Databricks natively exposes a /metrics endpoint on every cluster's driver node that Prometheus can scrape directly without any additional configuration.
  • C. Prometheus integration is only supported on Databricks Runtime for Machine Learning clusters; standard DBR clusters must use Datadog or Ganglia.
  • D. Databricks provides a built-in Prometheus sink that is enabled by default, but requires the cluster to be pinned to a Single-Node configuration.
Explanation

Databricks clusters expose the standard Apache Spark metrics system, which can be configured via spark.metrics.conf (typically delivered via an init script) to use a PrometheusSink or push metrics to a Prometheus Pushgateway. This is the supported integration pattern for Prometheus (option A). Option B is incorrect because Databricks does not automatically expose a /metrics HTTP endpoint for Prometheus scraping without configuration; network access to cluster nodes is also not directly available from outside the VPC without additional networking setup. Option C is a fabrication — Prometheus integration is not restricted to ML Runtime clusters; any Databricks Runtime cluster can be configured with the metrics sink. Option D is incorrect because the Prometheus sink is not enabled by default and has no dependency on Single-Node cluster configuration.

4. A Databricks SQL query should trigger a notification when a monitored metric (e.g., failed-record count) crosses a threshold. Which feature enables this?

  • A. Databricks SQL Alerts on a scheduled query✓ Correct
  • B. OPTIMIZE ZORDER
  • C. A broadcast join
  • D. VACUUM
Explanation

Databricks SQL Alerts run a query on a schedule and notify when a result crosses a defined threshold. OPTIMIZE ZORDER (B), broadcast joins (C), and VACUUM (D) are performance/maintenance operations, not alerting.

5. A data engineer needs to be notified automatically when a scheduled Databricks Job fails. What should they configure?

  • A. Job notifications/alerts on failure (email or webhook) in the Job settings✓ Correct
  • B. A ZORDER on the output table
  • C. A Delta Sharing recipient
  • D. A larger driver node
Explanation

Databricks Jobs support notifications on start/success/failure via email or webhooks, so the team is alerted on failures. ZORDER (B), Delta Sharing (C), and a bigger driver (D) don't provide failure alerting.

26 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free