Data Engineer Associate · 16% of the exam

Production pipelines: free practice questions

5 sample questions from our 40-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Databricks exam.

1. A data engineering team has a workflow with a task that calls an external REST API. The API is occasionally unavailable for short periods, causing the task to fail. The team wants the task to retry up to 3 times with a 5-minute wait between attempts before marking the run as failed. Where in the Databricks Workflow UI should this be configured?

A. In the cluster policy attached to the job cluster, setting a retry policy with backoff
B. In the individual task's settings within the workflow, specifying the maximum retries and retry interval✓ Correct
C. In the workflow-level settings under 'Advanced', enabling auto-retry for all tasks globally
D. In the Databricks runtime environment variables, setting DATABRICKS_RETRY_MAX and DATABRICKS_RETRY_INTERVAL

Explanation

Option B is correct: Databricks Workflows allows retry configuration (maximum number of retries and the interval between retries) to be set at the individual task level within the workflow UI, giving granular control per task. Option A is incorrect because cluster policies govern cluster configuration (instance types, autoscaling, etc.) and do not contain retry logic for tasks. Option C is incorrect because Databricks Workflows does not have a single global retry setting that applies to all tasks; retries are configured per task. Option D is incorrect because there are no standard Databricks environment variables that control task-level retry behavior in Workflows; this would not have any effect.

2. A Delta Live Tables (DLT) pipeline is configured to run in Triggered mode. A data engineer notices that the pipeline processes all available data and then stops. Later, new source data arrives, but the pipeline does not automatically pick it up. Which of the following BEST describes the behavior of a Triggered DLT pipeline and how to address this?

A. Triggered mode continuously monitors the source and processes new data as it arrives; the pipeline may have an internal bug causing it to stop
B. Triggered mode processes all available data at the time of execution and then shuts down the cluster; the pipeline must be re-triggered (manually or via a schedule) to process new data✓ Correct
C. Triggered mode is equivalent to Continuous mode but requires a manual start for each batch; switching to Continuous mode would fix the issue without changing behavior
D. Triggered mode is only available for streaming sources; the engineer should switch to batch mode to handle file-based sources

Explanation

Option B correctly describes Triggered mode behavior. In Triggered mode, a DLT pipeline starts, processes all data currently available in the source(s), updates all tables, and then the cluster shuts down. It does NOT continuously monitor for new arrivals. To pick up new data, the pipeline must be triggered again—either manually or on a schedule (e.g., via a Databricks Workflow). Option A is wrong because Triggered mode is designed to stop after processing; it does not continuously monitor sources. Option C is wrong because Triggered and Continuous modes have fundamentally different behaviors; Continuous mode keeps the cluster running and processes new data as it arrives. Option D is wrong because Triggered mode works with both batch and streaming sources; there is no separate 'batch mode' in DLT.

3. A data engineer is designing a Databricks Workflow for a financial reporting pipeline. The pipeline has the following requirements: (1) If any task fails, the on-call engineer must be notified immediately. (2) Certain non-critical tasks, if they fail, should not block downstream tasks. Which TWO Databricks Workflow features should the engineer use to meet these requirements? (Select TWO)

A. Configure job-level email notifications for the 'On failure' event✓ Correct
B. Set non-critical tasks to use the 'Run if' condition set to 'At least one parent succeeded'
C. Enable automatic retry with 5 retries on all tasks to minimize failure notifications
D. Configure the non-critical tasks with 'Continue on failure' behavior so downstream tasks proceed even if they fail✓ Correct
E. Use a Databricks SQL Alert instead of job notifications for failure detection

Explanation

Job-level 'On failure' email notifications ensure the on-call engineer is alerted whenever any task in the workflow fails, satisfying requirement 1. Setting non-critical tasks to 'Continue on failure' (available in task settings) allows downstream tasks to still execute even if the non-critical task fails, satisfying requirement 2. Setting 'Run if' to 'At least one parent succeeded' controls downstream execution logic based on parent outcomes but is designed for branching logic, not for allowing a failed task itself to not block — 'Continue on failure' is the direct setting. Configuring retries on all tasks would reduce failure notifications but adds unnecessary delay and does not address the notification or unblocking requirements. Using a Databricks SQL Alert is for query-based threshold monitoring, not Workflow task failure alerting.

4. A data engineer is designing a production pipeline that must process streaming data from an Apache Kafka topic with low latency (results should be updated within seconds of new data arriving). The pipeline has complex multi-hop transformations defined as Delta Live Tables. Which DLT pipeline mode is MOST appropriate?

A. Triggered mode with a 1-minute schedule, because it processes data in micro-batches and achieves near-real-time latency
B. Continuous mode, because it keeps the pipeline running and processes new data as soon as it arrives, achieving the lowest possible latency✓ Correct
C. Triggered mode with enhanced autoscaling, because autoscaling compensates for latency in Triggered mode
D. Development mode with a continuous trigger, because Development mode is optimized for low-latency streaming scenarios

Explanation

Continuous mode is correct for low-latency streaming requirements. In Continuous mode, the DLT pipeline keeps the cluster running and continuously processes new records as they arrive from the source (such as Kafka), achieving latency on the order of seconds. Option A is wrong because even a 1-minute schedule in Triggered mode means data waits up to a minute before being processed; this does not meet a 'within seconds' latency requirement, and Triggered mode also incurs cluster startup overhead. Option C is wrong because autoscaling affects compute capacity and does not reduce the inherent latency of Triggered mode—the cluster still needs to start and the pipeline still processes in discrete runs. Option D is wrong because Development mode vs. Production mode is about cluster lifecycle and retry behavior, not latency optimization; Development mode is not a streaming execution mode and cannot be paired with a 'continuous trigger' in the way described.

5. A data engineering team has a multi-step Databricks Workflow with the following tasks in order: (1) ingest raw data, (2) transform data, (3) generate a report. Task 3 fails due to a transient network error. The team wants to re-run only the failed task and any tasks that depend on it, without re-running the successful ingestion and transformation steps. Which Databricks Workflows feature should they use?

A. Create a new job run with all tasks configured from the beginning
B. Use the Repair Run feature to re-run only the failed task and its downstream dependents✓ Correct
C. Manually trigger Task 3 as a standalone job with the same cluster configuration
D. Clone the job, remove Tasks 1 and 2, and run the cloned job

Explanation

Repair Run is the correct answer. Databricks Workflows provides a 'Repair Run' feature that allows you to re-run only the failed tasks (and optionally their downstream dependents) from an existing job run, reusing the results of already-successful tasks. This avoids redundant reprocessing. Option A is wrong because re-running all tasks wastes compute and time on steps that already succeeded. Option C is wrong because manually triggering Task 3 standalone does not exist as a built-in Databricks feature and would not respect the job's dependency context. Option D is wrong because cloning and modifying a job is a manual, error-prone workaround and does not preserve the original run's context.

35 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free