Data Engineer Professional · 10% of the exam

Testing and Deployment: free practice questions

5 sample questions from our 25-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Databricks exam.

1. A team stores their Databricks jobs and notebook code in GitHub. They want a CI/CD pipeline that automatically deploys updated job definitions to a staging workspace whenever a pull request is merged to the `main` branch. Which combination of tools represents the recommended Databricks-native approach for this pattern?

A. GitHub Actions workflow calling `databricks bundle deploy` targeting the staging deployment target defined in `bundle.yml`✓ Correct
B. GitHub Actions workflow that uses the Databricks REST API directly to POST a new job definition JSON payload
C. Azure DevOps pipeline that runs `databricks workspace import` to upload notebooks and then `databricks jobs create`
D. GitHub Actions workflow that uses `dbfs cp` to copy wheel files and then triggers a job run via `databricks runs submit`

Explanation

Databricks Asset Bundles (DABs) are the recommended native approach for CI/CD. `bundle.yml` defines resources (jobs, pipelines, clusters) with environment-specific deployment targets (dev/staging/prod), and `databricks bundle deploy` idempotently deploys all defined resources to the target environment. Calling the REST API directly is error-prone, not idempotent, and bypasses the structured resource lifecycle management DABs provides. `databricks workspace import` and `databricks jobs create` is an older pattern that does not benefit from DAB's dependency resolution, variable substitution, or target management. Copying wheel files via `dbfs cp` and submitting runs addresses packaging but not job definition management or environment promotion.

2. An organization uses Delta Live Tables (DLT) for their ingestion pipeline. After a bad deployment introduced a data quality issue, they want to roll back the pipeline's output tables to their state from 48 hours ago. Which approach is MOST appropriate?

A. Delete the DLT pipeline and recreate it, pointing to a backup storage location created before the deployment.
B. Use Delta time travel to query each output table `AS OF` a timestamp 48 hours ago and overwrite the current tables with that historical data.✓ Correct
C. From the DLT pipeline UI, select the pipeline run from 48 hours ago and click 'Restore Pipeline State'.
D. Revert the pipeline code to the previous version in Git and trigger a full refresh; DLT will automatically restore the tables to their 48-hour-ago state.

Explanation

Delta time travel allows querying a Delta table's state at a specific point in time using `VERSION AS OF` or `TIMESTAMP AS OF`. By reading the table at the desired historical timestamp and overwriting the current table, the rollback is achieved precisely. This is the correct rollback strategy for Delta-based pipelines. Option A (delete and recreate) would work only if a separate backup exists, which is not mentioned, and causes unnecessary downtime. Option C is a fabricated UI feature; DLT does not have a 'Restore Pipeline State' button. Option D is incorrect because a full refresh re-processes source data from scratch using the current pipeline code; reverting the code and running a full refresh would regenerate data using the old logic but would not restore historical data quality — it would reflect the source data as it currently exists.

3. A team runs integration tests for a Databricks pipeline by spinning up an ephemeral cluster via a CI/CD job. The test notebook mounts test Delta tables, runs the pipeline logic, and asserts on output table contents. After completing, the cluster must be terminated and costs minimized. Which TWO practices are MOST important for managing costs and reliability in this integration test pattern? (Select TWO)

A. Use a single-node cluster (no workers) sized to the minimum required for the test workload.
B. Use `databricks runs submit --wait` with the `--new-cluster` flag so the cluster is created and automatically terminated after the run completes.✓ Correct
C. Configure the cluster with a 10-minute auto-termination policy as the sole cost-control mechanism.
D. Pin the cluster to a specific Databricks Runtime version matching production to ensure test fidelity.✓ Correct
E. Schedule the integration test job to run only on weekends to reduce weekday compute costs.
F. Use the default shared cluster pool to avoid cluster startup latency.

Explanation

Using `databricks runs submit --wait` with `--new-cluster` provisions an ephemeral cluster tied to that single run; the cluster is automatically terminated when the run finishes, eliminating the need for manual teardown or relying solely on auto-termination. Pinning the Databricks Runtime version to match production ensures that library versions, Spark behaviors, and Delta protocol features are consistent between tests and production, which is critical for integration test fidelity. Option A (single-node) may be insufficient for integration tests that test distributed behavior. Option C (auto-termination only) is a fallback, not a primary strategy — the cluster could still run for up to 10 minutes after completion needlessly. Option E (weekend scheduling) reduces test frequency and slows feedback loops, which defeats CI/CD goals. Option F (shared cluster pool) introduces test isolation issues since other workloads may be running on the pool.

4. A data engineering team is writing unit tests for a PySpark transformation function that filters and aggregates a sales DataFrame. They want to assert that two DataFrames are equal regardless of row order. Which library provides a purpose-built assertion function for this use case in PySpark unit tests?

A. pytest-spark, using the `assert_frame_equal` helper
B. Chispa, using the `assert_df_equality` function with `ignore_row_order=True`✓ Correct
C. pandas-testing, by converting both DataFrames with `.toPandas()` and calling `pd.testing.assert_frame_equal`
D. unittest, using the built-in `assertEqual` method on the DataFrame's `.collect()` result after sorting

Explanation

Chispa is a PySpark-specific testing library that provides `assert_df_equality`, which natively handles row-order independence, schema comparison, and null semantics — exactly what is needed for DataFrame assertions. pytest-spark does not provide an `assert_frame_equal` helper. Converting to pandas with `.toPandas()` works in small tests but does not scale and is not order-agnostic by default. Using `unittest.assertEqual` on `.collect()` results is fragile because list equality is order-sensitive, and manually sorting adds complexity without a guarantee of correctness across all data types.

5. A team has a Databricks Asset Bundle with the following partial `bundle.yml`: ```yaml bundle: name: sales_pipeline variables: env_prefix: default: dev targets: dev: workspace: host: https://adb-dev.azuredatabricks.net prod: workspace: host: https://adb-prod.azuredatabricks.net variables: env_prefix: prod resources: jobs: sales_job: name: "${var.env_prefix}_sales_job" tasks: - task_key: transform notebook_task: notebook_path: ./notebooks/transform ``` A CI/CD pipeline runs `databricks bundle deploy --target prod`. What will be the deployed job name in the production workspace?

A. `dev_sales_job`, because `env_prefix` defaults to `dev` and target-level variable overrides are not applied during deployment.
B. `prod_sales_job`, because the `prod` target overrides `env_prefix` to `prod`, which is substituted into the job name.✓ Correct
C. `${var.env_prefix}_sales_job`, because variable interpolation only occurs at runtime when the job executes, not at deployment time.
D. `sales_pipeline_prod_sales_job`, because the bundle name is automatically prepended to all resource names.

Explanation

In Databricks Asset Bundles, variable substitution using `${var.variable_name}` is resolved at deployment time. When `--target prod` is specified, the `prod` target's variable override (`env_prefix: prod`) takes effect, and `${var.env_prefix}` is resolved to `prod`. The deployed job name is therefore `prod_sales_job`. Option A is incorrect; target-level variable overrides are precisely the mechanism designed for environment-specific configuration. Option C is incorrect; DAB variable interpolation happens at bundle deployment time, not at job runtime. Option D is incorrect; the bundle name is not automatically prepended to resource names (though DABs does prefix resource names with the bundle name in some modes, the template syntax shown explicitly defines the full name).

20 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free