Free Databricks Data Engineer Professional practice test

Option A is correct — rotating/revoking the compromised credential is the single most critical immediate action; the secret must be treated as compromised the moment it was pushed. Option D is correct — to truly remove a secret from Git history, a history-rewriting tool like git filter-repo (or BFG Repo Cleaner) must be used on the remote, followed by a force-push; this is the standard Git remediation. Option F is incorrect — the Databricks Repos UI only manages the working state of the repo (branch switching, pull, commit, push); it cannot rewrite remote Git history. Option E is incorrect — deleting and re-cloning the Repo object in Databricks only affects the local workspace copy; the remote Git provider's history is completely unaffected. Option B is a good future prevention best practice but is not an immediate remediation for the already-exposed secret. Option C is incorrect — Databricks Unity Catalog audit logs record API actions, not notebook cell content or committed file contents; the secret would not be captured there.

In Unity Catalog, privileges follow a hierarchy that requires USE permissions at every level of the namespace. To create a table in main.bronze, the user must have USE CATALOG on main AND USE SCHEMA on main.bronze in addition to CREATE TABLE. Without either of these, navigation through the namespace fails. Option B (CREATE SCHEMA) is unnecessary — the schema already exists. Option C (MODIFY) applies to modifying existing table data, not creating new tables. Option D (READ FILES) is a volume/external location privilege, not required for managed table creation.

The correct Delta Sharing provider workflow is: (1) Create a recipient object that represents the external partner, (2) Create a share object that acts as a container for data assets, (3) Add the Delta table to the share, and (4) Grant the recipient access to the share so they can read the data. Option D has the order of recipient and share creation swapped and describes 'activating' the recipient before granting share access. Option A describes an older open-source Delta Sharing server model that does not apply to the Databricks-managed Delta Sharing workflow. Option B incorrectly introduces external locations and uses SELECT syntax that does not apply to shares.

The correct Databricks DBFS REST API endpoint for listing directory contents is GET /api/2.0/dbfs/list, and the path parameter is passed as a query parameter or in the request body as JSON. Option A uses an incorrect endpoint path /dbfs/ls which does not exist. Option C uses /dbfs/read, which is for reading file contents, not listing directories. Option B uses POST instead of GET; the list endpoint uses GET.

Unity Catalog enforces namespace traversal: a user must have USE CATALOG on prod and USE SCHEMA on prod.raw before any object-level privilege (including READ VOLUME) takes effect. Even with READ VOLUME, missing USE CATALOG/USE SCHEMA causes an access error. Option E is wrong — READ VOLUME is the correct privilege for reading volumes; SELECT applies to tables. Option C is plausible but describes an admin-level misconfiguration; the question specifies the volume already exists and the colleague has READ VOLUME, making missing USE grants the most common user-level cause. Option B is wrong — Unity Catalog volumes are accessed via the /Volumes/ path and do not require manual mounting. Option D is wrong — external volumes fully support access control.

Auto Loader (cloudFiles) incrementally and efficiently detects and ingests new files as they arrive, with schema inference/evolution and scalable file discovery. A one-time COPY INTO (B) isn't continuous, a full directory batch read (C) reprocesses everything, and Delta Sharing (D) shares data across orgs.

Option B is correct: VACUUM with an explicitly reduced retention (e.g., RETAIN 0 HOURS, bypassing the safety check) deletes files that may still be referenced by older table versions, breaking time travel for those versions. Option C is correct: delta.deletedFileRetentionDuration controls the minimum age of deleted (dereferenced) data files before VACUUM can remove them; setting it to 30 days ensures files needed for 3-day-old time travel are preserved. Option E is correct: time travel requires the physical Parquet data files to exist on storage — the Delta log tracks metadata (file paths, statistics), but if the files are physically deleted by VACUUM, the data is unrecoverable from the log alone. --- Option A is misleading/wrong in this scenario: the default is 7 days, which would normally protect a 3-day query, BUT the question states the error is occurring, implying the default was overridden (as described in B) — so A does not explain the failure. Option D is wrong: delta.logRetentionDuration preserves the transaction log entries (enabling time travel version lookups) but does NOT prevent VACUUM from deleting the underlying data files; both properties must be set appropriately. Option F is wrong: DESCRIBE HISTORY shows the audit log of table operations and is read-only; it does not restore deleted files or make previously vacuumed versions queryable.

expect_or_drop drops records that violate the constraint and records the violation counts in the DLT pipeline event log (accessible via the pipeline UI and system tables), satisfying both requirements. Option C is wrong because @dlt.expect (warn mode) does NOT drop records — it allows all records through and only logs a warning count; it does not prevent invalid data from being written. Option A is wrong because expect_or_fail halts the entire pipeline when any record violates the constraint; while it does count violations, stopping the pipeline is far more disruptive than the requirement calls for and does not simply reject individual records. Option B is wrong because expect_all is a convenience wrapper for applying multiple expectations at once (using warn semantics by default); it does not quarantine records into a separate system table — there is no such automatic quarantine behaviour in DLT.

Delta Sharing is an open protocol for securely sharing live data across organizations and platforms without copying it. Auto Loader (B) ingests files, cluster policies (C) govern compute, and OPTIMIZE ZORDER (D) is a performance operation.

Ganglia is a built-in cluster monitoring tool enabled by default on all Databricks clusters (both all-purpose and job clusters). It is accessible via the Metrics tab on the cluster detail page and provides real-time and historical views of CPU, memory, network I/O, and disk metrics for the driver and all worker nodes. Option C is incorrect because Ganglia is enabled by default, not opt-in. Option D is incorrect in claiming Ganglia is unavailable for job clusters—it is available on job clusters as well. Option B is incorrect; while Databricks has expanded Prometheus and Datadog integrations, Ganglia has not been fully removed or deprecated in current Databricks Runtimes as of the DE-Professional-2024 exam scope.

To isolate whether slowness is due to cold start, queue wait, or execution: (E) The queue wait time (time between submission and execution start) distinguishes queue delays from execution time; the warehouse start time relative to submission identifies cold start overhead. (C) Bytes read from cloud storage combined with the Query Profile (execution plan with per-node timing) pinpoints whether the slowness is within the query execution itself—e.g., a full table scan or a skewed join. Option D (compilation time and output rows) can indicate plan complexity but does not directly isolate the three phases. Option B (concurrent queries and auto-stop setting) provides useful context but does not directly measure queue wait or execution time from the Query History view. Option A (Photon status and result caching) are relevant performance factors but do not help isolate cold start vs. queue vs. execution.

OPTIMIZE compacts many small files into fewer larger ones (and optimized writes/auto-compaction prevent them), improving read performance. DELETE (B) removes rows, a broadcast join (C) is a query technique, and adding columns (D) doesn't address small files.

Photon is Databricks' vectorized native execution engine that speeds up SQL/DataFrame workloads, improving price/performance. VACUUM (C) is file cleanup, Delta Sharing (B) shares data, and widgets (A) parameterize notebooks.

OPTIMIZE ... ZORDER BY co-locates related data and improves file-level data skipping for the specified column, reducing files scanned. VACUUM (C) removes old files (and zero retention is dangerous), CSV (A) loses Delta features, and disabling statistics (D) worsens skipping.

Options B and C are correct. Unity Catalog provides two complementary programmatic ways to access lineage: (1) The Lineage Tracking REST API (/api/2.0/lineage-tracking/column-lineage and /table-lineage) allows querying upstream and downstream lineage for specific tables or columns — this is suitable for integration with external governance tools. (2) system.access.column_lineage is a Unity Catalog system table that records column-level lineage events (source column → target column mappings) captured during query and pipeline execution, enabling SQL-based lineage analysis.

Option A is incorrect: system.access.audit records access and operational events (who ran what, when) but does not contain data lineage mappings. There is no action_name = 'lineageTracked' event type in the audit log.

Option D is incorrect: While system.access.table_lineage does exist as a Unity Catalog system table for table-level lineage, this option pairs it with system.access.column_lineage — which is valid — but table_lineage alone is not the answer for column-level lineage. Option D by itself only addresses table-level lineage and the question asks for column-level. However, system.access.column_lineage in option C already covers the column-level use case, making C+B the best pair.

Option E is incorrect: Delta Lake transaction logs record data operations (adds, removes, schema changes) but do not embed column-level lineage provenance metadata. Lineage is tracked by Unity Catalog's lineage engine, not in the Delta log format itself.

Option C is correct. In the Databricks system.access.audit table, Unity Catalog DDL operations like creating tables are recorded with action_name = 'createTable' (camelCase, matching the UC API action name). The request_params column is a JSON-typed column (or MAP) that can be queried using the colon (:) operator for nested field extraction — request_params:catalogName extracts the catalog name. Filtering by this combination correctly identifies CREATE TABLE events in the gold catalog.

Option D is incorrect: The action_name for table creation in Unity Catalog audit logs is createTable (camelCase), not 'CREATE TABLE' (SQL syntax uppercase). The SQL DDL text is not the audit log action name.

Option B is incorrect: commandSubmit captures notebook/cluster command submissions at the workspace level, not Unity Catalog DDL operations. Parsing SQL text from request_params LIKE '%CREATE TABLE%' is unreliable and would not capture Unity Catalog API-level table creation events.

Option A is incorrect: service_name = 'dataSecurity' is not the correct service name for Unity Catalog operations. The service name for Unity Catalog in audit logs is unityCatalog. Additionally, tableCreate is not the standard action name.

Unity Catalog tags let you classify and label objects/columns (e.g., PII) to aid discovery and governance. Cluster policies (D) govern compute, widgets (C) parameterize notebooks, and COPY INTO (B) ingests data.

Delta time travel allows querying a Delta table's state at a specific point in time using VERSION AS OF or TIMESTAMP AS OF. By reading the table at the desired historical timestamp and overwriting the current table, the rollback is achieved precisely. This is the correct rollback strategy for Delta-based pipelines. Option A (delete and recreate) would work only if a separate backup exists, which is not mentioned, and causes unnecessary downtime. Option C is a fabricated UI feature; DLT does not have a 'Restore Pipeline State' button. Option B is incorrect because a full refresh re-processes source data from scratch using the current pipeline code; reverting the code and running a full refresh would regenerate data using the old logic but would not restore historical data quality — it would reflect the source data as it currently exists.

In Databricks Asset Bundles, variables declared at the top level with a default value are used unless a specific target overrides them in its variables block. The dev target has no variables block, so max_retries resolves to its default value of 1. Option C is wrong—YAML is not parsed last-wins for sibling keys at different levels; target-level variables are scoped to their target. Option A is incorrect; mode: development affects resource naming and permissions (e.g., prepending the user name to job names) but does not reset variable values to zero. Option B is incorrect; variables with defaults are optional to override per target.

The requirement has two parts: drop violating rows AND fail the pipeline if violations exceed a threshold. FAIL UPDATE causes the pipeline to fail when the expectation is violated, which satisfies the hard-stop requirement. However, note that standard DLT does not natively combine DROP ROW with FAIL UPDATE in a single constraint—FAIL UPDATE alone stops the pipeline on any violation, which is the closest native mechanism for a hard breach threshold. Option B uses DROP ROW, which silently removes bad records without failing the pipeline, so the '>5% fail' requirement is not met. Option C requires a non-existent 'alert threshold' configuration that is not a standard DLT feature. Option A is incorrect because omitting ON VIOLATION defaults to WARN (logging the violation) rather than dropping rows or failing.