Microsoft DP-750: Set up and configure an Azure Databricks environment Practice Questions

A Volume is the Unity Catalog object designed to store and govern files (non-tabular data) such as raw files, notebooks, or model artifacts. It sits within the three-level namespace (catalog.schema.volume) and allows file-based access. An External table represents structured tabular data backed by files in external storage, not a general-purpose file store. A Schema is a namespace container that holds tables, views, and volumes—it is not itself a file store. A Materialized view is a precomputed result set of a query refreshed on a schedule, not a file-storage object.

After creating a connection object in Unity Catalog, the next step is to create a foreign catalog using CREATE FOREIGN CATALOG ... USING <connection_name>. The foreign catalog mirrors the remote database's schemas and tables inside Unity Catalog, making them queryable with standard SQL and governed by Unity Catalog permissions. — External tables reference files in object storage, not relational database connections; the connection object cannot be used as a LOCATION credential. — External locations map a storage credential to a cloud storage path (ABFSS/GCS/S3); they are not related to database connections. — Using JDBC directly via spark.read bypasses Unity Catalog governance entirely and does not make the tables visible in the catalog namespace; it also requires manual credential management.

The Auto Termination setting is a first-class field in the Databricks cluster configuration UI and API. Setting it to 20 minutes causes the cluster to shut down automatically after 20 consecutive minutes of inactivity. — Auto-termination is not found under Advanced Options > Spark; those fields are for custom Spark configuration properties, not cluster lifecycle settings. — maxRetainedForReuse is a pool-level setting that controls how long idle instances are retained in an instance pool, not a per-cluster auto-termination setting. — There is no supported mechanism to add a termination script as a library; cluster termination is managed through the platform configuration, not user-defined scripts.

In Unity Catalog, a three-level namespace (catalog.schema.table) is required. When using a two-level reference (schema.table), Databricks resolves the catalog from the session's default catalog. If the default catalog is not set to corp_prod, the table won't be found. More specifically, even if the table exists, the engineer must have USE CATALOG on corp_prod AND USE SCHEMA on finance in addition to SELECT on the table—without USE CATALOG, the catalog cannot be traversed. An expired storage credential would produce a storage access error, not a table-not-found error. If the notebook used the legacy Hive metastore, Unity Catalog tables in corp_prod simply would not be resolvable, but the root cause is still the missing catalog context/privilege. MODIFY privilege is for write operations; SELECT alone is sufficient for reads—the issue is navigation privileges, not read privileges.

Photon is a vectorized C++ query engine that accelerates SQL operations on Delta tables. It provides the most significant gains for (1) complex SQL aggregations and window functions, which are highly amenable to vectorized execution, and (2) Delta table scan operations with predicate pushdown and file pruning, which Photon optimizes through native vectorized I/O. — Python UDFs executed row-by-row are not vectorized and run in a Python process; Photon cannot accelerate them because they leave the JVM/Photon execution path. To benefit from Photon, UDFs should be rewritten as built-in SQL functions or Pandas UDFs (vectorized). — Auto Loader's primary cost is file discovery (cloud storage listing) and Spark structured streaming overhead; Photon can accelerate the processing of records after ingestion, but the file discovery and trigger mechanisms themselves are not the Photon-accelerated portion. — scikit-learn model training runs as a single-threaded or multiprocessed Python workload on the driver node; Photon does not accelerate driver-side Python computation.