Microsoft DP-750: Deploy and maintain data pipelines and workloads Practice Questions

When retry_on_timeout is set to true, timeouts are treated like any other failure for retry purposes. The task will be retried up to max_retries (3) additional times, with at least min_retry_interval_millis (60,000 ms = 60 seconds) between each attempt. Without retry_on_timeout: true, a timeout would immediately fail the task without retries. retry_on_timeout does not override max_retries; it simply makes timeouts eligible for the configured retry logic. Timeouts do not have a special retry class that reduces the retry count.

Running OPTIMIZE with ZORDER BY on the date column physically co-locates rows with similar date values in the same files, allowing Delta's file skipping (using min/max statistics) to skip irrelevant files when a date filter is applied. Running ANALYZE TABLE to compute column-level statistics ensures accurate min/max stats are available for the file skipping engine. Repartitioning and reloading all data would work but requires re-ingestion, violating the constraint. VACUUM removes old file versions but does not affect query filtering performance. The delta.stats.collect setting is for controlling whether stats are collected at write time, and re-setting it at query time does not retroactively help existing files without a rewrite. Increasing maxPartitionBytes affects task parallelism, not data scanned.

The large shuffle read (390 GB) indicates a sort-merge join is occurring instead of a broadcast join, and the extreme task duration skew (3s median vs 4min max) indicates data skew on the join key. Broadcasting the 2 GB dimension table eliminates the shuffle entirely, directly addressing the 390 GB shuffle. Adding a salt column distributes skewed keys across multiple partitions, addressing the task duration skew. Increasing shuffle partitions reduces partition size but does not eliminate the shuffle or fix skew. Repartitioning the fact table by join key before a sort-merge join does not eliminate the shuffle cost. Delta caching and increasing executor memory do not address the shuffle or skew root causes.

The cron expression 0 6 * * 1-5 means: at minute 0, hour 6, every day of the month, every month, on Monday through Friday (1-5), which precisely captures 6:00 AM UTC on business days only. A continuous trigger with a 24-hour interval does not support day-of-week filtering. A file arrival trigger is event-based, not time-based, and cannot guarantee 6:00 AM execution on specific days. The 0 6 * * * cron runs every day including weekends; adding a notebook check is a workaround that would still consume cluster resources on weekends to just exit early, and is not the correct configuration-level solution.

The high 'Rows filtered' count after the scan indicates that Delta's file-level statistics (min/max values) are not aligned with the transaction_date filter, so many files pass file pruning but still contain irrelevant rows. Running OPTIMIZE with ZORDER BY (transaction_date) co-locates rows with similar transaction_date values within the same files and updates the Delta statistics, enabling more aggressive data skipping at the file level and reducing rows read. Option A is incorrect because VACUUM removes old, deleted data files to free storage but does not reorganize data or improve query performance for active data. Option D is incorrect because optimizeWrite reduces the number of small files for better write performance, but larger files do not inherently reduce rows read when the issue is statistics alignment. Option C is incorrect because while partitioning by transaction_date would solve the problem, it requires rewriting the entire table and fundamentally changing the table structure, whereas Z-ORDER is a less disruptive optimization. Additionally, the question implies the table already exists and Z-ORDER is in use, making Z-ORDER adjustment on transaction_date the more targeted fix.