Developing Code for Data Processing using Python and SQL Study Guide — Data Engineer Professional

Developing Code for Data Processing using Python and SQL – DE Professional Study Guide

This is the largest domain (22% of the exam) and covers writing production-grade data processing code on Databricks with PySpark and Spark SQL. The Professional exam expects you to reason about correctness, performance, and maintainability — not just syntax.

Delta DML and Upserts

• MERGE INTO — atomic upsert: update matched rows, insert unmatched, optionally delete — in one transaction. The standard way to apply change batches.
• DELETE / UPDATE — transactional row operations recorded in the Delta log. Never edit underlying Parquet files directly.