Data Ingestion and Loading – DE Associate Study Guide
This domain accounts for 21% of the exam and covers getting data into the lakehouse: Auto Loader, COPY INTO, reading files, and creating Delta tables.
Auto Loader
Auto Loader (the cloudFiles source) incrementally ingests new files as they arrive in cloud storage, tracking which files it has already processed. It scales to millions of files via directory listing or file notification mode, and supports schema inference and schema evolution (automatically adding newly appearing columns). Use it for continuously arriving files.
Unlock with a paid plan
4 more sections (about 136 words) continue below. Every paid plan unlocks all Data Engineer Associate study guides, the full question bank and mock exams, from $4.97/month.