Professional Data Engineer · 18% of the exam

Maintaining and automating workloads: free practice questions

5 sample questions from our 45-question bank for this domain — answers and explanations included. These are the same scenario-based style as the real Google Cloud exam.

1. A Cloud Workflows execution that orchestrates a multi-step data processing pipeline is failing with a QuotaExceeded error when calling the BigQuery Jobs API. The pipeline runs several times per day. What is the MOST appropriate way to handle this error within the Workflows definition?

  • A. Add a try/except block in the workflow YAML that catches HTTP 429 errors, waits using sys.sleep with exponential backoff, and retries the BigQuery API call up to a configured maximum number of attempts.✓ Correct
  • B. Increase the BigQuery Jobs API quota in the Google Cloud Console and redeploy the workflow without any error-handling changes.
  • C. Switch from the BigQuery Jobs API to the BigQuery Storage Write API to avoid the quota limit.
  • D. Add a parallel branch in the workflow that catches the error and immediately sends a Pub/Sub notification to an ops team, halting the workflow execution.
Explanation

Cloud Workflows natively supports try/except error handling and the sys.sleep step, making exponential backoff retry logic the idiomatic and correct solution for transient quota errors (HTTP 429 / QuotaExceeded). This is the recommended pattern in Google's Workflows documentation. Option B (quota increase) may be appropriate as a long-term fix but does not handle the error programmatically within the workflow — the next burst could still trigger the error. Option C confuses two unrelated APIs; the BigQuery Storage Write API is for streaming data ingestion, not for submitting query jobs, so it does not address the Jobs API quota. Option D notifies humans and halts execution, which is appropriate for unrecoverable errors but incorrect for a transient, retryable quota error — it introduces unnecessary manual intervention.

2. A junior data engineer accidentally deployed a broken version of a Cloud Composer DAG that has been writing malformed records into a BigQuery table for the past 3 hours. The correct DAG version is in your Git repository. You need to: (1) stop the broken DAG immediately, (2) restore the correct DAG with minimal downtime, and (3) re-process only the affected 3-hour window. What sequence of steps should you follow?

  • A. Pause the broken DAG in the Airflow UI → deploy the correct DAG file to the Composer environment's Cloud Storage /dags bucket → delete the malformed rows from BigQuery using a DML statement → manually trigger a DAG run with the corrected logical date range.✓ Correct
  • B. Delete the Cloud Composer environment → recreate it → redeploy all DAGs from Git → trigger a full historical backfill from the beginning.
  • C. Pause the broken DAG → revert the DAG file in Cloud Storage → run `gcloud composer environments run` to trigger a backfill using the correct date range.
  • D. Mark all running task instances as failed in the Airflow UI → redeploy the corrected DAG from Git to Cloud Storage → trigger a manual DAG run for the last 24 hours to ensure all data is refreshed.
Explanation

Option A is the correct, minimal-impact sequence: pausing the DAG immediately halts further writes; replacing the DAG file in the /dags GCS bucket automatically syncs the corrected version to Airflow; deleting only the malformed rows via DML surgically corrects the data; and triggering a targeted DAG run for the 3-hour window re-processes only the affected data. Option B (deleting the environment) is destructive, time-consuming, and far exceeds the scope of the problem. Option C is close but omits the critical step of removing the malformed data from BigQuery before re-running; re-running without cleaning the table would append duplicate or compounding errors. Option D marks tasks as failed rather than pausing (which may allow in-flight runs to continue writing) and triggers a 24-hour re-run, which is broader than necessary and could reprocess already-correct data.

3. Your organization has multiple Dataflow jobs that periodically fail due to hitting downstream BigQuery write quotas. You want a long-term solution that scales automatically with increased data volume. Which combination of approaches would best address this? A) Increase the BigQuery slot reservation to guarantee write capacity and implement exponential backoff in Dataflow's BigQueryIO write transform B) Set up Cloud Monitoring alerts for quota-related failures and manually scale slot reservations when alerts trigger C) Batch write operations in Dataflow using micro-batching and implement a Pub/Sub intermediate queue between Dataflow and BigQuery D) Use BigQuery Streaming Inserts instead of batch loads to avoid quota limits E) Implement Dataflow's windowing and trigger strategies to stagger writes, combined with monitoring of BigQuery slot usage

  • A. Increase the BigQuery slot reservation to guarantee write capacity and implement exponential backoff in Dataflow's BigQueryIO write transform✓ Correct
  • B. Set up Cloud Monitoring alerts for quota-related failures and manually scale slot reservations when alerts trigger
  • C. Batch write operations in Dataflow using micro-batching and implement a Pub/Sub intermediate queue between Dataflow and BigQuery
  • D. Use BigQuery Streaming Inserts instead of batch loads to avoid quota limits
  • E. Implement Dataflow's windowing and trigger strategies to stagger writes, combined with monitoring of BigQuery slot usage✓ Correct
Explanation

The correct answers are A and E. Option A directly addresses quota limits by guaranteeing write capacity through slot reservations, while exponential backoff handles transient failures. Option E (windowing/triggers + monitoring) controls write concurrency and provides visibility into usage, preventing quota saturation. Together, they provide both capacity and intelligent throttling. Option B is reactive, not scalable. Option C adds unnecessary complexity with Pub/Sub. Option D (Streaming Inserts) doesn't avoid quota limits—they have different quotas and can still be exceeded, and they're not appropriate for all use cases.

4. Your company stores BigQuery tables partitioned by ingestion date. After reviewing storage costs, you discover that many partitions older than 90 days contain data that is queried less than once per month. You want to reduce storage costs while maintaining query access to historical data. What is the MOST cost-effective solution?

  • A. Set a partition expiration of 90 days on the table to automatically delete old partitions.
  • B. Enable long-term storage pricing by ensuring partitions are not modified for 90 consecutive days, and avoid unnecessary table updates on old partitions.✓ Correct
  • C. Export partitions older than 90 days to Cloud Storage as Avro files and delete them from BigQuery, then use BigQuery external tables to query the archived data.
  • D. Convert the table to a non-partitioned table and apply a table expiration so the entire table is replaced every 90 days.
Explanation

BigQuery automatically applies long-term storage pricing (approximately 50% discount) to any table or partition that has not been modified in 90 consecutive days. The key action is to avoid unnecessary writes or updates to old partitions (e.g., not running UPDATE/MERGE on them), which would reset the 90-day clock. No configuration change is needed — simply letting unmodified partitions age unlocks the discount while keeping full query access. Option A (90-day partition expiration) would DELETE the data, not retain it, which violates the requirement of maintaining query access. Option C moves data to Cloud Storage, which reduces BigQuery storage costs further but adds complexity, ETL overhead, and external table query performance drawbacks — not 'most cost-effective' given long-term pricing already achieves a 50% reduction with no operational change. Option D destroys historical data by replacing the table every 90 days.

5. You manage a Cloud Composer DAG that processes customer data and writes results to BigQuery. A data quality check task fails periodically when input data is malformed. Instead of failing the entire DAG, you want to trigger a notification and continue with downstream tasks using a placeholder result. Which error handling pattern should you implement? A) Set `trigger_rule='all_done'` on downstream tasks so they execute regardless of upstream failure B) Use `trigger_rule='none_skipped'` and wrap the failing task in a try-except block C) Set `trigger_rule='none_failed'` on the quality check task to prevent it from stopping the DAG D) Create a custom sensor that polls for data quality status and uses `trigger_rule='none_failed_or_skipped'` with a fallback BranchPythonOperator

  • A. Set `trigger_rule='all_done'` on downstream tasks so they execute regardless of upstream failure✓ Correct
  • B. Use `trigger_rule='none_skipped'` and wrap the failing task in a try-except block
  • C. Set `trigger_rule='none_failed'` on the quality check task to prevent it from stopping the DAG
  • D. Create a custom sensor that polls for data quality status and uses `trigger_rule='none_failed_or_skipped'` with a fallback BranchPythonOperator
Explanation

The correct answer is A. The `trigger_rule='all_done'` allows downstream tasks to run regardless of whether upstream tasks succeed or fail. Combined with an alerting task (e.g., sending a notification), this pattern lets the pipeline continue with a fallback path. Option B's `none_skipped` rule skips tasks only if previous tasks were skipped, not failed. Option C sets the rule on the failing task, not downstream—this doesn't solve the problem. Option D is overcomplicated; it uses sensor logic unnecessarily. A simple `trigger_rule='all_done'` on downstream tasks is the standard Airflow pattern for this scenario.

40 more questions in this domain

Practice the full bank with instant grading, flashcards, and a timed mock exam.

Start practicing free
Maintaining and automating workloads — Free Professional Data Engineer Practice Questions | DataCertPrep — Certification Prep