Google Cloud Professional Data Engineer: Designing data processing systems Practice Questions

Option 1 is correct. Stateful ParDo with a bloom filter (or dedup state) can deduplicate events in Dataflow before writing to BigQuery, ensuring clean data upstream. This is the most reliable approach. Option 2 is partially viable but INSERT OR IGNORE is not a native BigQuery operation (you'd need MERGE or upserts via DML, which is slower). Option 3 is misleading; exactly-once processing in Dataflow does not deduplicate application-level duplicates (sensor retries)—it only prevents framework-level duplication. Option 4 is poor practice; deduplicating during queries is inefficient (slower analytics) and masks data quality issues.

Option 1 is correct: The DLP API is a detection and classification service, not an encryption or key management service. It identifies sensitive data but doesn't encrypt it or manage keys. Encryption is handled separately via CMEK, TDE (Transparent Data Encryption), or other mechanisms. The DLP API's role is to find sensitive data so you can decide how to protect it, not to encrypt it directly.

Option 0 is incorrect: This is exactly what the DLP API does—it has pre-built and custom infoTypes to detect SSNs, medical record numbers, phone numbers, credit card numbers, and other PII.

Option 2 is incorrect: DLP API can inspect BigQuery tables and Cloud Storage files directly using built-in connectors. Classification and inspection are core DLP features.

Option 3 is incorrect: The DLP API can generate de-identification transformations including redaction rules (replacing sensitive values with 'REDACTED'), masking, and bucketing. It also supports custom transformation rules. This is a standard DLP capability.

Option 2 is correct because on-demand cluster creation eliminates idle compute costs outside the 2-hour window; Cloud Scheduler can trigger cluster creation and teardown automatically. Option 1 fails the cost requirement—even with 1 minimum node running 22 hours daily unnecessarily increases costs. Option 3 is feasible but less efficient; BigQuery's federated queries on Cloud Storage are slower for complex Spark transformations and can be more expensive at scale. Option 4 is reasonable but Dataflow has higher base costs than Dataproc for batch; Dataflow excels at streaming and autoscaling, but on-demand cluster creation is more cost-effective here.

Option 1 is correct. This row key design ensures that queries filtering by tenant and metric type hit a contiguous range of rows (good for Bigtable's range-based retrieval), and reverse_timestamp (e.g., Long.MAX_VALUE - timestamp) groups recent events together, avoiding hot spots. Option 2 is suboptimal because leading with reverse_timestamp causes hot spots during writes (all recent events concentrate on a few tablets). Option 3 is poor because metric_type (likely low cardinality) at the front creates uneven distribution and scattered queries. Option 4 uses sequential timestamp which again causes hot spots on the tablet containing the current time range.

Option 0 is correct: BigQuery snapshots are read-only logical copies; they cannot be directly queried or restored. To recover, you must clone the snapshot into a new table using bq cp --clone, which requires a copy operation. For large tables (e.g., 100+ GB), this operation may take minutes to hours depending on the table size and system load. The DR plan should account for this recovery time in the RTO calculation. This is a realistic constraint of BigQuery's snapshot model.

Option 1 is incorrect: Snapshots can be moved (referenced) across projects using cross-project snapshot sharing. The source and target projects must have proper IAM permissions, but it's technically possible. The limitation isn't movement but the clone operation required for recovery.

Option 2 is incorrect: Snapshots don't expire after 24 hours in different projects. They persist for the retention period (30 days in this case), regardless of project boundaries. Storing in Cloud Storage is an alternative but adds complexity (serialization/deserialization costs and latency).

Option 3 is incorrect: While snapshots can be shared via cross-project permissions, requiring 'bigquery.admin' on the source project is overly permissive. Sharing snapshots requires 'bigquery.dataEditor' on the dataset containing the snapshot. The row count validation is good practice but isn't the primary limitation to address.