Google Cloud Professional Machine Learning Engineer · 20% practice allocation

Google Cloud Professional Machine Learning Engineer: Serving and scaling models Practice Questions

5 free sample questions from the Serving and scaling models domain, with answers and explanations. The full Professional Machine Learning Engineer bank has 251 questions across 6 domains. Questions are scenario-based and written from the official Google Cloud exam guide.

You are preparing to scale a Vertex AI endpoint that serves a real-time recommendation model. The endpoint currently processes 1,000 requests per second with an average latency of 150 ms, but traffic is expected to spike to 5,000 requests per second during peak shopping hours. Your model uses a single GPU and batch processing is not viable. Which two actions should you take to handle this scale?

A. Switch to batch prediction mode and queue requests in Pub/Sub to be processed asynchronously
B. Increase the request timeout threshold to allow more time for processing during peak hours
C. Deploy multiple replicas across different machine types and configure autoscaling based on CPU and memory utilization✓ Correct
D. Deploy the model on Cloud Run instead of Vertex AI to automatically scale with containerized workloads
E. Implement model quantization to reduce model size and increase throughput on the existing hardware
F. Enable autoscaling on the Vertex AI endpoint and increase the machine type to accommodate more GPU memory✓ Correct

Explanation

The correct answers are enabling autoscaling (0) and deploying multiple replicas with autoscaling (4). These directly address the need to handle a 5x traffic increase for real-time serving. Autoscaling allows the endpoint to dynamically add/remove replicas based on metrics like CPU, GPU utilization, and request queue depth, which is essential for handling traffic spikes. Deploying multiple replicas across different machine types provides redundancy and better resource utilization. (1) is incorrect because Cloud Run is better for request-driven workloads but may not be ideal for sustained GPU-based inference at this scale. (2) Quantization can help with latency but alone won't solve capacity issues. (3) Batch prediction contradicts the real-time requirement. (5) Increasing timeout doesn't address the root capacity problem.

You deployed two versions of a churn prediction model on a Vertex AI endpoint using traffic splitting: 90% to Model A (baseline logistic regression, 15ms latency) and 10% to Model B (new gradient boosting, 45ms latency). After one week, you observe that Model B achieves 3% higher recall for at-risk customers, but the p-value is 0.08 (not statistically significant at 0.05 level). What is the recommended next step?

A. Accept Model B as production model since it provides business value even without statistical significance
B. Immediately shift 50% traffic to Model B since it shows higher recall regardless of p-value
C. Roll back Model B completely and retrain it on additional data
D. Continue the A/B test with current traffic split for another 2-3 weeks to accumulate statistical power✓ Correct

Explanation

Option 1 is correct. The p-value of 0.08 indicates the result lacks sufficient statistical power at the chosen α=0.05 threshold. Continuing the test allows accumulation of more observations to potentially achieve significance. Option 0 violates statistical rigor (premature decisions lead to false positives). Option 2 assumes the model is flawed rather than under-powered statistically. Option 3 ignores statistical requirements for production decisions in ML systems.

You are serving a large language model (LLM) for code completion on a Vertex AI endpoint. Inference latency is critical—developers need responses in <500 ms. The model is 12 GB and currently runs on a single GPU per replica. You observe that 60% of requests hit 800+ ms latency, mostly due to compute time. Which optimization would provide the fastest improvement?

A. Add more GPU replicas and use load balancing to distribute requests, reducing queuing delay
B. Apply 8-bit quantization to reduce model size and memory bandwidth overhead✓ Correct
C. Migrate the LLM to Vertex AI Generative AI API which has native optimizations
D. Implement batch prediction with a 100ms accumulation window to maximize throughput per compute cycle

Explanation

8-bit quantization reduces the model from 12 GB to ~3 GB, significantly improving GPU memory bandwidth and reducing compute time per inference. This directly addresses the 800+ ms latency bottleneck. Option 0 addresses queuing, not compute latency. Option 2 (batch prediction) adds latency by design and is meant for throughput, not latency-sensitive workloads. Option 3 moves the problem rather than solving it. Quantization is the most direct path to sub-500ms latency.

You are deploying a knowledge distillation setup on Vertex AI where a large teacher model (500 MB, 800 ms inference) trains a smaller student model (20 MB, 50 ms inference). After distillation, the student model achieves 96% of the teacher's accuracy. For production serving, which deployment strategy minimizes cost while maintaining quality?

A. Serve the student model for 95% of traffic and the teacher for 5%; use the teacher to re-distill the student quarterly to maintain knowledge currency
B. Serve only the student model, because distillation already achieved acceptable accuracy and cost is minimized✓ Correct
C. Serve the teacher model only; distillation is just a training technique and serves no purpose after training is complete
D. Serve both teacher and student; use traffic splitting to compare them continuously, then deprecate the teacher once the student is proven better

Explanation

Option 0 (correct): Distillation is complete before serving. The student achieves 96% of teacher accuracy with 16x lower latency and 25x smaller size—it is the superior serving choice. Continuing to serve the teacher wastes resources. Option 1 is unnecessary once distillation is validated. Option 2 is incorrect—distillation is not a training-time-only concept; the student model is the artifact used in production. Option 3 quarterly re-distillation adds operational overhead without clear benefit if the student is already performing well.

Your organization is deploying a large language model (LLM) for code completion on a Vertex AI endpoint. The model has 13 billion parameters and requires 40 GB of GPU memory. Current inference latency is 800 ms per request. Users report that the experience is sluggish. You are considering three approaches: (A) deploying on a larger GPU (H100 vs current V100), (B) applying knowledge distillation to create a 3 billion parameter student model, and (C) implementing request batching with a 100 ms maximum wait time. Which two approaches would best reduce user-perceived latency for this interactive application?

A. Apply knowledge distillation to create a smaller student model with latency around 150–200 ms, even if it slightly reduces code completion quality✓ Correct
B. Implement request batching with a 100 ms maximum wait time to process multiple requests in parallel
C. Keep the current setup but implement a client-side cache to avoid redundant requests to the endpoint
D. Deploy on H100 GPUs, which have 3x higher memory bandwidth and can reduce latency to approximately 250–300 ms✓ Correct
E. Deploy the model on Cloud Run with a custom serving container to reduce infrastructure overhead
F. Quantize the model to INT8 to reduce memory footprint and increase throughput

Explanation

The correct answers are (0) and (1). For an interactive code completion use case, both approaches directly reduce latency: H100 deployment addresses the hardware bottleneck with 3x memory bandwidth, reducing latency significantly, and knowledge distillation produces a substantially faster model (150–200 ms vs 800 ms) that users will perceive as more responsive. (2) Request batching is counterproductive for interactive applications because introducing a 100 ms wait time increases perceived latency, defeating the purpose of the optimization. (3) Quantization helps but is less impactful than GPU upgrade or distillation for this scale of model. (4) Caching only helps with repeated requests, not the latency of unique queries. (5) Cloud Run doesn't provide performance benefits over Vertex AI for this use case and adds operational complexity.

Test yourself across every Professional Machine Learning Engineer domain

The free Professional Machine Learning Engineer practice test mixes about 20 questions weighted like the exam. The full 251-question bank adds instant grading, flashcards and a timed mock exam.

Take the free Professional Machine Learning Engineer practice test Practice the full bank

Study this domain

Serving and scaling models study guide (Professional Machine Learning Engineer) →

Other Professional Machine Learning Engineer domains

← All Google Cloud Professional Machine Learning Engineer study resources