A Generative AI engineer deploys a LangGraph-based multi-step RAG agent to a Databricks Model Serving endpoint. After deployment, they observe that the endpoint occasionally returns HTTP 504 (Gateway Timeout) errors during complex multi-hop reasoning tasks. The endpoint is configured with workload_size: Small and scale-to-zero enabled. Which TWO configuration changes would MOST directly address the timeout errors? (Select TWO)
- A. Disable scale-to-zero so that at least one instance is always warm, eliminating cold-start latency that may contribute to timeouts✓ Correct
- B. Enable inference tables on the endpoint to capture request payloads, which automatically increases the timeout threshold
- C. Increase the
max_provisioned_throughputparameter to allow more concurrent token generation per second - D. Switch the endpoint's
workload_typefromCPUtoGPU_SMALLto accelerate the LangGraph orchestration logic - E. Increase the
workload_sizetoMediumorLargeto allocate more CPU and memory, reducing per-request processing time✓ Correct
Explanation
HTTP 504 Gateway Timeout errors on complex multi-hop agent tasks are most commonly caused by (1) insufficient compute resources causing slow per-request execution, and (2) cold-start latency when scale-to-zero is enabled and the first request after a period of inactivity must wait for instance initialization. Increasing workload_size provides more CPU/memory for faster processing, and disabling scale-to-zero eliminates cold starts. Enabling inference tables captures telemetry but has no effect on the timeout threshold. max_provisioned_throughput is a parameter for Foundation Model API endpoints (token throughput), not custom model serving. Switching to GPU for LangGraph orchestration logic (Python control flow) would not meaningfully reduce latency since LangGraph coordination is CPU-bound, not GPU-acceleratable.