AI Advanced · 18% practice allocation

AI Advanced: Production AI Architecture Practice Questions

5 free sample questions from the Production AI Architecture domain, with answers and explanations. The full AI Advanced bank has 794 questions across 5 domains. Questions follow the AI Advanced course content.

1.

An AI product team is using the Vercel AI SDK's useChat hook to build a chat interface. They want to display a typing indicator while the assistant is generating a response and hide it once streaming is complete. Which property of the useChat hook should they use to drive this behavior?

  • A. streamStatus — it exposes 'streaming', 'idle', and 'error' states that can be used directly.
  • B. messages[messages.length - 1].isStreaming — each message object tracks its own streaming status.
  • C. isLoading — it is true while a request is in-flight or streaming, and returns to false when the stream ends.✓ Correct
  • D. isPending — it transitions to true after the first token arrives and remains true until the full response is committed.
Explanation

useChat exposes an isLoading boolean that is true from the moment a request is submitted until the streaming response is fully received, making it the correct property to show/hide a typing indicator. B is wrong — isPending is not a property exposed by the Vercel AI SDK's useChat hook. C is wrong — there is no streamStatus property in the useChat API. D is wrong — individual message objects in the useChat messages array do not carry an isStreaming field; streaming state is tracked at the hook level.

2.

A team is modeling the monthly cost of a new AI feature. The workload is 200,000 requests/month. Average input is 800 tokens and average output is 300 tokens per request. They plan to use GPT-4o at $2.50/1M input tokens and $10.00/1M output tokens. The team's architect suggests that enabling OpenAI prompt caching (assuming 60% of input tokens are cacheable prefix and get a 50% discount) will significantly reduce costs. What is the estimated monthly cost WITH prompt caching enabled?

  • A. Approximately $880✓ Correct
  • B. Approximately $700
  • C. Approximately $640
  • D. Approximately $760
Explanation

Without caching: input 200,000 × 800 = 160M tokens × $2.50/1M = $400; output 200,000 × 300 = 60M × $10.00/1M = $600; total $1,000. With caching, 60% of input (96M tokens) is billed at half price: 96M × $1.25/1M = $120, and the other 64M at full price: 64M × $2.50/1M = $160. Input becomes $280, output stays $600, so the total is about $880 (a $120 saving). $760, $700 and $640 all assume savings larger than caching 60% of input at 50% off can produce.

3.

A team exposes a vLLM server internally using its OpenAI-compatible API (/v1/chat/completions). A developer tries to connect an existing OpenAI Python SDK client to the vLLM endpoint by changing only the base_url. Which additional change is REQUIRED for the client to authenticate and route correctly to vLLM?

  • A. Replace the openai.ChatCompletion.create() method with openai.Completion.create() because vLLM only supports the legacy completions endpoint, not the chat completions endpoint.
  • B. Add a custom X-VLLM-Auth header with the server's internal token; vLLM rejects requests without this proprietary header even when no --api-key flag was set.
  • C. Disable SSL verification in the SDK client because vLLM only serves HTTP and the SDK enforces HTTPS by default, causing all requests to fail.
  • D. Set api_key to any non-empty string (e.g., 'vllm'). vLLM's OpenAI-compatible server does not validate API keys by default but the OpenAI SDK requires the field to be non-null.✓ Correct
Explanation

The OpenAI Python SDK requires the api_key parameter to be set and non-empty or it raises a validation error before sending the request. Since vLLM by default does not enforce API key validation (unless --api-key is passed at startup), any non-empty placeholder string satisfies the SDK requirement. B is wrong — the OpenAI SDK does not enforce HTTPS; you can point base_url to an HTTP endpoint without disabling SSL verification. C is wrong — vLLM fully supports the /v1/chat/completions endpoint, which mirrors OpenAI's chat completions API. D is wrong — there is no X-VLLM-Auth proprietary header in vLLM; authentication is handled via the standard Authorization: Bearer <token> header, and only when --api-key is configured at startup.

4.

An engineering team is optimizing costs for a document-summarization pipeline using the Anthropic API. Each request sends the same 8,000-token reference document as part of the system prompt, followed by a short user instruction that changes per request. They enable Anthropic's prompt caching feature. What is the MINIMUM number of tokens a prompt prefix must contain to be eligible for caching under Anthropic's rules?

  • A. 8,192 tokens
  • B. 2,048 tokens
  • C. 1,024 tokens
  • D. 4,096 tokens✓ Correct
Explanation

Anthropic requires prompt cache prefixes to be at least 1,024 tokens for Claude Haiku and at least 2,048 tokens for Claude Sonnet and Opus — however, the commonly documented minimum for the general prompt caching feature across the main Claude 3 model line (Sonnet/Opus) is 2,048 tokens, with the threshold widely cited in their documentation as 1,024 tokens for Haiku. The question reflects the cross-model threshold of 1,024 tokens as a floor, but the correct answer here is 1,024. NOTE — actually, per Anthropic's published documentation, the minimum cacheable prefix is 1,024 tokens (for Claude Haiku) and 2,048 tokens for Sonnet/Opus. The most broadly applicable minimum is 1,024 tokens. However, given that this is a scenario about a large 8,000-token system prompt, the team's setup easily clears any threshold. The correct answer is 1,024 tokens as the minimum floor.

5.

A team is self-hosting Llama 3 70B using vLLM on a single NVIDIA A100 80GB GPU for internal API use. They are seeing out-of-memory (OOM) errors under moderate load. Which TWO actions are MOST likely to resolve the OOM errors while keeping the model self-hosted? (Select TWO)

  • A. Increase the HTTP request timeout so that slower inference does not trigger OOM errors
  • B. Reduce the max_model_len (maximum context length) parameter in vLLM to lower KV-cache memory consumption✓ Correct
  • C. Switch from FP16 to FP32 precision to improve numerical stability and reduce memory pressure
  • D. Load the full model in FP16 and disable KV-cache entirely to free GPU memory
  • E. Enable tensor parallelism across multiple GPUs to distribute the model's weight memory footprint✓ Correct
Explanation

Option E is correct: tensor parallelism (e.g., --tensor-parallel-size 2 with two GPUs) shards the model weights across multiple devices, directly reducing per-GPU memory usage — ideal for a 70B model that is large relative to a single A100's 80GB capacity. Option B is correct: vLLM pre-allocates KV-cache proportional to max_model_len; reducing this value (e.g., from 8192 to 2048 tokens) frees substantial GPU memory, often resolving OOM without changing hardware. Option C is wrong — FP32 uses twice the memory of FP16 and would worsen OOM errors, not improve them. Option A is wrong — HTTP timeouts are a networking concern and have no relationship to GPU memory allocation. Option D is wrong — disabling the KV-cache entirely would make vLLM non-functional for batched inference; the KV-cache is fundamental to vLLM's PagedAttention architecture.

Test yourself across every AI Advanced domain

The free AI Advanced practice test mixes about 20 questions weighted like the exam. The full 794-question bank adds instant grading, flashcards and a timed mock exam.

Take the free AI Advanced practice testPractice the full bank

Study this domain

Production AI Architecture study guide (AI Advanced) →

Other AI Advanced domains

← All AI Advanced study resources