Google Vertex AI & Gemini
This domain covers 22% of the ai-intermediate exam and tests your ability to select, configure, and operate Google Cloud's production AI stack. Questions range from choosing the right Vertex AI service for a given architecture requirement, to knowing exact API behaviors, IAM roles, evaluation methods, and tuning strategies. Expect scenario-based questions where two answers seem plausible — success depends on knowing precise service distinctions, default behaviors, and when one approach is strictly preferred over another.
Vertex AI Overview: Model Garden, Workbench, Pipelines, Matching Engine
Vertex AI is Google Cloud's unified ML platform. It consolidates model serving, training, pipelines, and vector search under a single API surface and IAM boundary.
| Component | Purpose | Key Detail |
|---|---|---|
| Model Garden | Browse, deploy, and tune 100+ foundation models | Includes Google, open-source (Llama, Mistral), and third-party models |
| Workbench | Managed JupyterLab notebooks | Two modes: Instances (user-managed) and Managed Notebooks (auto-managed) |
| Pipelines | Orchestrate ML workflows | Based on Kubeflow Pipelines (KFP) v2 SDK |
| Matching Engine | Approximate nearest neighbor (ANN) vector search | Supports dot-product, cosine, L2 distance; index types: TreeAH, Brute Force |
| Feature Store | Serve and share ML features | Batch and online serving |
| Model Registry | Version and track trained models | Links to Endpoints for deployment |
Matching Engine specifics the exam tests:
- Index creation is separate from IndexEndpoint deployment — you create the index, then deploy it to an endpoint.
- Supports streaming updates (upsert/delete without full rebuild) and batch updates.
- TreeAH (Tree + Asymmetric Hashing) is the default for large-scale ANN; Brute Force is exact search for small datasets or ground-truth evaluation.
- Distance measures:
DOT_PRODUCT_DISTANCE,COSINE_DISTANCE,SQUARED_L2_DISTANCE,L1_DISTANCE,MANHATTAN_DISTANCE.
Common traps:
- Confusing Matching Engine with Vertex AI Vector Search — they are the same product (Vector Search is the current product name; Matching Engine was the original name).
- Thinking Workbench Managed Notebooks require manual kernel management — they do not; Google manages the underlying infrastructure.
- Assuming Model Garden only hosts Google models — it hosts open-source and partner models too.
Gemini on Vertex AI: Models, Versions, and Context Caching
Gemini is Google's multimodal model family, available on Vertex AI under the google publisher in Model Garden.
| Model | Best For | Context Window | Key Capability |
|---|---|---|---|
| gemini-1.5-pro | Complex reasoning, long documents | Up to 2M tokens | Best quality, higher latency/cost |
| gemini-1.5-flash | High-volume, low-latency tasks | Up to 1M tokens | Faster and cheaper than Pro |
| gemini-2.0-flash | Next-gen speed + multimodal | Up to 1M tokens | Tool use, real-time streams, image generation |
| gemini-2.0-pro | Frontier reasoning | Up to 2M tokens | Best coding/reasoning in 2.0 family |
Context Caching:
- Context caching stores a portion of the prompt (system instructions, large documents) server-side, so repeated calls don't re-process the same tokens.
- Available for
gemini-1.5-proandgemini-1.5-flash. - Minimum cacheable token count: 32,768 tokens.
- Cache has a configurable TTL (time-to-live); default is 1 hour, max is unspecified but configurable.
- Billed at a reduced input token rate for cache hits; cached tokens cost less than uncached.
- Use case: RAG pipelines where the same large document corpus is queried repeatedly.
Common traps:
- Assuming context caching eliminates output token costs — it only reduces input token cost for the cached portion.
- Confusing context window size with context caching — context window is the max tokens per call; caching is about reusing tokens across calls.
- Choosing gemini-1.5-pro when cost/latency is the primary constraint — flash is the correct answer for high-throughput, cost-sensitive workloads.
Worked scenario: A customer service platform queries the same 500-page product manual with every user request. The correct architecture uses gemini-1.5-flash with context caching on the product manual, reducing per-request token cost while maintaining low latency.
Unlock with a paid plan
11 more sections (about 2,845 words) continue below. Every paid plan unlocks all AI Intermediate study guides, the full question bank and mock exams, from $4.97/month.