Optimize GenAI Systems and Model Performance
This domain accounts for 14% of the AI-300 exam and tests your ability to design, evaluate, and optimize Retrieval-Augmented Generation (RAG) pipelines and fine-tuned models on Azure. Expect scenario-based questions where you must select the correct retrieval strategy, evaluation metric, embedding model, or fine-tuning approach given specific constraints. The exam emphasizes Azure AI Search, Azure OpenAI Service, and Azure AI Foundry as the primary platforms.
Tuning Similarity Thresholds, Chunk Sizes, and Retrieval Strategies
Core Concepts
Similarity threshold controls the minimum relevance score a retrieved document must achieve before being passed to the language model. Setting it too low floods the context with noise; too high starves the model of useful context.
Chunk size determines how source documents are split before indexing. Chunk size has a direct tradeoff:
| Chunk Size | Advantage | Disadvantage |
|---|---|---|
| Small (128–256 tokens) | High precision, focused answers | May lose surrounding context |
| Medium (512 tokens) | Balanced recall and precision | Default starting point |
| Large (1024+ tokens) | Retains more context per chunk | Dilutes relevance score, slower retrieval |
Chunk overlap (typically 10–20% of chunk size) prevents information loss at boundaries. In Azure AI Search, chunking is configured in the skillset or through the integrated vectorization pipeline.
Retrieval strategies in Azure AI Search:
| Strategy | Mechanism | Best For |
|---|---|---|
| Vector search | cosine/dot-product similarity on embeddings | Semantic intent matching |
| Keyword (BM25) | term frequency-inverse document frequency | Exact term, product codes, IDs |
| Hybrid search | RRF fusion of vector + BM25 scores | Most production RAG scenarios |
| Semantic ranker | Cross-encoder reranking on top-k results | Highest precision, adds latency |
Top-k (number of retrieved chunks passed to the LLM) directly affects context window usage and cost. Azure AI Search default top-k is 50 for index queries but RAG pipelines typically use 3–5 chunks.
Reciprocal Rank Fusion (RRF) is the default fusion algorithm when hybrid search is enabled in Azure AI Search. RRF score = Σ 1/(k + rank_i) where k=60 by default.
Common Traps
- Confusing similarity threshold (a post-retrieval filter) with top-k (a count limit)
- Assuming larger chunks always improve answer quality—they dilute embedding specificity
- Forgetting that semantic ranker requires Standard S1 or higher tier in Azure AI Search (not free or Basic)
- Setting threshold too high in low-data domains causes zero results without fallback
- Overlap does not double-count tokens; it prevents boundary truncation
Worked Scenario
Requirement: A customer support RAG system retrieves irrelevant chunks when users ask about specific product model numbers. Vector search alone is failing.
Reasoning: Product model numbers are lexical tokens that embeddings may not distinguish well. The correct fix is hybrid search (vector + BM25), which ensures exact keyword matches surface alongside semantic results. Adding a semantic ranker as a second-stage reranker further improves precision without changing the retrieval count.
Embedding Model Selection and Fine-Tuning for Domains
Core Concepts
Embedding models convert text into dense vectors. Selection depends on domain vocabulary, vector dimensionality, and latency requirements.
| Model | Provider | Dimensions | Max Tokens | Notes |
|---|---|---|---|---|
| text-embedding-3-small | Azure OpenAI | 1536 (reducible) | 8191 | Cost-efficient, supports dimension reduction |
| text-embedding-3-large | Azure OpenAI | 3072 (reducible) | 8191 | Highest accuracy, higher cost |
| text-embedding-ada-002 | Azure OpenAI | 1536 | 8191 | Legacy; prefer text-embedding-3-x for new builds |
| E5 / BGE models | HuggingFace (via AI Foundry) | 768–1024 | Varies | Fine-tunable for domain adaptation |
Dimension reduction in text-embedding-3 models uses Matryoshka Representation Learning (MRL)—you can truncate vectors to smaller sizes (e.g., 256) with minimal accuracy loss, reducing storage and query cost.
Domain fine-tuning of embedding models is done when out-of-the-box models underperform on specialized vocabulary (legal, medical, financial). The process uses contrastive learning with positive/negative pairs:
- Create anchor–positive–negative triplets from domain data
- Fine-tune using Azure AI Foundry custom model training or open-source frameworks (sentence-transformers)
- Deploy fine-tuned embedding model to a managed endpoint
- Re-index all documents with the new model (index must be rebuilt)
Critical: Changing an embedding model requires re-embedding and re-indexing all documents. You cannot mix vectors from different models in the same Azure AI Search vector field.
Common Traps
- Using ada-002 for new projects—exam will use it as a distractor; correct answer is text-embedding-3-small or text-embedding-3-large
- Forgetting that dimension reduction is a model parameter set at query time and index time, not a post-processing filter
- Assuming you can fine-tune text-embedding-3 models directly in Azure OpenAI Service—you cannot; only completions models support Azure OpenAI fine-tuning
- Mixing embedding models across shards of the same index—not supported; vectors must be homogeneous
Unlock with a paid plan
8 more sections (about 2,128 words) continue below. Every paid plan unlocks all AI-300 study guides, the full question bank and mock exams, from $4.97/month.