Implement Generative AI and Agentic Solutions
This domain covers building production-grade generative AI applications and agents using Azure AI Foundry (formerly Azure AI Studio) and Azure OpenAI Service. It accounts for 33% of the AI-103 exam, making it the single heaviest-weighted domain. Questions test your ability to select the right architecture, configure the right parameters, and reason about tradeoffs between approaches such as RAG vs. fine-tuning, vector vs. hybrid search, and streaming vs. synchronous calls — plus, from the merged agent-building content below, designing and operating Foundry Agent Service agents. Expect scenario-based questions requiring you to justify a design choice, not just recall a definition.
Chat Completions, System Prompts, and Prompt Engineering Patterns
Core Concepts
Chat completions is the primary API pattern for conversational AI on Azure OpenAI. The request body contains a messages array with roles: system, user, and assistant. The system message is the first message in the array and sets the model's persona, boundaries, and behavior for the entire conversation.
| Role | Purpose | Position in messages array |
|---|---|---|
system | Persona, rules, context, safety guardrails | First (index 0) |
user | Human turn input | Alternating |
assistant | Model response; can be injected for few-shot | Alternating |
Key parameters:
| Parameter | Default | Range | Effect |
|---|---|---|---|
temperature | 1.0 | 0.0–2.0 | Randomness; 0 = deterministic |
top_p | 1.0 | 0.0–1.0 | Nucleus sampling; alternative to temperature |
max_tokens | model-dependent | 1–context window | Caps output length |
frequency_penalty | 0 | -2.0–2.0 | Reduces repetition of frequent tokens |
presence_penalty | 0 | -2.0–2.0 | Encourages new topic introduction |
stop | null | Up to 4 strings | Stops generation at specified sequences |
Do not set both
temperatureandtop_pto non-default values simultaneously — Microsoft documentation recommends altering only one.
Prompt Engineering Patterns
| Pattern | Description | When to Use |
|---|---|---|
| Zero-shot | Instruction only, no examples | General tasks, capable models |
| Few-shot | 2–5 examples as assistant/user pairs | Format enforcement, specialized style |
| Chain-of-thought (CoT) | "Think step by step" in prompt | Reasoning, math, multi-step logic |
| ReAct | Interleave reasoning and tool calls | Agent workflows |
| Meta-prompting | Prompt instructs model to generate its own prompt | Self-refinement pipelines |
Common Traps
- Exam may say "increase creativity" — answer is raise
temperature, nottop_p(both work but temperature is the primary knob). - System prompt is not a security boundary — it can be overridden by prompt injection. Guardrails require Azure AI Content Safety in addition.
- Few-shot examples go in the
messagesarray asuser/assistantpairs, not in the system message text. max_tokenslimits output only; total token budget = input + output ≤ context window.
Scenario
Requirement: A legal firm wants consistent JSON-formatted responses. The model keeps varying structure.
Reasoning: Set temperature: 0 for determinism, use structured outputs (JSON mode or response_format), and add few-shot user/assistant examples showing the exact JSON schema. Do not rely solely on the system prompt because format drift still occurs at higher temperatures.
RAG with Azure AI Search
Retrieval-Augmented Generation (RAG) grounds model responses in enterprise data without retraining. The architecture combines Azure AI Search (retrieval) with Azure OpenAI (generation).
RAG Architecture Flow:
Data flow from query to grounded response in a RAG pipeline
Indexing
An Azure AI Search index stores documents as JSON. For RAG, each document chunk is stored with its embedding vector in a vector field (type Collection(Edm.Single)).
Indexers automate ingestion from Azure Blob Storage, Cosmos DB, SQL, and SharePoint. Skillsets apply AI enrichment (OCR, entity extraction, embedding generation) during indexing via the enrichment pipeline.
| Component | Role |
|---|---|
| Data source | Connection to source (Blob, SQL, etc.) |
| Indexer | Orchestrates pull from source into index |
| Skillset | Chain of cognitive/custom skills applied to content |
| Index | Target schema with searchable, filterable, vector fields |
Search Modes
| Mode | Algorithm | Best For | Requires Embedding? |
|---|---|---|---|
| Full-text (keyword) | BM25 | Exact terms, structured queries | No |
| Vector search | HNSW (Hierarchical Navigable Small World) | Semantic similarity, paraphrase matching | Yes |
| Hybrid search | BM25 + HNSW fused via RRF (Reciprocal Rank Fusion) | Best of both; production default | Yes |
| Semantic ranker | Cross-encoder ML re-ranking | Re-ranks top-50 results for relevance | No (adds to hybrid) |
- Semantic ranker is a paid add-on (requires Standard tier or above); it re-ranks the top 50 results using a Microsoft-hosted ML model and generates captions/answers.
- Vector search is available on Basic tier and above (not Free tier).
- HNSW is the index algorithm; exhaustive KNN is available for small indexes or exact recall at higher cost.
Common Traps
- Semantic ranker ≠ vector search. Semantic ranker re-ranks text results using language understanding; vector search retrieves by embedding similarity. They are complementary, not substitutes.
- Free tier of Azure AI Search does not support vector search.
- RRF fusion score is not the same as cosine similarity — do not confuse them.
topparameter controls number of results returned;k(in vector queries) controls nearest neighbors explored.
Unlock with a paid plan
10 more sections (about 5,375 words) continue below. Every paid plan unlocks all AI-103 study guides, the full question bank and mock exams, from $4.97/month.