Implement computer vision solutions Study Guide — AI-103

Implement computer vision solutions

This domain is worth 13% of AI-103 and covers the generative and analytical sides of visual AI in Azure AI Foundry: creating images and video with Azure OpenAI models (DALL-E 3, GPT-image-1/mini, Sora), editing them with masks and reference media, extracting meaning from visual input with multimodal chat models, Azure AI Vision Image Analysis 4.0, and Azure AI Content Understanding, and governing all of it with Azure AI Content Safety, Prompt Shields, and Content Credentials. Questions are overwhelmingly service-selection and parameter-detail questions: given a requirement (inpaint a region, caption 500 product photos, extract per-segment video fields, block an embedded prompt-injection attack), pick the correct model/API/feature and the correct parameter values. Expect 8-12 scored items, several as case-study or drag-and-drop sequencing of an API workflow.

Generating images from text prompts and reference media

Azure image generation is exposed through Azure OpenAI in Foundry Models via the Images API: POST {endpoint}/openai/deployments/{deployment}/images/generations?api-version=.... Deployments are model-specific; you cannot call an editing operation on a model that does not support it.

Other must-know parameters for GPT-image-1: output_format (png, jpeg, webp), output_compression (0-100, JPEG/WebP only), input_fidelity (low/high — use high to preserve faces, logos, and fine detail from reference images).

Reference media is only meaningful for GPT-image-1-family models: you pass one or more source images to /images/edits (with or without a mask) so the model preserves subject identity, style, or product appearance. DALL-E 3 cannot accept any image input — style transfer with DALL-E 3 must be done purely in prose in the prompt.

Decision tree for choosing an image-generation or image-understanding path

Common traps

• Choosing DALL-E 3 for an inpainting or reference-image scenario. DALL-E 3 is generation-only.
• Answers that set n=4 on DALL-E 3 — invalid; n must be 1.
• Assuming GPT-image-1 returns a URL. It returns base64 only; you must decode and persist it yourself.
• Forgetting that DALL-E 3 URLs are temporary (~24 hours) — a "store the asset permanently" requirement means download the bytes or request b64_json.
• Using style=vivid or quality=hd with GPT-image-1 (those belong to DALL-E 3).
• Believing generated images are unwatermarked: Azure OpenAI attaches C2PA Content Credentials metadata identifying the image as AI-generated.

Worked scenario. A retailer must generate 8 variants of a product hero shot per request, with a transparent background for web overlay, preserving the exact product from a supplied photo. → GPT-image-1 (or mini), /images/edits with the product photo as reference, input_fidelity=high, background=transparent, output_format=png, n=8. DALL-E 3 fails on all three of: references, transparency, and n>1.

Generating video from text prompts and reference media

Video generation uses Sora models (sora, sora-2) deployed in Azure OpenAI. Unlike images, video generation is asynchronous and job-based:

1. POST /openai/v1/video/generations/jobs with model, prompt, height, width, n_seconds, n_variants.
2. Poll GET /openai/v1/video/generations/jobs/{jobId} until status is succeeded (other statuses: queued, preprocessing, running, cancelled, failed).
3. Read the generations[] array and download bytes from GET /openai/v1/video/generations/{generationId}/content/video (and /content/thumbnail for a still).

Key rules: the width/height must match a supported resolution pair; requesting multiple variants at 1080p is rejected (variant count shrinks as resolution grows); a reference image supplied via input_reference must match the requested output dimensions. Jobs can be listed and deleted; generated assets are retained for a limited period, so download and store them.

Sora job lifecycle — the sequence exams ask you to order

Common traps

• Expecting a synchronous response with the MP4 inline. Video generation always goes through a job + poll + download flow.
• Answers claiming Sora v1 produces audio, or that you can request arbitrary durations like 60 seconds.
• Mismatching input_reference dimensions to the output size.
• Assuming you can remove the visible watermark — that violates the Azure OpenAI code of conduct and is never the correct answer.
• Confusing Sora (generative video) with Azure AI Content Understanding / Video Indexer (analytical video).