Free AI Advanced practice test

The pattern described — correct output but excess tool invocations — is a classic agent efficiency or step-optimality issue that task success rate alone cannot capture. Adding a redundant_tool_call_rate or tracking average_tool_calls_per_task via trajectory evaluation directly measures this inefficiency and can surface cost and latency implications. The retrieval accuracy / tool_output_relevance answer is wrong: the tool presumably returns correct results; the problem is calling it too many times. The BLEU-score answer is wrong: BLEU measures text similarity of outputs, not the efficiency of the action sequence, and replacing task success rate would remove a valuable metric. The memory-leak / session_state_size answer is wrong: repeated tool calls may bloat session state slightly, but the root problem is agent reasoning, not a memory management bug. The latency-regression / SLA answer is wrong: latency is a downstream symptom, not the primary dimension being exposed, and a wall-clock SLA alone does not explain or quantify the root cause of redundant calls.

A LangGraph state reducer defines how the values a node returns are merged into the existing state for that key. On a list, operator.add concatenates, so each node's output list is appended to the accumulated list. This lets multiple nodes add annotations without overwriting each other. The option about keeping only the most recent list describes the default behavior with no reducer (plain replacement), which is the opposite of what operator.add does. The deduplication option is wrong because operator.add only concatenates and never removes duplicates. The locking option is wrong because reducers only define how values are merged; they do not lock keys against concurrent writes.

The manager agent's planning loop generating more reasoning steps or re-delegations is a correct area to check. In hierarchical mode the manager LLM drives delegation, and more complex inputs can trigger more re-planning cycles, each of which uses tokens. A more expensive model tier is also correct: a dependency update that quietly changes the default model is a common real-world cause of cost spikes with no change in quality. Larger tool outputs are correct too: longer search results or scraped pages fill more of the context on every LLM call, which directly raises cost. A decrease in the number of tasks is wrong, because fewer tasks would generally lower cost, not raise it. An easier-to-satisfy success rate metric concerns the evaluation metric itself and does not directly cause cost to rise. More frequent manager LLM invocations due to crew size or validation steps is plausible. However, it is essentially a subset of the planning-loop area, which captures the issue more precisely, so it is a reasonable secondary consideration rather than one of the top three.

A SequentialAgent is designed precisely for workflows where a fixed, ordered sequence of sub-agents must execute one after another, passing outputs downstream — no LLM is needed to decide the order. An LlmAgent is inappropriate as the top-level orchestrator when order is deterministic; it uses LLM inference to decide routing, adding unnecessary cost and non-determinism. A ParallelAgent runs sub-agents concurrently, which violates the strict sequential dependency (the LLM analysis step depends on the API result). A LoopAgent is for iterative retry or polling patterns, not a one-pass linear pipeline.

CrewAI's hierarchical process assigns a manager agent that plans, delegates, and reviews work across worker agents as the job goes on. This matches the orchestrator pattern described. The sequential process runs tasks in a fixed order and cannot delegate or reassign work on the fly. CrewAI has no built-in 'parallel process' as a first-class process type. Task concurrency can be configured, but this option misrepresents how CrewAI works. Embedding a LangGraph graph inside a CrewAI Crew is technically possible, but it is not CrewAI's standard, purpose-built answer to this requirement.

Options B and D are correct. B is correct: the ephemeral cache TTL is a minimum of 5 minutes, and cache hits refresh (extend) the TTL — so actively used cache entries remain valid longer. D is correct: reading from cache costs significantly fewer tokens than a full cache miss (cache reads are billed at a lower rate than standard input tokens), making prompt caching cost-effective for repeated prefixes like long system prompts. Option A is wrong because it states the TTL is 'fixed at 5 minutes' and implies no refreshing — the TTL resets on hits. Option C is wrong; cache entries do expire after inactivity and are not permanent. Option E is wrong; cache entries are scoped to the API key that created them and are not shared across keys even within the same organization. Option F is wrong; while ephemeral is the primary type, Anthropic's documentation references other cache behaviors and the landscape may evolve — more importantly, stating it as the only type is misleading.

The Anthropic tool input_schema follows JSON Schema conventions. Required parameters are listed in a top-level required array, while optional parameters are simply absent from that array. Option B correctly places only query in required and omits max_results from it, while also including a default hint. Option D incorrectly embeds required as a per-property boolean attribute, which is not valid JSON Schema syntax used by the Anthropic API. Option A incorrectly lists both fields as required, which would force Claude to always provide max_results. Option C uses "type": "function" at the schema root and a parameters key—this is OpenAI function-calling format, not the Anthropic input_schema format.

Option C is correct: it follows the JSON Schema standard used by the Anthropic tool input_schema. Required fields are listed in a top-level required array, properties use "type": "string" with an "enum" array for constrained values, and the top-level key is "properties". Option A incorrectly places required as a per-property key instead of a top-level array — this is not valid JSON Schema. Option B uses "type": "enum" which is not a valid JSON Schema type; the correct pattern is "type": "string" with an "enum" keyword. Option D replaces the "properties" keyword with "fields", which is not valid JSON Schema and will not be parsed correctly by the API.

tool_choice: {"type": "any"} forces Claude to call at least one tool from the provided list on every turn, preventing it from responding with plain text only. This is the correct mechanism to enforce tool-only responses. Option D (auto) allows Claude to decide whether to call a tool or respond directly — it does not guarantee tool use. Option A (required) is not a valid tool_choice type in the Anthropic API; the valid types are auto, any, and tool (for specifying a particular tool). Option C is incorrect: stop_sequences affect where Claude stops generating text but do not route the model toward tool calls; Claude could still produce text before hitting the stop sequence.

The 'complementary layers' answer is correct. OpenTelemetry is an open standard for collecting and exporting telemetry data, and LLM platforms like Langfuse can ingest OTel traces or provide their own SDKs. The two complement each other: OTel gives you vendor-neutral instrumentation, and LLM platforms add AI-specific analysis, scoring, and dashboards on top of that telemetry. The mutual-exclusivity claim is false, because many teams use both in their stack. The proprietary-protocol claim is false, because Langfuse supports OTel ingestion and does not replace it with a proprietary protocol. The claim that GenAI conventions make LLM platforms obsolete overstates OTel's current scope. The GenAI semantic conventions standardize attribute names, but they do not provide evaluation scoring, human feedback collection, or experiment comparison.

Options A and C are correct. (A) Domain expert annotators are critical for clinical content — crowdworkers lack the medical knowledge to reliably assess clinical accuracy, completeness, or appropriate terminology. (C) A clear, multi-dimensional rubric ensures consistency across annotators and produces structured, granular signals suitable for reward model training. Binary ratings alone are too coarse for nuanced fine-tuning. Option B is wrong because binary ratings sacrifice granularity needed for fine-tuning; a rubric is preferred. Option D is wrong because anonymizing annotators removes the ability to compute inter-annotator agreement (IAA), which is essential for measuring and ensuring annotation quality. Option E is wrong because collecting only positive feedback creates severe selection bias — a fine-tuning dataset needs both positive and negative examples to learn meaningful quality distinctions.

The standard Braintrust CI pattern runs an experiment against a versioned dataset, applies scoring functions, and gates the build on an aggregate score threshold, all inside the CI job. An S3-based string diff skips Braintrust's experiment tracking and scoring framework entirely. A shadow deployment compares live traffic, which is an A/B testing pattern, not a regression test against a golden dataset. Wiring Langfuse datasets to Braintrust webhooks creates fragile coupling and is not a supported or recommended integration pattern.

Per-team cost attribution needs a field that maps each request to a specific team or cost center. Adding team_id lets you group and total token costs by team directly. session_id links requests within a user session but does not say which team owns them. request_id is a deduplication and tracing key, not a cost-center identifier. The environment tag separates prod from staging traffic but does not break costs down by team.

The canonical circuit breaker state machine is: Closed (normal operation, failures counted) → threshold met → Open (all requests immediately fail/fail-fast) → timer elapses → Half-Open (one probe request allowed) → probe succeeds → Closed. Option A correctly describes this sequence for a 3-minute outage where the provider recovers before or during the half-open probe. B is wrong — the circuit goes from Closed directly to Open (not Half-Open) after the failure threshold; Half-Open is the state after the timer, not immediately after failures. C is wrong — the circuit opens after the failure threshold (5 consecutive errors), not after the first failure; retry counts are separate from circuit breaker state transitions. D is wrong — when the probe in Half-Open state fails, the circuit returns to Open (not Closed); the timer resets and the cycle repeats until a probe eventually succeeds.

In LiteLLM's router, setting rpm_limit: 0 on a model entry is interpreted as zero available capacity, so the router will skip that model and immediately route to the next available option in the fallback chain. This would cause all traffic to bypass the self-hosted model even when it is healthy. — Option A is wrong: the model_list priority order affects routing, but the symptom is that the self-hosted model is skipped entirely, not that GPT-4o takes priority in a tie; an rpm_limit of 0 more precisely explains the complete bypass. — Option B is wrong: LiteLLM's health checks specifically probe the /health or /models endpoints; a valid HTTP 200 from the OpenAI-compatible API would typically be accepted. — Option C is wrong: mixing cloud and self-hosted providers is a core and well-documented LiteLLM use case.

For a use case where precision matters — specific IDs, ticket numbers, account references — a sliding window that retains exact verbatim text is safer than summarization. Summarization inherently compresses and abstracts information, and LLM-generated summaries can drop or paraphrase specific numerical identifiers, which would be catastrophic for a support bot. The tradeoff is that the sliding window will eventually drop early turns (a problem for 200-turn sessions), while summarization preserves coverage at the cost of potential precision loss. Option D correctly identifies the core tradeoff (verbatim preservation vs. compression risk) and is the best answer given the requirements. Option A is incorrect because summarization does not 'guarantee full session history' — it produces a lossy compressed representation and may omit critical specifics. Option C invents a turn-count threshold (50 turns) that has no basis in practice, and the claim that LLMs hallucinate less with compressed context is unsupported. Option B is false; the two approaches differ significantly in accuracy for precision-sensitive workloads — they are not equivalent.

A is correct: provider-level prompt caching (e.g., OpenAI auto-cache, Anthropic with breakpoints) requires no application code changes for OpenAI and delivers savings proportional to the cached prefix's share of total input tokens. B is correct: self-hosting with vLLM can dramatically reduce per-token costs at scale, but it introduces significant operational burden — GPU procurement, CUDA environment management, model weight updates, and 24/7 infrastructure reliability. C is correct: semantic caching's ROI depends entirely on query repetition. For a legal research tool where every query is unique, hit rates may be near zero; for an FAQ bot, hit rates can be very high. — Option D is wrong: Anthropic does charge a cache write fee (25% more than standard input tokens), but the economics are still favorable because cache reads cost 90% less than standard input; over multiple requests, savings easily exceed write costs. — Option E is wrong: self-hosted inference eliminates provider network latency but adds internal network latency plus potential GPU scheduling overhead; it does not guarantee lower latency in all cases. — Option F is wrong: GPTCache ships with support for multiple off-the-shelf embedding models (e.g., sentence-transformers) and does not require custom training to begin production use.

Indirect prompt injection occurs when malicious instructions are embedded in external content that the LLM processes (such as documents, web pages, or emails) rather than being typed directly by the attacker into the user interface. Direct prompt injection involves the attacker directly crafting the user-facing input field. Adversarial example attacks typically target model classification via imperceptible input perturbations. Model inversion attacks attempt to reconstruct training data from model outputs.

The Transparency principle in the Microsoft RAI framework specifically addresses the need for AI systems to be understandable—meaning stakeholders can inspect, explain, and if necessary contest AI-driven decisions. This is directly relevant when an employee needs to understand why an AI-assisted performance score was generated. Fairness (D) addresses equitable treatment across groups, not individual decision explainability. Reliability & Safety (A) addresses consistent performance and harm prevention, not decision justification. Privacy & Security (C) governs data protection, not decision interpretability. Inclusiveness (E) focuses on designing systems that work for all users, not on the ability to explain individual decisions.

Prompt injection (OWASP LLM01) occurs when an attacker crafts input that overrides or manipulates the model's original instructions, causing it to perform unintended actions — exactly what happened here. Insecure output handling (LLM02) refers to downstream systems failing to sanitize LLM outputs before using them, not the initial attack vector. Training data poisoning (LLM03) involves corrupting the data used to train or fine-tune the model. Model denial of service (LLM04) involves resource exhaustion attacks, not instruction override.