Databricks Generative AI Engineer Associate: Application development Practice Questions

DatabricksVectorSearch (from langchain_community.vectorstores) is the correct class to wrap a Mosaic AI Vector Search index, and calling .as_retriever() on it produces a standard LangChain retriever interface. There is no standalone VectorSearchRetriever class in langchain_community.retrievers for Databricks. DatabricksEmbeddings is an embeddings wrapper used for generating embeddings, not for wrapping a vector store as a retriever. FAISS.from_existing_index() is used for local FAISS indexes and has no connection to Mosaic AI Vector Search.

ConversationSummaryMemory progressively summarizes older parts of the conversation, allowing the assistant to maintain coherent context across many turns while keeping the total token count manageable — directly addressing the concern of staying within the context window. ConversationBufferMemory retains the full raw history and will grow unbounded, eventually exceeding context limits. ConversationTokenBufferMemory does enforce a token limit but does so by truncating old turns entirely, losing that context rather than summarizing it. VectorStoreRetrieverMemory retrieves relevant past turns but may miss sequential context needed for coherent multi-turn dialogue.

The databricks-agent evaluation preset in MLflow automatically computes LLM-as-judge metrics including groundedness (whether the answer is supported by the retrieved context) and relevance_to_query (whether the answer addresses the user's question). BLEU (E) and ROUGE-L (A) are n-gram overlap metrics suited for translation/summarization, not built into the agent preset. Perplexity (C) requires access to model logits and is not part of this preset. Exact_match (B) is a lexical metric not included in the agent evaluation suite.

When correct documents are retrieved but answers still lack detail, two common causes are: (1) the prompt template does not explicitly instruct the model to ground its answer in the provided context, causing the LLM to rely on its parametric knowledge instead of the retrieved chunks; and (2) over-aggressive chunking splits relevant detail across multiple small chunks, so critical information lands in chunks ranked outside the top-k and is never passed to the LLM. Option D is incorrect — index type affects sync latency, not response content truncation. Option B is a misconception — pay-per-token pricing does not impose output token length limits; output length is controlled by the max_tokens parameter. Option C describes an embedding model mismatch, which would degrade retrieval quality (wrong documents returned), but retrieval logs already confirm correct documents are being retrieved, ruling this out. Option F is an infrastructure/deployment issue unrelated to content quality in responses.

Options A, B, and D are correct. A: Pay-per-token has no reserved capacity, so there is zero cost when the tool is idle, which is advantageous for low-utilization internal tools. B: Bursty, unpredictable traffic is the classic use case for pay-per-token; Provisioned Throughput is designed for sustained, predictable high load. D: Pay-per-token requires no upfront capacity decisions, which reduces operational burden for a small team. Option C is wrong because Provisioned Throughput generally offers more consistent and lower latency under high load; pay-per-token can experience higher latency during shared-capacity peaks. Option E is wrong because pay-per-token Foundation Model API endpoints serve Databricks-hosted base models; deploying fine-tuned private models requires Provisioned Throughput or a custom serving endpoint. Option F, while partially true, is not a distinguishing reason to choose pay-per-token over Provisioned Throughput specifically, since both are managed by Databricks.