Manage memory, state, and execution Study Guide — GH-600

Manage Memory, State, and Execution

This domain tests your ability to design and implement memory strategies for AI agents, persist agent state across sessions and tool boundaries, and detect and correct context drift during extended execution. It accounts for 13% of the exam, making it one of the heavier-weighted domains. Expect scenario-based questions that require selecting the correct memory tier, artifact strategy, or drift-correction mechanism given a specific requirement.

Choosing Between Short-Term, Long-Term, and External Memory

Short-term memory (also called in-context memory) lives inside the active context window of the model. It is ephemeral: it is lost when the session ends, and it consumes tokens. It is appropriate for single-turn tasks, within-session reasoning chains, and scratch-pad computations that do not need to persist.

Long-term memory persists beyond a single session. In agentic frameworks built on GitHub Copilot and Azure AI services, long-term memory is implemented by writing structured state to durable storage — typically Azure Cosmos DB, Azure Blob Storage, or a vector store such as Azure AI Search — and retrieving relevant facts at the start of subsequent sessions.

External memory refers to retrieval-augmented storage that the agent queries on demand. It is not loaded wholesale into context; instead, a retrieval step fetches the top-k relevant chunks. This avoids bloating the context window and is the correct choice when the knowledge base is large, changes frequently, or must be shared across multiple agents.

Common traps

• Exam distractors will offer "store everything in the context window" for cross-session tasks — this is wrong; short-term memory does not survive session boundaries.
• Confusing external memory (queried selectively) with long-term memory (loaded at session start). External memory uses a retrieval step; long-term memory is often a structured state object loaded in full.
• Assuming long-term memory is automatically maintained — the agent (or the framework) must explicitly write to durable storage; it does not happen by default.
• The exam may describe a "large document corpus" and ask which memory tier to use; the answer is external/retrieval-augmented, not long-term.

Worked scenario: An agent must remember a user's preferred code style across all future sessions. The knowledge base is one small preference object. → Use long-term durable state (write to Cosmos DB at session end, load at session start). External memory retrieval would add unnecessary latency for a single small object.

Scoping Agent Memory to Task-Relevant Information

Memory scoping means including only the information the agent needs to complete the current task. Unscoped memory causes context bloat, increases token costs, degrades model accuracy (the model attends to irrelevant facts), and increases the risk of context drift.

Scoping strategies:

• Task-scoped memory: load only the state record for the current task ID.
• Role-scoped memory: restrict which agent roles can read or write specific memory keys.
• Recency windowing: load only the N most recent interactions or state snapshots.
• Relevance filtering: use a retrieval step (semantic similarity) to select only memory entries above a relevance threshold before injecting them into context.

Relevance threshold is a scored cutoff (e.g., cosine similarity ≥ 0.75 in Azure AI Search) below which memory entries are excluded from context injection.

Common traps

• The exam may present an agent that loads all historical state at startup — this is a scoping failure, not a memory-type error.
• Confusing relevance filtering (a retrieval-time operation) with pruning (a maintenance-time operation removing obsolete entries from storage).
• "Load the full user history" is almost never the correct answer when a recency window or task scope is available.