[EAAPL-WRK007] Context Compression

Category: Agentic Workflows Sub-category: Memory Management Architecture Version: 1.0 Maturity: Emerging Tags: context-window, summarisation, context-compression, rolling-window, memory-management Regulatory Relevance: ISO 42001 §8.4, APRA CPS 230

1. Executive Summary

The Context Compression Pattern defines techniques for managing an agent's context window during long-running workflows: rolling summarisation of earlier history, selective retention of high-relevance content, episodic retrieval of previously compressed material, and hierarchical memory tiers. Without active context management, long-running agentic workflows eventually exceed the model's context window, causing either hard failures or silent truncation of critical earlier context. This pattern prevents context overflow while preserving the reasoning continuity needed for high-quality multi-step task completion.

For CIO/CTO audiences: every AI model has a limit on how much information it can hold in working memory at one time (the context window). For short tasks this is not a problem. For long-running workflows — a multi-day research project, a complex code generation task, an extended client engagement — the working memory fills up and important earlier context gets forgotten. This pattern is the equivalent of a skilled professional's note-taking discipline: compressing older details into summaries, retaining only the most relevant specifics, and retrieving historical detail on demand. It enables AI agents to work effectively on tasks that span thousands of steps without losing sight of earlier work.

2. Problem Statement

Business Problem

Enterprise long-running tasks — regulatory analysis across hundreds of documents, code review of large codebases, extended document drafting sessions — generate more intermediate context than any current LLM context window can hold. Without context management, these tasks either fail with context overflow errors or produce degraded outputs because critical earlier findings have been silently truncated from the model's working memory.

Technical Problem

LLM context windows are finite (ranging from 8K to 1M tokens across current models). Long-running agentic workflows accumulate thought-action-observation records that grow without bound. Naive strategies (truncate oldest content, always truncate longest observations) destroy reasoning continuity and produce confusion in later reasoning steps that reference context that has been silently removed.

Symptoms of Absence

• Agent loses track of earlier findings, re-asks questions already answered, or contradicts its own earlier conclusions
• Hard context overflow errors terminate workflows mid-execution
• Observations are silently truncated; agent reasons about incomplete tool results without knowing they are incomplete
• No mechanism to retrieve earlier reasoning context on demand

Cost of Inaction

• Reliability: Context overflow errors terminate tasks, wasting all prior computation
• Quality: Silent truncation produces outputs that contradict or ignore earlier findings
• Trust: Users discover that the agent has "forgotten" key earlier context, destroying confidence

3. Context

When to Apply

• Tasks generate context that may approach or exceed the model's context window budget
• Long-running agents with many tool calls (>15 iterations)
• Tasks spanning multiple sessions (inter-session context preservation)
• Tasks requiring recall of information from early stages when synthesising final outputs

When NOT to Apply

• Short tasks with predictably small context footprints
• Tasks where complete context history must be preserved verbatim (use an explicit log, not the context window)
• Tasks where summarisation would destroy required precision (highly technical numerical content)

Prerequisites

• EAAPL-AGT002 (Stateful Agent Memory) for persistent episodic storage
• Token counting utility for real-time context budget monitoring
• Summarisation prompt templates per content type (tool observations, reasoning steps, user turns)
• Context budget thresholds for triggering compression

Industry Applicability

4. Architecture Overview

Context compression is implemented as a continuous monitoring and management layer that wraps the agent's context window, operating proactively before overflow occurs.

Token Budget Monitor The token budget monitor tracks the current context window usage in real time, counting tokens after every addition (thought, observation, user message). It emits threshold events at configurable percentages of the context window capacity: a warning threshold (e.g., 70%) triggers proactive compression, a critical threshold (e.g., 85%) triggers aggressive compression.

Rolling Summarisation When the warning threshold is crossed, the Rolling Summariser identifies the oldest N thought-action-observation blocks in the scratchpad and replaces them with a structured summary. The summary preserves: key findings (entities, decisions, conclusions), pending actions, and unresolved questions. The raw blocks are archived to episodic memory (EAAPL-AGT002) before being removed from the active context. The summary explicitly marks itself as a compression artefact so that the agent's reasoning knows it is working with summarised rather than raw context.

Selective Retention Not all context has equal relevance. The Relevance Scorer evaluates each context block against the current task objective and assigns a relevance score. High-relevance blocks (directly referenced by the current reasoning step) are retained in full. Medium-relevance blocks are summarised. Low-relevance blocks are archived to episodic memory. Relevance scoring is performed by a lightweight LLM call or by embedding similarity to the current task objective.

Episodic Retrieval When the agent's reasoning references content that has been compressed out of the active context ("Earlier I noted that clause 12.3 had a specific obligation — what was it exactly?"), the Episodic Retriever performs a semantic search against the episodic memory store (EAAPL-AGT002) and injects the retrieved content back into context as a bounded retrieval observation. This on-demand retrieval prevents the need to retain all historical context proactively.

Hierarchical Memory Tiers Context is managed across three tiers: (1) Active Context — full fidelity, in-window; (2) Summary Buffer — compressed representations of recent history, in-window; (3) Episodic Archive — full-fidelity earlier content, out-of-window but retrievable. The size allocated to each tier is configurable per task type.

5. Architecture Diagram

6. Components

7. Data Flow

Error Flow

8. Security Considerations

Sensitive Content in Episodic Archive

• The episodic archive persists sensitive intermediate content (PII, commercial-in-confidence data) that may have shorter retention requirements than the task output
• Mitigation: Apply same retention and access controls to episodic archive as to task audit records; support time-limited archival with automatic deletion

OWASP LLM Top 10

9. Governance Considerations

Compression Audit Trail

• Every compression event must be logged: which blocks were compressed, what summary was produced, and what was archived. This ensures that the full task context can be reconstructed for audit purposes even though it was not present in the active context at every point.

Governance Artefacts

10. Operational Considerations

SLOs

Monitoring

• Context budget usage trending per task type: approaching ceiling early indicates need for earlier compression or higher budget threshold
• Summary to raw token ratio: tracks compression efficiency
• Retrieval usage rate: high retrieval rate indicates compression is too aggressive (too much important content being compressed)

11. Cost Considerations

Indicative Cost Comparison

12. Trade-Off Analysis

Architectural Tensions

13. Failure Modes

14. Regulatory Considerations

ISO 42001

• §8.4: Compression decisions affect the AI system's operational quality; compression algorithms must be documented and tested against quality benchmarks.

APRA CPS 230

• For agents operating in material business processes, the full reasoning context (including archived episodic records) must be retainable for operational resilience investigations.

Australian Context

• Privacy Act 1988: Content archived to episodic storage inherits the privacy sensitivity of the source task; retention and deletion policies must match the organisation's records management framework.

15. Reference Implementations

AWS

Azure

On-Premises

16. Related Patterns

17. Maturity Assessment

Overall Maturity: Emerging

18. Revision History