Context Engineering for Agents: Caching, Compaction, Subagents

Last updated: October 17, 2026

Context engineering for agents means deciding which tokens go into each model request. That covers four things: a cached stable prefix, a history that is trimmed or compacted, tools and data loaded only when needed, and side work handed to subagents that report back in a few lines. Anthropic and OpenAI both now ship server-side compaction and explicit cache controls. The practical default is a stable cached prefix plus provider compaction. Add tool-result clearing and subagents once tool output starts to take over the window.

This post is the runtime and API half of the topic. For the file-level half (CLAUDE.md, system prompts, skills and memory files), see Context Engineering: The Highest-Leverage Skill in AI-Assisted Development. Everything below is about what your agent loop sends to the API.

Official Sources#

The five levers#

Anthropic's engineering team defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference." It matters even with huge windows because of what the Claude docs call context rot: as the token count grows, accuracy and recall degrade. Opus 5.5, Sonnet 5.5, Haiku 5.5 and the Fable and Mythos 5.x models all ship with a 1M-token window by default, with no beta header needed. Treat that number as a ceiling, not a target.

Three facts from the context windows page shape everything that follows:

• Everything counts. That includes the system prompt, every message (tool results, images and documents too), tool definitions and the model's output, including thinking.
• Thinking sticks around. On Opus 4.5+, Sonnet 4.6+, Haiku 5.5 and Fable/Mythos 5.x, the API keeps earlier thinking blocks by default, and they count as input.
• Cached tokens still take up the window. Caching changes what you pay, not whether the tokens count.

So there are five things you can do to the tokens in a request:

%% title: What fills an agent context window
flowchart TD
  P["System prompt + tool definitions"] --> H["Conversation history"]
  H --> R["Tool results"]
  R --> T["Kept thinking blocks"]
  T --> N["New user turn"]
  P -. "1 cache" .-> L1["Cached prefix, cheaper reads"]
  R -. "2 clear" .-> L2["Placeholder for stale results"]
  H -. "3 compact" .-> L3["Summary block"]
  R -. "4 retrieve just in time" .-> L4["Paths and IDs, not payloads"]
  R -. "5 delegate" .-> L5["Subagent returns a short report"]

Caching makes the stable part cheap. The other four levers make the growing part smaller. In tool-heavy agent loops, tool results usually grow fastest (Anthropic's engineering post calls clearing old tool results one of the safest, lightest-touch forms of compaction), which is why three of the five levers act on them.

Lever 1: Cache the stable prefix#

Caching comes first because it costs nothing in quality. It only works if the start of your prompt stays byte-identical from one request to the next. That means ordering the prompt: stable instructions first, then tools, then anything that changes.

Anthropic. There are two ways to place breakpoints, and you can use both:

• Automatic caching: put cache_control: {type: "ephemeral"} at the top level of the request. The API moves the breakpoint to the last cacheable block, so a growing conversation keeps caching itself.
• Explicit breakpoints: put cache_control on individual blocks, up to 4 per request. Automatic caching takes one of those 4 slots.

The default TTL is 5 minutes, refreshed for free on every hit, and ttl: "1h" is optional. Prices are multiples of the base input price: a 5-minute write costs 1.25x, a 1-hour write costs 2x, and a read costs 0.1x by default. Opus 5.5 and Sonnet 5.5 read at 0.05x, and Fable 5.1 and Mythos 5.1 read at 0.025x.

The minimum cacheable prompt depends on the model. It is 512 tokens on most Claude 5.x models (Fable 5.1, Mythos 5.1, Opus 5.5, Opus 5, Sonnet 5.5, Fable 5, Mythos 5 and Haiku 5.5), but 1,024 tokens on Claude Sonnet 5. A prompt under the minimum is not cached, and nothing tells you. Check the per-model table in the prompt caching docs for the model you actually run.

The cache invalidates in the order tools, then system, then messages. Changing a tool definition invalidates everything. Changing thinking parameters or output_config.effort invalidates the messages cache. One detail helps long-running agents: on Fable 5.x, Mythos 5.x, Opus 5.5, Opus 5, Opus 4.8, Sonnet 5.5 and Haiku 5.5 (but not Sonnet 5), adding a {role: "system"} message mid-conversation leaves the caches intact. You can add a late instruction without rewriting the prefix.

The request below caches the stable prefix for an hour and lets automatic caching follow the conversation. It also turns on context editing, which the next section covers.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

const STABLE_INSTRUCTIONS = "You are a release engineer for this repo. ..."; // long, rarely edited

const tools: Anthropic.Beta.Messages.BetaTool[] = [
  {
    name: "read_file",
    description: "Read a file from the repository by path.",
    input_schema: {
      type: "object",
      properties: { path: { type: "string" } },
      required: ["path"],
    },
  },
  {
    name: "run_tests",
    description: "Run the test suite and return the output.",
    input_schema: { type: "object", properties: {} },
  },
];

export async function agentTurn(
  messages: Anthropic.Beta.Messages.BetaMessageParam[],
) {
  const response = await client.beta.messages.create({
    model: "claude-opus-5-5",
    max_tokens: 8192,
    betas: ["context-management-2025-06-27"],
    tools,
    // Explicit breakpoint: the stable prefix (tools + system) lives for an hour.
    system: [
      {
        type: "text",
        text: STABLE_INSTRUCTIONS,
        cache_control: { type: "ephemeral", ttl: "1h" },
      },
    ],
    // Automatic caching: moves a 5-minute breakpoint to the last cacheable block.
    cache_control: { type: "ephemeral" },
    messages,
    context_management: {
      edits: [
        {
          type: "clear_tool_uses_20250919",
          trigger: { type: "input_tokens", value: 120_000 },
          keep: { type: "tool_uses", value: 4 },
          // Only clear if it frees enough to be worth rewriting the cache.
          clear_at_least: { type: "input_tokens", value: 20_000 },
          exclude_tools: ["run_tests"],
        },
      ],
    },
  });

  const u = response.usage;
  console.log({
    cacheRead: u.cache_read_input_tokens,
    cacheWrite: u.cache_creation_input_tokens,
    uncached: u.input_tokens,
    edits: response.context_management?.applied_edits,
  });
  return response;
}

OpenAI. On GPT-5.6 and later, caching is on by default. Writes cost 1.25x, reads cost 0.1x (0.05x on GPT-6.1 Sol), and the minimum prefix is 1,024 tokens. The newer prompt_cache_options object gives you control similar to Anthropic's:

• mode: "implicit" (the default) lets OpenAI pick one breakpoint and also writes any explicit ones you mark. mode: "explicit" turns the implicit breakpoint off. If you mark no explicit breakpoints, the request is not cached at all.
• You mark a breakpoint on a content block with prompt_cache_breakpoint: {mode: "explicit"}. Each request can create up to 4 cache writes.
• ttl supports only "30m", which is also the default.
• prewarm: true writes the cache without generating any output. That is useful right before a burst of parallel requests that share a prefix.

{
  "model": "gpt-6-astra",
  "prompt_cache_options": { "mode": "explicit", "ttl": "30m" },
  "input": [
    {
      "role": "developer",
      "content": [
        {
          "type": "input_text",
          "text": "<long stable instructions>",
          "prompt_cache_breakpoint": { "mode": "explicit" }
        }
      ]
    },
    { "role": "user", "content": "Next task..." }
  ]
}

OpenAI's guide lists two habits for keeping the prefix stable. To stop the model from calling a tool, set tool_choice: "none" or narrow allowed_tools rather than deleting the tool, because deleting it changes the prefix. On GPT-6, change reasoning effort mid-thread with a configuration_update input item instead of editing the request's reasoning settings. Both are OpenAI features. On Anthropic the trade-off is different: the Claude API has no allowed_tools or configuration_update, and changing tool_choice keeps the tools and system caches but invalidates the messages cache. That still beats editing a tool definition, which invalidates everything. On Claude models that support per-message effort (beta), an effort change sent in a role: "system" message inside messages leaves the cached prefix intact.

Models before GPT-5.6 use the older prompt_cache_retention field for retention, and the SDK now marks it as deprecated in favour of prompt_cache_options.ttl. prompt_cache_key is a different case. On GPT-5.6+, it is still supported but optional: routing is automatic, so you no longer need it for hit rates, and it only separates cache accounting between customers or users (which also helps prevent one tenant from probing another's cache hits).

This post skips the break-even math. Prompt Caching in the Claude API: A Production Guide covers when caching pays off and when it is just overhead.

Lever 2: Clear stale tool results (context editing)#

Anthropic's engineering post calls tool result clearing "one of the safest lightest touch forms of compaction." Once the agent has read a file and acted on it, the raw contents rarely need to stay in context. Context editing removes them on the server. Your client keeps its full history, and the API sends the model a trimmed version.

It needs the context-management-2025-06-27 beta header and works on all supported Claude models. These are the clear_tool_uses_20250919 defaults:

clear_at_least is the field people tend to skip. Clearing a tool result changes the prompt at that point, which invalidates the cached prefix from there on. If a pass frees only a few thousand tokens, you pay for a cache rewrite and gain very little room. Setting a floor (20,000 tokens in the snippet above) means clearing only runs when it is worth the rewrite.

Use exclude_tools for results the agent must keep seeing, such as the latest test run or a plan file. To drop old thinking blocks as well, add clear_thinking_20251015 to the same edits array and put it first in the list. Every response reports what happened in context_management.applied_edits, with cleared_tool_uses and cleared_input_tokens. count_tokens returns context_management.original_input_tokens, so you can compare before and after.

A migration note: the client-side compaction_control option on the SDK tool runner is deprecated in the TypeScript and Ruby SDKs and was removed from the Python SDK in v1.0. Anthropic recommends server-side compaction instead.

Context editing is the API version of a broader pattern: keep intermediate state out of the prompt and pass back short receipts. The 98% Context Reduction Pattern covers the application-side version.

Lever 3: Compact the history#

Clearing removes specific blocks. Compaction replaces a run of history with a summary. It frees much more room, but it is lossy, so the instructions you give the summarizer matter.

Anthropic: on demand or at a threshold#

Anthropic ships two modes. Its compaction overview recommends on-demand wherever it is available.

On-demand (beta header compact-2026-09-04). Your app decides when to compact. Send the conversation as it stands, with compaction: {type: "summarize"} at the top level. The API generates no reply. It returns one compaction block, which holds readable summary text and a signature, with stop_reason: "compaction". You then remove the summarized messages, put the block first in messages, and send it on every later request with the same beta header.

%% title: On-demand compaction swap
sequenceDiagram
  participant App
  participant API as Messages API
  App->>API: messages + compaction summarize
  API-->>App: compaction block, stop_reason compaction
  App->>App: replace summarized messages with block
  App->>API: block first, then next user turn
  API-->>App: normal reply

Because your app controls the timing, on-demand compaction can keep recent turns word for word after the block, and it can run in the background while the conversation continues. Custom instructions (up to 16,384 characters) replace the default summarization prompt for that request. You cannot put compaction and context_management on the same request, so run clearing and compaction as separate calls.

import Anthropic from "@anthropic-ai/sdk";

type Msg = Anthropic.Beta.Messages.BetaMessageParam;

const client = new Anthropic();
const MODEL = "claude-sonnet-5-5";
const BETAS = ["compact-2026-09-04"];
const SYSTEM = "You are a coding agent working in a TypeScript monorepo.";
const COMPACT_AT = 300_000; // set near your real input budget

const KEEP = [
  "Keep: the task goal, decisions made and why, file paths touched,",
  "failing test names, open TODOs and anything the user said not to do.",
  "Drop: raw tool output that has already been acted on.",
].join(" ");

export async function turn(history: Msg[], userText: string): Promise<Msg[]> {
  history.push({ role: "user", content: userText });
  const reply = await client.beta.messages.create({
    model: MODEL,
    max_tokens: 8192,
    system: SYSTEM,
    betas: BETAS, // required on every request once a compaction block exists
    messages: history,
  });
  history.push({ role: "assistant", content: reply.content });

  const u = reply.usage;
  const contextSize =
    u.input_tokens +
    (u.cache_read_input_tokens ?? 0) +
    (u.cache_creation_input_tokens ?? 0) +
    u.output_tokens;
  if (contextSize < COMPACT_AT) return history;

  const summary = await client.beta.messages.create({
    model: MODEL,
    max_tokens: 8192,
    system: SYSTEM, // same system (and tools) as the rest of the conversation
    betas: BETAS,
    messages: history,
    compaction: { type: "summarize", instructions: KEEP },
  });

  // A failed summary still returns HTTP 200. Check stop_reason before swapping.
  if (summary.stop_reason !== "compaction") {
    console.warn("compaction skipped:", summary.stop_reason);
    return history;
  }

  // The block replaces every summarized message and must come first.
  return [{ role: "assistant", content: summary.content }];
}

The context size here adds up all three input fields. With caching on, input_tokens covers only the tokens after the last breakpoint, so checking it alone would badly undercount the real context size.

The on-demand docs warn about two mistakes that raise no error at all:

• Leaving summarized messages after the block. The request succeeds, but it sends the old content again alongside its summary, so you pay for both and gain no room.
• Dropping the block from a later request. The request also succeeds, but the model no longer has the summary, so everything it covered is gone.

A few more gotchas from the same docs:

• A failed summary still returns HTTP 200, with empty content. Check stop_reason before swapping.
• Images and documents inside the summarized range are gone after the swap. Re-attach anything the agent still needs.

Two usage and SDK notes:

• Billing is reported in usage.iterations. On a compaction response, the top-level input_tokens and output_tokens are 0.
• If you use the SDK tool runners, compactBeforeNextTurn() (TypeScript) and compact_before_next_turn() (Python) do the swap for you.

Threshold (beta header compact-2026-01-12). The API decides when, inside the request that crosses the trigger. It is the edit type compact_20260112 inside context_management.edits. The default trigger is 150,000 input tokens and the minimum is 50,000. instructions is optional. pause_after_compaction (default false) stops the response right after the block, so you can add content before the model continues.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();

export async function step(messages: Anthropic.Beta.Messages.BetaMessageParam[]) {
  const response = await client.beta.messages.create({
    model: "claude-opus-5-5",
    max_tokens: 8192,
    betas: ["compact-2026-01-12"],
    messages,
    context_management: {
      edits: [
        {
          type: "compact_20260112",
          trigger: { type: "input_tokens", value: 200_000 }, // default 150k, min 50k
          instructions: "Preserve file paths, decisions and failing test names.",
        },
      ],
    },
  });

  // Top-level usage excludes the compaction pass. Sum the iterations instead.
  const iterations = response.usage.iterations ?? [];
  const billedInput = iterations.reduce(
    (sum, it) =>
      sum +
      it.input_tokens +
      (it.cache_read_input_tokens ?? 0) +
      (it.cache_creation_input_tokens ?? 0),
    0,
  );
  const compacted = iterations.some((it) => it.type === "compaction");
  console.log({ billedInput, compacted });

  // Append the whole response (including any compaction block) and keep going.
  // The API ignores everything before the latest compaction block.
  messages.push({ role: "assistant", content: response.content });
  return response;
}

The main difference between the two modes is where the block goes. In on-demand mode the block goes first, and you remove the summarized messages yourself. In threshold mode the block follows the messages it summarizes, and the API ignores everything before it. To keep caching effective with threshold compaction, the docs suggest a cache_control breakpoint at the end of the system prompt, so the system cache survives compaction. You can also put one on the compaction block itself.

If you use Claude Code rather than the API, Claude Code Idle Compaction: What It Does and How to Turn It Off shows the same trade between cache expiry and compaction inside a product.

OpenAI: in-band or standalone#

OpenAI also offers two forms.

• In-band: set context_management: [{type: "compaction", compact_threshold: 200000}] on POST /responses. When the conversation crosses the threshold, the server emits a compaction item in the same response stream. It works with store: false, so it suits zero data retention (ZDR) setups.
• Standalone: POST /responses/compact (client.responses.compact) is fully stateless. It returns a new compacted window, which can include kept items as well as the compaction item. Pass that window unchanged to the next /responses call and do not prune it. The window you send for compaction must still fit the model's context window.

The key difference from Anthropic is the summary format. Anthropic's compaction block holds readable text plus a signature, so you can log it, audit it or show it to a user. OpenAI's compaction item is encrypted and opaque. With Anthropic you can inspect the summary. With OpenAI you never handle the summary contents, which suits ZDR setups, but you cannot check what survived.

OpenAI state: previous_response_id vs Conversations vs stateless#

How you compact on OpenAI depends on where the conversation state lives. The conversation state guide offers three options:

1. previous_response_id chaining. The server links each response to the one before it. Response objects are stored for 30 days by default.
2. The Conversations API. A durable conversation ID with no 30-day TTL, for threads that last longer than a month or move between devices.
3. Stateless input chaining. Set store: false and resend every output item yourself, including encrypted reasoning items.

One cost detail is easy to miss. Even with previous_response_id, every earlier input token in the chain is billed as input. Server-side state saves you bandwidth and bookkeeping, not tokens. Caching and compaction are what cut the bill.

%% title: Choosing OpenAI conversation state
flowchart TD
  A{"Must avoid stored state? ZDR or store false"} -->|Yes| B["Stateless input chaining + /responses/compact"]
  A -->|No| C{"Thread must outlive 30 days or span devices?"}
  C -->|Yes| D["Conversations API + compact_threshold"]
  C -->|No| E["previous_response_id + compact_threshold"]

With previous_response_id plus compaction, send only the new user message and never prune by hand, because the server already holds the compacted window. With stateless chaining, resend everything. You may drop items that come before the latest compaction item.

import OpenAI from "openai";
import { toResponseInputItems } from "openai/lib/responses/ResponseInputItems";

const client = new OpenAI();

// Option 1: server-side state + in-band compaction.
export async function chained(previousId: string | undefined, userText: string) {
  const response = await client.responses.create({
    model: "gpt-6-astra",
    previous_response_id: previousId,
    input: [{ role: "user", content: userText }], // only the new message
    context_management: [{ type: "compaction", compact_threshold: 200_000 }],
  });
  const details = response.usage?.input_tokens_details;
  console.log({
    cached: details?.cached_tokens,
    written: details?.cache_write_tokens,
    compacted: response.output.some((item) => item.type === "compaction"),
  });
  return response.id;
}

// Option 2: stateless (store: false), compact explicitly when you choose.
export async function compactStateless(window: OpenAI.Responses.ResponseInput) {
  const compacted = await client.responses.compact({
    model: "gpt-6-astra",
    input: window,
  });
  // Pass the compacted window as-is. Do not prune it.
  const next: OpenAI.Responses.ResponseInput = [
    ...toResponseInputItems(compacted.output),
    { role: "user", content: "Continue with the migration plan." },
  ];
  return client.responses.create({
    model: "gpt-6-astra",
    store: false,
    include: ["reasoning.encrypted_content"],
    input: next,
  });
}

One SDK detail: the output of responses.compact is typed as output items, so spreading it straight into input fails the type-check. Pass it through toResponseInputItems from openai/lib/responses/ResponseInputItems first. If you are still moving off the Assistants API, Assistants to Responses API: A Migration Field Guide explains the state model this builds on.

Lever 4: Retrieve just in time#

Anthropic's post describes "just in time" agents. They keep lightweight identifiers, such as file paths and stored queries, in context and load the actual data at runtime with tools. The context holds a pointer instead of the data. The data comes in only when needed, and Lever 2 can clear it afterwards.

Two provider features make this concrete.

The memory tool. Anthropic's memory tool is {type: "memory_20250818", name: "memory"}. It works on all Claude 4 and later models with no beta header. It runs client-side: your code implements view, create, str_replace, insert, delete and rename against a /memories directory. The docs recommend pairing it with compaction. Facts the agent writes to memory survive summarization. Facts that exist only in the conversation survive only if the summarizer keeps them.

Deferred tool loading. Tool definitions count toward the window too. Anthropic's tool search docs note that a setup with several MCP servers can use around 55k tokens on definitions alone. Use tool_search_tool_regex_20251119 or tool_search_tool_bm25_20251119, and set defer_loading: true on the tools the agent rarely needs. The model then sees a search tool and loads matching definitions on demand. You can defer up to 10,000 tools, and each search returns 5 matches by default. The docs say this typically cuts definition tokens by over 85 percent. Deferred tools are left out of the prefix, so the cache still works. OpenAI's tool search with defer_loading: true makes the same cache-friendly choice: tools it discovers are added at the end of the context, so the earlier cached content stays intact.

The rule of thumb on both providers: anything that changes often belongs at the end of the prompt or behind a tool call. Never put it in the middle of the cached prefix.

Lever 5: Isolate side work in subagents#

Some tasks read far more than they need to report. Examples: scanning a 40,000-line CI log, grepping a codebase for every caller of a function, or reviewing a long diff. Doing that in the main thread fills its context with material that matters for one step. A subagent does the reading in its own context window and returns a summary.

The Claude Agent SDK keeps the two contexts strictly apart. For a non-fork subagent, the only thing passed in from the parent is the Agent tool's prompt string. The subagent's tool calls and results stay inside it, and only its final message comes back. Anthropic's engineering post says subagents often return a condensed summary of 1,000 to 2,000 tokens, even after exploring with tens of thousands.

%% title: Subagent context isolation
sequenceDiagram
  participant Parent
  participant Sub as log-reader subagent
  participant Tools
  Parent->>Sub: Agent tool prompt string only
  loop many reads
    Sub->>Tools: Grep, Read
    Tools-->>Sub: large log chunks
  end
  Sub-->>Parent: 10-line report
  Note over Parent: parent context grows by the report, not the logs

import { query } from "@anthropic-ai/claude-agent-sdk";

for await (const message of query({
  prompt:
    "The nightly build failed. Use the log-reader agent to find the root cause, then propose a fix.",
  options: {
    allowedTools: ["Read", "Grep", "Glob", "Agent"],
    maxBudgetUsd: 2,
    agents: {
      "log-reader": {
        description:
          "Reads large CI and server logs and reports the first real error. Use for any log over a few hundred lines.",
        prompt:
          "Find the first causal error in the logs you are pointed at. Reply in at most 10 lines: the error, file and line, and the evidence. Do not paste raw log output.",
        tools: ["Read", "Grep", "Glob"], // read-only, and no Agent tool
        model: "haiku",
        maxTurns: 15,
        autoCompactWindow: 120_000,
      },
    },
  },
})) {
  if (message.type === "result" && message.subtype === "success") {
    console.log(message.result);
  }
}

These AgentDefinition fields matter most for context:

• tools: a read-only list stops the subagent from spawning its own subagents or editing files.
• model: a cheaper model for bulk reading.
• maxTurns: a cap on how many turns the subagent can take.
• autoCompactWindow: lowers the token count at which the subagent compacts its own conversation (100,000 to 1,000,000 tokens; it can only lower the window the subagent would otherwise inherit). Requires TypeScript SDK v0.3.296+.

The SDK also limits how far subagents can spread. CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH defaults to 3 and CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS defaults to 20. maxBudgetUsd has no default limit, so set one. In transcripts, the tool appears as Agent in tool_use blocks and as Task in the system:init tools list.

Subagents have real costs. Each one has its own prefix, so it pays for its own cache writes. The parent sees only the report. If the subagent summarizes badly, the detail is lost, and the parent's context has no copy to recover it from. The main defence is a tight output format in the subagent's prompt ("at most 10 lines: error, file and line, evidence").

A default recipe#

For a new agent on either provider, this order gets most of the benefit with the least risk:

1. Order and cache the prompt. Put stable instructions first, then tools, then content that changes. Add a long-TTL breakpoint after the stable part and let automatic or implicit caching follow the conversation.
2. Measure before tuning. On Anthropic, compare cache_read_input_tokens with cache_creation_input_tokens. On OpenAI, compare input_tokens_details.cached_tokens with cache_write_tokens. If reads are a small share, something in your prefix is changing.
3. Turn on provider compaction with your own instructions. Name what must survive: the goal, decisions and their reasons, file paths, failing tests and user constraints.
4. Add tool-result clearing once tool output dominates. On Anthropic, use clear_tool_uses_20250919 with a clear_at_least floor, and exclude_tools for results the agent must keep.
5. Move lasting facts out of the conversation. Use the memory tool or plain files, so the facts survive any summary.
6. Hand exploration to subagents with limits. Give them read-only tools, a cheap model, maxTurns and a budget.

Verify It Works#

Each lever leaves a trace in the response. Check for it before you assume a lever is working:

• Caching (Anthropic): usage.cache_read_input_tokens is above zero on the second request with the same prefix. Total input is cache_creation_input_tokens plus cache_read_input_tokens plus input_tokens.
• Caching (OpenAI): usage.input_tokens_details.cached_tokens is above zero, and cache_write_tokens shows up on the request that created the cache entry.
• Context editing: context_management.applied_edits lists cleared_tool_uses and cleared_input_tokens. An empty list means the trigger or the clear_at_least floor was not reached.
• Anthropic compaction: a stop_reason of "compaction" in on-demand mode, and a compaction entry in usage.iterations in either mode.
• OpenAI compaction: an output item with type: "compaction" in the response, or in the output of responses.compact.
• Subagents: the parent's input tokens grow by about the size of the subagent's report, not by the size of what the subagent read.

The five TypeScript snippets in this post were type-checked with tsc --noEmit (strict, moduleResolution: bundler) against @anthropic-ai/sdk@0.135.0, openai@7.36.0 and @anthropic-ai/claude-agent-sdk@0.3.302, using TypeScript 7.0.2. They were not run against the live APIs.

Troubleshooting#

400 compaction_block_misplaced. On-demand compaction found summarized messages in front of the block. The block must come first, either as its own assistant message or as the first content block of the first message.

400 on a later request: compaction is not one of the expected content block types. That request left out the beta header. Every request that carries the block needs compact-2026-09-04 (or compact-2026-01-12 in threshold mode), not just the request that created it. The error message does not mention the header, so this is easy to misread as a malformed block.

Python requests with a compaction block are rejected. A plain model_dump() adds citations: null and text: null to the block. Use client.beta.messages directly, or serialize with to_dict() or model_dump(exclude_none=True).

Empty summary with HTTP 200. The compaction call hit max_tokens or ended some other way. Check stop_reason. Anything other than "compaction" means you should not swap. max_tokens also covers any thinking the summarizer does, so allow several thousand tokens. If the last assistant turn ends in a tool call that has no result yet, the request is rejected, so send pending tool results first.

Cache hit rate drops right after compaction. This is expected on both providers. The summary is new content, so everything after it has to be written to the cache again. OpenAI's caching guide says to compare total input cost before and after, not just the hit rate. On Anthropic, a breakpoint at the end of the system prompt keeps that part cached through compaction.

Nothing is cached at all. The prefix is under the minimum (512 tokens on most Claude 5.x models, 1,024 on Sonnet 5 and on OpenAI GPT-5.6+), and it is skipped without an error. On Anthropic, the cache lookback also checks at most 20 positions per breakpoint, so a breakpoint far from the last cached position can miss.

OpenAI cache misses after switching to explicit mode. In mode: "explicit" there is no implicit breakpoint, so a cache entry written earlier by an implicit breakpoint may not line up with your explicit breakpoints. Expect one round of new writes after the switch.

FAQ#

What is context engineering for AI agents?#

Anthropic defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference." For an agent, that means deciding what goes into every request: a cached stable prefix, a trimmed or compacted history, data loaded just in time, and side work pushed into subagents.

Is compaction the same as context editing?#

No. Anthropic context editing (clear_tool_uses_20250919, clear_thinking_20251015) removes specific stale blocks such as old tool results and leaves everything else word for word. Compaction replaces a run of history with a summary block. Clearing is lighter and loses less. Compaction frees more room but is lossy.

Does prompt caching reduce context window usage?#

No. Cached tokens still take up space in the context window. Caching changes what you pay for the repeated prefix and how fast it is processed. It does not change whether those tokens count toward the limit. To shrink the window you need clearing, compaction, retrieval or subagents.

Should I use previous_response_id or send the full history to OpenAI?#

Use previous_response_id with compact_threshold for simple stored threads. Use the Conversations API for durable threads with no 30-day TTL, and stateless chaining with store: false for zero data retention. All prior input tokens in a previous_response_id chain are still billed as input, so it saves bandwidth and bookkeeping, not tokens.

When should an agent use subagents instead of compaction?#

Use a subagent when a side task will read far more than it needs to report, such as scanning logs or exploring a codebase. The subagent uses up its own context and returns a short summary. Use compaction when the main thread itself has grown long and the problem is the history, not a side task.

Sources#

• Anthropic, Effective context engineering for AI agents: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
• Claude docs, Context windows: https://platform.claude.com/docs/en/build-with-claude/context-windows
• Claude docs, Prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching
• Claude docs, Context editing: https://platform.claude.com/docs/en/build-with-claude/context-editing
• Claude docs, Compaction overview: https://platform.claude.com/docs/en/build-with-claude/compaction
• Claude docs, Compaction on demand: https://platform.claude.com/docs/en/build-with-claude/compaction-on-demand
• Claude docs, Threshold compaction: https://platform.claude.com/docs/en/build-with-claude/compaction-threshold
• Claude docs, Memory tool: https://platform.claude.com/docs/en/agents-and-tools/tool-use/memory-tool
• Claude docs, Tool search tool: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool
• Claude Agent SDK, Subagents: https://code.claude.com/docs/en/agent-sdk/subagents
• OpenAI, Conversation state: https://developers.openai.com/api/docs/guides/conversation-state
• OpenAI, Compaction: https://developers.openai.com/api/docs/guides/compaction
• OpenAI, Prompt caching: https://developers.openai.com/api/docs/guides/prompt-caching

Continue Reading#

• Context Engineering: The Highest-Leverage Skill in AI-Assisted Development - the file-level half: CLAUDE.md, system prompts and skills.
• The 98% Context Reduction Pattern - keep intermediate state in code and files, and return receipts instead of full output.
• Claude Code Idle Compaction: What It Does and How to Turn It Off - the trade between cache expiry and compaction inside Claude Code.