Tool context and prompt caching

Two things are true of every request the built-in harness makes: the tool definitions are the largest fixed cost in it, and almost none of them will be used. This page explains what Control Center does about that, and why the answer is not simply “send fewer tools”.

The problem is attention, not bytes

Control Center registers about 126 tools. Sent in full, their definitions cost roughly 23,000 tokens on every request — a cost no compaction can touch, because tool definitions travel in a separate part of the request from the conversation.

The token bill is the obvious complaint, and with prompt caching it is also the smaller one. The real cost is selection accuracy. Published evaluations of tool-calling agree that models pick the right tool reliably up to somewhere around 30–50 tools in context, and get progressively worse beyond that. The named cause is not the token count but semantic overlap: once a catalogue contains several tools whose descriptions read alike, the model is choosing between look-alikes rather than recalling a capability.

At 126 tools, an agent is well past that band on every single turn — including the many turns where it only needed to read a file.

The two-tier surface

A run is therefore shown two things:

• a resident set — roughly two dozen tools, sent with full schemas on every request;
• a name index of everything else, grouped under headings like “Tickets and projects” or “Memory and knowledge”, costing a few hundred tokens for the whole catalogue.

Measured on the current catalogue, that takes the tool block from ~23.1k tokens to ~5.0k, about 78% smaller, and puts the always-visible set inside the band where selection is reliable.

The resident set is the tools a run uses in most runs: the file and shell vocabulary (read, write, edit, search, bash, …), the discovery tools, the task checklist, memory search, and whatever verb the current mode must call to deliver its output. A mode’s own output verb is always resident — a run must never have to go looking for the one call that produces its deliverable.

Deferred does not mean unavailable

This is the part that makes the design safe rather than merely cheap. A deferred tool is callable from the first turn. Only its schema is withheld.

The agent sees the name in its prompt. If it calls that name directly, the harness loads the schema and runs the tool in the same step — no error, no round trip. The model does not need to know that the tool was ever “unloaded”.

When the agent knows the task but not the name, search_tools takes a plain-language description (“assign a ticket to someone”), ranks the run’s whole surface, and returns the matches with their schemas already loaded, so the very next turn can call one. If a search finds nothing, it says so plainly and suggests different wording or a wider search — deliberately, because an agent that reads an empty search as “this capability does not exist” will invent a substitute instead of looking again.

What deferral never does

Deferral is applied after the mode’s tool filter, never instead of it. It hides a schema; it cannot reveal a tool the mode denied. An activated tool still passes the approval prompt and the action guardrails exactly as a resident one would, and a tool cannot load a name that was not on the run’s surface to begin with.

The MCP surface external clients see is not affected. tools/list still returns the complete catalogue. An earlier attempt to trim it broke agents outright: MCP clients validate tool names against their own cached list and refuse anything unlisted without ever asking the server, so a “hidden but callable” tool was simply unreachable. Deferral lives in the harness, where loading a tool mid-run is something the harness can actually do.

Turning it off

--tool-deferral=off (or CC_SERVER_TOOL_DEFERRAL=off) makes every admitted tool resident again, producing exactly the requests the server made before deferral existed. See the cc_server CLI reference.

Prompt caching

Agent workloads are lopsided: they send enormous prompts and receive small answers, turn after turn, with the same prefix each time. Providers cache that prefix, and a cache read costs about a tenth of a fresh one. Cache hit rate is therefore a first-class property of the harness rather than an optimisation — and on Anthropic it also buys throughput, because cached tokens do not count against the per-minute input limit.

Caching works on an exact prefix match, which yields one governing rule.

Never rewrite what was already sent

Any byte that changes invalidates the cache from that point onward. So the harness only ever appends: new messages at the end, and newly activated tool schemas after the cached tool block rather than merged into it. That ordering is the reason activation is cheap — the cached prefix stays identical and the run pays only for the schemas it just loaded.

The same rule reshaped an existing behaviour. The loop trims stale tool output from older turns to reclaim context; it used to do that on every tool-bearing turn, editing the middle of the conversation each time. That reclaimed a few dozen tokens and invalidated the cache for everything after the edit, permanently. It now waits until there is enough to reclaim to be worth the rewrite, making it a rare deep trim instead of a continuous leak.

Where the breakpoints go

A provider allows a limited number of cache markers, and Control Center spends them deliberately:

Marker	Covers	Lifetime
Last resident tool	The tool block	1 hour
Last system block	Tools + system prompt	1 hour
Previous turn’s tail	History up to the last request	5 min
Current turn’s tail	The full history	5 min

The last two are a rolling pair, and the reason for the pair is subtle: a cache lookup searches backwards a bounded number of blocks for something an earlier request stored. A single turn that runs many tools in parallel can emit more blocks than that window, at which point one tail marker quietly misses and the request pays full price with nothing in the response to say so. Marking the exact spot the previous request stopped makes the hit structural instead of a race.

The tools-and-system prefix takes the long lifetime because it is constant for the whole run, is shared by every run and subagent with the same shape, and because the clock starts when a request begins — a turn that streams for several minutes has already spent most of a short window before the next request forms.

Subagents share one prefix

Subagents of the same type produce byte-identical prompt prefixes, so they should read one cache entry rather than each writing their own. But an entry only becomes readable once the request that wrote it starts responding, so a fan-out launched all at once has every child paying the premium for a prefix they all share.

The harness lets the first child of a given shape start, then releases its siblings as soon as it begins streaming. The wait is bounded: a slow first child costs a cache hit, never the fan-out itself. For the same reason, every subagent gets the same resident tool set regardless of its profile — varying it per profile would fragment the shared prefix for no gain, since the profile has already decided what the child may call.

Watching it work

Both halves are observable rather than assumed:

• The context explorer on a conversation reports the resident tools, the deferred index and what each withheld schema would cost, built from the same code path a real run uses.
• Each run records which deferred tools it loaded and what triggered it, so a search that finds nothing is visible as a retrieval miss rather than disappearing into an agent quietly doing something adjacent.
• Observability reports cache effectiveness as cache reads over all prompt tokens processed, cache writes included — a run that keeps rebuilding its prefix scores low, instead of scoring like one that never needed a cache.