AI Agent Caching: Four Production Patterns That Cut LLM Costs

AI agent caching is the most overlooked cost lever in production LLM systems. The average UK service business running two to three AI agents in production pays three to five times more than necessary on API costs — not because of model selection or prompt length, but because the same work is being done repeatedly without a cache in between. NeurIPS 2025 research on agentic plan caching showed 50% cost reduction in production agent systems. Draup's May 2026 engineering case study reported 25% cost savings on a live workload with zero quality degradation. The difference between those firms and the ones still burning through their API budget every month is four patterns. Here they are.

Why AI Agents Are Paying 3x More Than They Should

Token costs in agent workflows are not linear. They multiply. A simple three-step agent — classify, retrieve, respond — might call the LLM six or eight times per run: once to classify the input, once to generate retrieval queries, once or twice to evaluate retrieved results, once to compose the response, and once to check it. If each call carries a 2,000-token system prompt, and the agent runs 500 times a day, that is six million tokens spent on system prompts alone — before a single token of actual work has been done.

The three sources of waste are predictable:

• Repeated system prompts. Every LLM call re-processes the same instructions, persona, and constraint text on every request. At Claude 3.5 Sonnet pricing, a 2,000-token system prompt costs roughly £0.006 per call. At 500 calls per day, that is £3 per day — £90 per month — just in system prompt processing, before counting the actual task tokens.
• Semantically identical queries. Many agent workloads are highly repetitive. An email triage agent, a lead qualification agent, or a client reporting agent will receive inputs that are semantically similar — different specific details, same underlying task structure. Without a cache, each one triggers a full LLM call.
• Repeated reasoning paths. When an agent solves a class of problem it has solved many times before — extracting a meeting time from an email, summarising a client brief, formatting a cost estimate — it re-derives the same plan every time. Agentic plan caching eliminates this.

The cost optimisation post covers the Cache, Route, Compress framework that treats caching as one of three major cost levers. This post goes deeper on the caching pillar specifically — because for most UK service businesses, it is the one with the fastest payback.

At 500 agent runs per day, a 2,000-token system prompt costs £90 per month before a single token of actual work is processed. Prompt caching eliminates that cost in under an hour of configuration.

Pattern One: Prompt Caching (The Low-Hanging Fruit)

Claude's prompt caching API is the fastest win available to any UK service business running agents on Anthropic models. It works by marking specific portions of the prompt as cache-control breakpoints. Anthropic caches those portions on their infrastructure and charges a 10% read fee on subsequent calls that hit the cache — compared to the full input token price for a cold read. The cache TTL is one hour, meaning any agent that runs more than once per hour on the same system prompt sees immediate savings.

The implementation is minimal. In an API call to Claude, you add a cache_control parameter to the message blocks you want cached:

• Your system prompt — always cache this. It changes rarely and it is the largest single repeated cost.
• Large tool definitions — if your agent has ten tools with detailed schemas, those definitions add 1,000+ tokens per call. Cache them.
• Few-shot examples — if you use three to five examples to shape output format, cache those too. They are large, repetitive, and unlikely to change between calls.

The practical savings depend on system prompt length and call frequency, but for a typical service business AI operating system, prompt caching reduces input token costs by 70-90% on the cached portions. For a system prompt of 3,000 tokens called 1,000 times per day, the reduction moves from processing 3,000,000 tokens per day to 300,000 — a saving that pays for itself in the first billing cycle.

One caveat: the cache is per-prefix. Anything that changes between calls — the user's query, the dynamic context injected for that specific request — must come after the cached block. If your current agent architecture interleaves dynamic content with system instructions, the context engineering post covers the structural patterns that separate static from dynamic context efficiently.

Pattern Two: Semantic Result Caching

Prompt caching reduces the cost of calling the LLM. Semantic result caching eliminates the LLM call entirely for queries that are semantically similar to ones the system has already answered.

The mechanism is vector-based. When an agent receives a query, it embeds the query using a small, cheap embedding model. It then checks a vector database for similar queries it has answered before. If a sufficiently similar query exists in the cache — above a similarity threshold you set — it returns the cached result without calling the LLM at all. If no similar query exists, it proceeds normally and stores the result for future use.

This pattern is most effective for agents that handle high-volume, narrow-domain queries. An email classification agent that sees hundreds of similar enquiry types per week is a strong candidate. A client FAQ agent that answers recurring questions across a client book is another. For an agent handling genuinely diverse, novel inputs every run, the cache hit rate will be low enough that the overhead of the embedding lookup outweighs the savings — do not apply this pattern universally.

The practical implementation stack for UK service businesses running n8n and Claude:

1. Embed each incoming query using a small embedding model (text-embedding-3-small or a self-hosted all-MiniLM-L6-v2 via HuggingFace)
2. Check a vector database — Qdrant or Pinecone both work — for similar past queries at a similarity threshold of 0.92 or higher for professional service contexts where precision matters
3. On a cache hit above threshold, return the cached result and log it as a hit for monitoring
4. On a cache miss, run the LLM call, store the result with its embedding, and proceed normally

A 0.92 similarity threshold sounds conservative, but for production service business agents it is necessary. Professional contexts — compliance queries, client communications, proposal drafting — require high confidence that a cached answer genuinely applies to the new query. Track your cache hit rate: anything above 25% for a new agent, rising to 40%+ within a month, indicates the pattern is working well for your workload.

Pattern Three: Agentic Plan Caching

Agentic plan caching is more recent than the first two patterns and represents the most architecturally interesting development in agent efficiency engineering. The idea, formalised in a NeurIPS 2025 paper, is to cache not just the final result but the plan the agent derived to solve the problem — the sequence of reasoning steps and tool calls — and reuse it for structurally similar new requests.

This matters because LLM reasoning is expensive. A multi-step agent that plans its approach before executing spends a significant portion of its token budget on the planning step alone. If the planning output is highly similar across a class of similar inputs — and for service business agents, it often is — that planning budget can be amortised across many runs rather than paid fresh each time.

The NeurIPS research showed 50.31% cost reduction and 27.28% latency reduction across multiple real-world agent applications. The savings are largest for agents with complex, structured planning phases — workflow agents that need to decide which of ten possible paths to take, or research agents that need to decompose a brief into sub-tasks before executing.

Practical implementation without specialised tooling:

• After a successful agent run, extract the planning steps as a structured template — replacing specific client details with placeholder variables
• Store the template with the class of query it was derived from, embedded and indexed
• When a new query arrives, check for a matching plan template; if found, adapt it to the new specific details using a small, cheap model rather than a full reasoning call
• Validate the adapted plan before executing — a lightweight confidence check prevents the rare case where a template has been applied to a query it does not actually fit

This pattern pairs well with the state machine architecture, where plans map cleanly to defined state transitions and the template validation step aligns naturally with state entry conditions.

Pattern Four: Workflow Checkpoint Caching

The first three patterns reduce cost on successful agent runs. Checkpoint caching addresses the cost of failure.

When a multi-step agent fails partway through — a tool call returns an error, a network timeout occurs, an external API rate-limits the agent — the default behaviour is to retry from the beginning. For a five-step agent where steps three to five are expensive LLM calls, a failure at step four means paying for steps one through three twice. At scale, partial retries are a meaningful hidden cost in most production agent stacks.

The checkpoint pattern saves the agent's intermediate state after each successful step. If the workflow fails, it restarts from the last successful checkpoint rather than from the beginning. This is not just a cost control mechanism — it is a reliability mechanism. The long-running agents post covers the Checkpoint-Restore pattern as one of four async execution approaches. Applied to cost control, the same pattern has direct economic impact.

For UK service businesses running n8n:

• After each major step, write the current execution state — inputs processed, outputs so far, current position in the workflow — to a datastore (n8n's built-in data store, Supabase, or Redis all work)
• At the start of each step, check whether a checkpoint exists for this run and skip already-completed steps if one is found
• Set a checkpoint TTL long enough to cover your retry window — 24 hours for most service business workflows is appropriate
• Clear checkpoints on successful completion to avoid unnecessary storage growth

The compounded saving is significant for agents with expensive early steps. A document generation agent that ingests a 20,000-token knowledge base before drafting — and which is occasionally interrupted by upstream timeouts — pays for that ingestion on every retry without checkpoint caching. With it, the expensive ingestion step runs once per run, and only the failed step is repeated.

Building Your Caching Stack This Week

The four patterns stack. You do not need to implement all of them simultaneously, and not every agent will benefit from all four. A practical decision framework:

1. Start with prompt caching on every agent you run on Claude. It is a configuration change, not a code change, and it has a positive return from the first day. No agent that calls Claude more than once per hour should be running without it.
2. Add semantic result caching to agents with high-volume, narrow-domain inputs. Email triage, FAQ answering, and lead classification are the first candidates. Skip it for creative or highly variable workloads — the hit rate will not justify the overhead.
3. Evaluate agentic plan caching for agents with complex planning phases. If your agent spends more than 20% of its token budget on planning and routing decisions, this pattern has high value. If the work is straightforward tool execution with minimal planning, skip it.
4. Add checkpoint caching to any multi-step agent that runs for more than 30 seconds or calls external APIs. The retry cost exposure on long workflows is real and the implementation is low-complexity.

For a two-agent production stack running 500 times per day, implementing all four patterns typically moves LLM spend from £400-600 per month to £150-250 — without changing a single output the agents produce. The CI/CD pipeline post covers how to verify that caching changes have not affected output quality, which matters in professional service contexts where outputs go directly to clients. The token budget management post covers the complementary patterns around per-run caps and cost attribution that give you visibility into where spend is going before you start cutting it.

If you want help designing a caching architecture for your AI operating system — identifying which patterns apply to your agents, estimating the savings, and wiring the implementation — book a free 30-minute call with the Quantum Flow team. We build AI operating systems for UK service businesses, and the caching layer is typically where we recover the most cost in the first month.