Most AI agents deployed by UK service businesses are spending 40–60% of their token budget on context they do not need. The fix is not a better model or a shorter system prompt. It is context engineering: the discipline of designing exactly what information enters the model's attention window on every single call. Four patterns cut token costs by up to 80% while making agents more reliable, not less.
Why Most Agent Token Bills Are a Context Problem
When an AI agent runs over budget, the first instinct is to blame the model. Swap Claude Sonnet for Haiku. Shorten the system prompt. Add token limits. None of these fixes the underlying problem.
The underlying problem is that most agents were built with an implicit assumption: more context means better outputs. In practice, after a long workflow run, agents are sending the full history of every tool call, every result, every intermediate thought, and every previous turn — a growing context window that costs money and, beyond a certain point, actively degrades quality as the model's attention dilutes over irrelevant information.
Context engineering is the discipline of deciding what goes into that window. Not prompt engineering — the craft of writing better instructions. Context engineering is the architecture of what information is visible to the model at each point in a workflow, and why.
The data from production deployments supports this. Agents using structured context management see median context size reductions of around 60% with no capability loss. Smart memory systems cut token costs 80–90% while improving response quality by 26%. The gains come not from using cheaper models, but from sending less irrelevant data to any model.
Most AI agent token costs are not a model problem. They are a context problem. The model is processing exactly what you told it to — the issue is you told it too much.
Pattern One: Tiered Memory — Stop Sending Everything Every Time
Production agents use three layers of memory, but most UK service business deployments conflate them into one: a growing conversation history appended to every call. The three-tier model works differently.
Working memory is the active context window — what the model sees right now. It should contain only what is needed for the current step: the task at hand, the relevant constraints, and the most recent tool results.
Episodic memory is session state — everything that happened in this workflow run. It lives in a database or file, not in the context window. The agent retrieves specific pieces of episodic memory when they become relevant, rather than loading all of it on every call.
Semantic memory is long-term knowledge — your firm's processes, client profiles, product information — stored in a vector database and retrieved via RAG when needed. It never sits in the full context window unless retrieved for a specific step.
The practical implementation: maintain a session state object that accumulates tool results and decisions. On each model call, send only the current task prompt, the relevant constraint set, and the last two or three meaningful steps — not the full history. When the agent needs something from earlier in the run, retrieve it by reference.
A workflow agent that previously sent 80,000 tokens of accumulated history on its final summary call drops to 12,000 tokens of working memory using this pattern. Same output quality. 85% less spend on that call. This is distinct from the cross-session memory problem covered in AI Agent Memory Architecture — that post addresses what persists between sessions. This pattern is about what you send to the model within a single run.
Pattern Two: Just-In-Time Retrieval Over Stuffed Context
The most common context engineering mistake in UK service business deployments is loading everything that might be relevant into the system prompt at the start of the workflow. A client onboarding agent that loads the full client profile, the firm's complete onboarding playbook, all previous correspondence, and the full service catalogue into its initial context is carrying 30,000 tokens of background on every call — most of which is never referenced on most calls.
Just-in-time retrieval flips this architecture. The system prompt stays lean — role definition, core constraints, output format. Knowledge is retrieved the moment it is needed, using the same RAG architecture that grounds agents in your firm's knowledge base. The agent calls a retrieval tool at the point where specific information is required, pulls only the relevant chunks, and those chunks leave working memory once the step is complete.
This mirrors the discipline in production tool calling — tools are invoked just-in-time, not pre-loaded. Context retrieval should follow the same principle.
A client reporting agent built on this pattern reduced its average context per call from 22,000 tokens to 6,500. Cost fell by 70%. Output quality improved because the model was attending to thousands of tokens of directly relevant material rather than a sea of possibly outdated background.
The practical rule: nothing enters the context window on the basis of "it might be useful". Everything in the context window should be there because it is definitely needed for this specific step.
Pattern Three: Constraint Re-injection to Prevent Drift
Long workflows produce a well-documented failure mode: instruction drift. An agent that starts a 15-step workflow following its output format precisely begins to vary it by step 10. Critical constraints defined at the top of the system prompt are being crowded out by the accumulated history of earlier steps.
The fix is constraint re-injection: a lean block of critical constraints re-appended at the end of every context window, immediately before the model generates its response. Not a repeat of the full system prompt — just the hard requirements: output format, prohibited behaviours, escalation triggers, and any active constraints specific to this workflow run.
In practice this adds 150–300 tokens per call — a negligible cost. What it prevents is the single most common failure mode in multi-step production agents: a workflow that runs correctly through 12 steps and then produces malformed output on step 13 because its format constraints had been diluted by accumulated history.
This matters especially for UK service businesses where agents produce client-facing documents. A proposal-writing agent that drifts from its approved structure on the final section, or a contract review agent that stops flagging a particular clause type halfway through a document, creates more problems than it solves. Constraint re-injection keeps behaviour predictable at every step, not just the first. It pairs directly with the human-in-the-loop pattern: the constraints re-injected typically include exactly when to escalate for human review.
Pattern Four: Structured Error Compression
Tool failures are one of the biggest hidden drivers of context bloat. When a tool call fails, most agent implementations add the full error output to context — sometimes hundreds of lines of stack trace, API response body, or raw HTTP errors. By the time an agent has encountered three or four tool failures in a workflow, it may be carrying several thousand tokens of error text that the model cannot act on meaningfully.
Structured error compression solves this at the plumbing layer. When a tool call fails, a compression function runs before the result enters context. The model receives one structured object:
- Error class — what category of failure this is: auth error, rate limit, not found, timeout, parse error
- Retry eligibility — whether retrying is likely to succeed
- Action prompt — a single sentence describing what the agent should do next
The full error output goes to a log file. If the model needs more detail to diagnose the failure, it calls a read-log tool to retrieve it on demand. Most of the time it does not need to.
This single pattern reduced median context size by roughly 60% in one production deployment — because that agent was calling external APIs frequently, and API failures were generating several kilobytes of JSON error payloads per failure event. The model received {"error": "rate_limit", "retryable": true, "action": "Wait 60s and retry"} instead of 800 tokens of raw HTTP response.
Structured error compression pairs naturally with the fault tolerance patterns — retry budgets, circuit breakers, and fallback paths all work better when the context they operate from is clean rather than cluttered with raw error payloads. And it complements the tool calling patterns that prevent many failures from occurring in the first place.
Putting It Together: The Context Budget
The practical way to apply these four patterns is to define a context budget before writing a single line of agent workflow. A context budget specifies the maximum token allocation for each block in the context window:
- System prompt — role definition, core constraints, output format: 1,500–2,500 tokens
- Session history — last two or three meaningful steps only: 2,000–4,000 tokens
- Retrieved context — just-in-time retrieval per call: 2,000–5,000 tokens
- Constraint re-injection block — hard requirements appended before generation: 200–300 tokens
- Tool results — current step only, compressed on failure: 1,000–2,000 tokens
A typical business workflow agent runs on a 8,000–12,000 token context window with this budget. An agent without a budget routinely hits 60,000–100,000 tokens on complex workflow runs. The cost difference is not marginal.
Define those numbers before you build. Then enforce them with a context management layer that measures the token count of each block before assembling the final prompt, trims or summarises where limits are exceeded, and logs overages for review. This is not exotic engineering — it is the same discipline applied to memory allocation in any software system. You would not write a database query that loads an entire table to find one row. The same principle applies to context.
The planner-executor pattern works in part because the executor receives only the current step and its constraints — not the planner's full reasoning history. The deployment pattern for safe agent updates relies on consistent, predictable context structures that can be versioned and tested. Context engineering is the foundation these patterns sit on.
The agent that costs 80% less to run is not using a cheaper model or a shorter system prompt. It is using a smaller, cleaner context window on every single call — by design, not by accident.
If your AI agents are running over budget or producing inconsistent outputs in multi-step workflows, context engineering is almost certainly part of the answer. It is the layer most builds skip because it is invisible in a demo — but it is the first thing that shows up on your API invoice. If you want to review how your current agent architecture handles context, or design a new deployment with a proper context budget from the start, get in touch. We build production AI operating systems for UK service businesses that hold up well beyond the pilot.