LLM token prices dropped 80% last year. Your AI agent bill is still too high. Here is why — and the three-pillar engineering framework that UK service businesses use to fix it without touching output quality.
Something counterintuitive is happening across every AI engineering team right now. The cost per million tokens from the major providers has collapsed since 2023 — GPT-4-class capability that cost $30 per million input tokens now costs $3. Google's flash models price in at under $0.50. Anthropic's Haiku is cheaper still. By any reasonable measure, AI should be getting cheaper to run.
And yet. AI inference now accounts for over 85% of enterprise AI budgets. Teams that deployed their first agents in late 2024 have watched monthly bills grow from hundreds to tens of thousands of pounds — not because providers raised prices, but because usage exploded. A single agentic task — planning a step, calling tools, verifying results, refining an output — can consume 50,000 to 500,000 tokens. Run that across a full working day of agent activity, and the maths compounds fast.
This is the AI cost paradox: capability got cheaper per token, but cost per business outcome did not fall at the same rate — because outcomes now require far more tokens. The good news is that the gap is an engineering problem, and engineering problems have solutions. The framework below is what we use in production for UK service business AI operating systems and what we install as part of every build.
The Cost Paradox No One Warned You About
To understand why costs stay high despite falling prices, you need to understand what an agentic task actually costs — not in theory, but in token terms.
A simple single-turn chatbot response might use 500 tokens. A basic RAG retrieval plus response is closer to 2,000–5,000 tokens: the system prompt, the retrieved context chunks, the conversation history, and the model's output. But a full agentic loop — where an agent receives a task, reasons about it, selects tools, makes API calls, reads results, checks its work, and produces a structured output — can easily consume 50,000 to 200,000 tokens per task cycle.
Multiply that by the number of tasks your agent runs each day. An email triage agent processing 80 emails, a proposal agent generating three proposals, a client reporting agent compiling six reports — across a typical five-person service business AI operating system, the daily token count can reach 5–10 million tokens without anyone noticing. At frontier model prices, that is £50–100 per day before you have billed a single client for the work.
The problem is not the price of tokens. The problem is the number of tokens consumed by agentic tasks that were designed without cost in mind.
The solution is not to use cheaper models everywhere — that path leads to degraded output quality and lost client trust. The solution is architecture: building systems that use expensive tokens sparingly, cheap tokens strategically, and cached tokens whenever possible. That is what the three-pillar framework delivers.
The Three-Pillar Framework: Cache, Route, Compress
Teams combining all three pillars report 47–80% reductions in monthly LLM spend. The pillars work independently — you can implement them in any order — but their effects compound when all three are in place.
Pillar 1: LLM Routing
Routing is the practice of sending each task to the cheapest model capable of handling it well, rather than sending everything to your best frontier model by default. In practice, it means classifying each incoming task before the main model call and directing it to the appropriate tier.
A practical routing structure for a UK service business AI operating system looks like this: a flash-tier model (Google Gemini Flash, Claude Haiku, GPT-4o-mini) handles approximately 70% of tasks — email classification, template-driven document drafts, structured data extraction, routine CRM updates. A mid-tier model handles 20% — tasks requiring nuanced judgement, client-facing writing that must match a specific tone, or reasoning across multiple documents. Your frontier model handles the remaining 10%: complex proposals, sensitive client communications, multi-step analysis with high error cost.
The economics are stark. If your current average cost is £15 per million tokens (frontier model used for everything), routing to this structure brings your blended average below £2.50 per million tokens — an 83% reduction, with quality preserved exactly where it matters. This mirrors the architecture pattern described in our post on multi-agent orchestration, where specialised agents handle defined task categories rather than one generalised agent handling everything.
Pillar 2: Prompt Caching
Prompt caching is the most underused cost lever in most AI operating systems, and it delivers savings from day one with minimal implementation effort.
Every call to your AI agent includes a system prompt — often a long one, containing role definition, instructions, tool descriptions, business context, and examples. If you run 500 agent tasks per day and each one sends the same 4,000-token system prompt, you are paying to process 2 million tokens of system prompt per day. With prompt caching enabled, that 4,000-token prompt is stored after the first call and billed at up to 90% discount on subsequent calls — from the first token, not from some minimum threshold.
Anthropic, OpenAI, and Google all offer native prompt caching. For Anthropic's models, the cache prefix discount is 90% on cache reads. For Google's Gemini models, context caching applies to prefixes over 32,768 tokens and offers savings of up to 75%. The implementation change is minimal — typically a flag in your API call, not a redesign of your system — and the saving accumulates every time the cached prefix is reused.
For agents built on a RAG architecture, the same principle applies to retrieved context. Batching retrievals and caching common knowledge chunks — standard contract clauses, client profile templates, standard operating procedures — dramatically reduces the cost of grounding your agents in business knowledge.
Pillar 3: Output Compression
The third pillar is about the tokens your agent produces, not just the tokens it consumes. Agents built without cost discipline produce verbose outputs — long explanations, repeated context, unnecessary caveats — that are then passed as inputs to the next agent in the chain, compounding the cost downstream.
Compression means defining structured output schemas for every agent-to-agent handoff. Instead of an orchestrator agent receiving a 2,000-word narrative from a research agent, it receives a typed JSON object with exactly the fields the next step needs. Output schema enforcement — using Pydantic models in Python, Zod schemas in TypeScript, or the structured output modes offered by every major provider — cuts inter-agent token counts by 60–80% compared to unstructured prose. It also makes your observability layer significantly easier to build, because structured outputs are queryable and auditable in ways that prose is not.
Compression also applies to system prompts themselves. Most system prompts written during initial development contain redundancy — context that appeared necessary when the agent was being tuned but that the model does not actually need at inference time. A structured prompt audit typically cuts system prompt length by 30–40% without any change in output quality. Shorter prompts mean lower input costs on every call.
The Gateway Layer: Your Cost Control Centre
The three pillars are most effective when applied through a single gateway layer that sits between your agents and the model providers. Without a gateway, routing decisions, caching controls, and usage tracking are scattered across individual agent codebases — hard to change consistently, impossible to monitor centrally.
An LLM gateway — LiteLLM, OpenRouter, and Cloudflare AI Gateway are the leading options in 2026 — provides a unified API that your agents call, with routing rules, caching configuration, budget limits, and usage analytics managed in one place. You can change your routing strategy across your entire operating system by updating a configuration file rather than modifying every agent. You can see your spend by model, by agent, by task type, and by client — and set hard spending caps that halt requests before an unexpected usage spike turns into an unexpected bill.
For UK service businesses, Cloudflare AI Gateway has particular appeal because it runs within Cloudflare's UK data residency zones — relevant to businesses with data protection obligations under UK GDPR and the Data Protection Act. It also adds latency analytics, useful for identifying which model calls are slowing down your agent pipelines, since cost and latency are often correlated: the same frontier model that costs the most also responds slowest.
A gateway also handles model fallback. When a provider has an outage or rate limits your account, the gateway automatically routes to a configured fallback rather than returning an error to the agent. For production AI operating systems serving client-facing workflows, this is an availability feature as much as a cost one. The security controls that sit on your agent inputs can also be applied at the gateway layer, giving you a single enforcement point rather than duplicating security logic across individual agents.
What to Measure and What to Cut
Before you implement any of the three pillars, instrument your current system. You cannot optimise what you cannot measure — and the agents consuming the most tokens are rarely the ones you expect.
The four metrics that matter for cost optimisation are:
- Cost per task: Total spend divided by completed tasks, tracked per agent. This is your unit economics baseline. An email triage agent costing £0.40 per email and a proposal agent costing £1.20 per proposal give you very different optimisation priorities.
- Cache hit rate: The percentage of calls that hit a cached prefix. Below 60% suggests your system prompts are varying too much between calls — either adding unnecessary dynamic content or not being structured to maximise the stable prefix.
- Model tier distribution: What percentage of calls go to each tier, and whether that matches your routing intent. If 80% of calls are still hitting the frontier tier after implementing routing, your classifier is misfiring.
- Token efficiency: Output tokens per useful business outcome. This surfaces agents that are verbose without adding value — a common result of over-engineered chain-of-thought prompting that made sense during development but is not needed in production.
These metrics belong in your observability stack alongside error rates and latency. The post on AI agent observability covers how to set up the monitoring layer that makes all four trackable. Once you have baseline numbers, the optimisation sequence is: fix routing first (largest absolute saving), add caching second (fastest to implement), then address output compression (highest quality return, requires most careful testing).
What you should not cut: quality on client-facing outputs. The routing tiers above are calibrated to protect quality where it matters — frontier model for high-stakes outputs, flash model for classification and routing decisions. Cost optimisation that degrades the output a client sees is not optimisation. It is false economy.
Putting the Numbers Together
The impact of the full three-pillar framework in a deployed UK service business AI operating system is predictable enough to model before implementation. A typical five-agent operating system processing 300 tasks per day at an unoptimised running cost of £80/month will reach the following, conservatively, with all three pillars active:
- Routing: Moving 70% of tasks to flash-tier models reduces blended token cost by approximately 65%. That alone drops the token spend component from £80/month to £28/month.
- Caching: A 70% cache hit rate on 4,000-token system prompts, running 300 tasks per day, saves approximately 250 million input tokens per month. At flash-tier pricing with a 90% cache discount, that is around £12/month saved.
- Compression: Cutting inter-agent output verbosity by 60% reduces the input token count for downstream agents proportionally. On a system where agents chain frequently, this can reduce total token consumption by a further 20–30%.
Combined and applied across a real system, the result is typically a post-optimisation spend of £18–25/month — a reduction of 69–78%. The engineering work to implement all three pillars typically takes one to two days. The ROI calculation is not subtle.
The right AI cost target for a UK service business is not "as cheap as possible". It is "spend exactly what produces measurable business value, and not a token more". The three-pillar framework makes that precision achievable.
If you are running an AI operating system and want a cost review — or building one and want cost engineering built in from the start — talk to us. We run the instrumentation, identify the savings, and implement the framework as part of every build. Your agents should be saving you money, not costing it.