Most UK service businesses running AI agents make the same expensive mistake: they route every task through the same frontier model. The email triage, the data extraction, the complex reasoning, the simple classification — all of it hits Claude Opus or GPT-4 at the same price per token. AI agent model routing fixes that by matching each task to the cheapest model that can handle it reliably. Done properly, it cuts LLM costs by 40 to 70% without touching output quality. Here are the four patterns that work in production.
Why Every Agent Calling the Same Model Is a Production Problem
When you build your first AI agent, you pick the best model you can find and wire everything through it. That is the right call at the start — you need reliability while you are still learning what your agent needs to do well. The problem starts at scale.
In a production AI operating system, not all tasks are equal. An agent that processes 500 email enquiries a day, drafts client reports, classifies support requests, and pulls structured data from documents is running dozens of different task types. Some require genuine reasoning — connecting context from multiple sources, making a judgement call, writing something nuanced. Most do not. Most are pattern recognition, extraction, classification, and generation against a fixed template.
Frontier models charge frontier prices for every token regardless of whether the task requires their capability. A three-token classification — "is this a sales enquiry, a support request, or a complaint?" — costs the same per token as a 500-word proposal draft. At low volume, that does not matter. At the volume a production AI operating system runs, it adds up fast. The token budget management post covered how to cap and attribute costs — routing is the upstream complement: choosing the right model in the first place, not just controlling how many tokens it consumes.
In most production agent stacks: 70% of tasks are simple enough for a small fast model. 20% need a mid-tier model. 10% need a frontier model. Routing matches spend to actual requirement — and cuts the LLM bill by the difference.
The Four AI Agent Model Routing Patterns
These four patterns address different routing problems. Most production agent stacks use two or three of them in combination.
Pattern 1: Complexity-Tier Routing
Complexity-tier routing is the most common pattern. You define two or three tiers — typically a small fast model, a mid-tier model, and a frontier model — and route each task to the cheapest tier that meets a defined quality threshold for that task type.
The router is itself an AI component, usually a small fast model running a classification prompt: given a description of the task and the available tiers, which tier is appropriate? The router's inference cost is negligible compared to the savings it generates when it correctly routes a batch of simple tasks away from a frontier model. Build the router first against a labelled sample of tasks from your production logs, then validate it before deploying to live traffic.
In practice, the tier boundaries for a UK service business agent stack look like this: the small model (Claude Haiku, GPT-4o Mini, Gemini Flash) handles classification, extraction, template completion, and routing decisions. The mid-tier model handles moderate drafting, structured generation, and multi-step extraction. The frontier model handles complex reasoning, sensitive client communications, and anything requiring genuine contextual judgement.
Practitioner reports from production LLM workloads in 2026 put the cost reduction at 47 to 80% when complexity-tier routing is combined with caching. The caching patterns post covers the cache layer — routing and caching are the two patterns that do the most cost work in production, and they stack.
Pattern 2: Domain-Based Routing
Domain-based routing sends specialised queries to models fine-tuned or optimised for that domain. Rather than routing by complexity, you route by subject matter: legal document analysis goes to a model with legal fine-tuning, financial calculations to a numerically strong model, and general business drafting to a generalist frontier model.
This pattern is most relevant for businesses processing domain-specific content at high volume: law firms reviewing contracts, financial advisers processing client data, compliance teams analysing regulatory documents. The benefit is not just cost — domain-tuned models often outperform frontier models on their specific tasks, so routing by domain can improve quality while reducing cost simultaneously.
The agent evaluation framework post covers how to test whether a domain-specific model actually beats the frontier alternative on your specific tasks. Do not assume it will without benchmarking against your own data — the claims from model vendors about domain performance rarely match production results on your specific workload.
Pattern 3: Semantic Cache Routing
Semantic cache routing is not strictly model routing — it routes away from models entirely. When an incoming request is semantically similar to a previous request, the cached response is returned rather than re-running the LLM call. The cache is vector-based: the request is embedded and compared to a cache of past requests by cosine similarity. When the similarity exceeds a threshold, the cached response is returned.
This is the cheapest routing outcome possible — the model cost is zero. The accuracy depends on how tight you set the similarity threshold. Too loose and the cache returns responses to questions they did not quite answer. Too tight and the cache hit rate drops to irrelevance. In practice, most UK service business agent stacks find a threshold that returns useful hits on 20 to 40% of incoming requests — a meaningful cost saving on the most common, repetitive query types.
The semantic cache sits upstream of both the model router and the tier routing decision. The request path is: check cache → if hit, return cached response; if miss, route to the appropriate model tier. Combined with complexity-tier routing, this is typically where the 47 to 80% cost savings in practitioner reports come from. The underlying caching mechanics — including semantic similarity thresholds and cache invalidation — are covered in the caching patterns post.
Pattern 4: Failover Routing
Failover routing handles provider reliability rather than cost. When a primary model provider returns an error, rate limit, or timeout, the router automatically retries on a secondary provider. The failover target may use a different model — a comparable alternative from another provider — keeping the agent running when the primary is down.
Three of the five major model providers had multi-hour production incidents in a 90-day window earlier in 2026. For a UK service business running a production AI operating system — where agents are handling client-facing workflows — a provider incident should not take the whole system down. Failover routing ensures it does not.
The practical implementation is straightforward: define a provider priority list, set a timeout threshold for a primary provider response (typically 10 to 15 seconds for most agent calls), and route to the secondary on timeout or error. The fault tolerance patterns post covers the broader reliability architecture — failover routing is the specific slice that addresses provider-level failures.
Build Sequence: Where to Start
The right build sequence for a UK service business adding routing to an existing agent stack:
- Start with semantic cache routing. It is the simplest to add, requires no changes to your existing model calls, and shows immediate cost impact on high-volume repetitive queries. Implement with a vector database (Supabase pgvector works well in n8n) and a similarity threshold around 0.92 as a starting point. Adjust based on your hit rate and accuracy checks.
- Add complexity-tier routing second. Review a sample of 200 to 300 recent agent calls from your logs. Classify them manually into simple, moderate, and complex. Train a small router model — or use a few-shot prompt with Claude Haiku or GPT-4o Mini — to classify new tasks. Deploy with a human review pass on a subset of routed calls for the first two weeks before trusting it fully.
- Add failover routing third. This is the cheapest insurance you can buy for a production agent stack. Implement a provider timeout wrapper around your primary LLM calls, with the secondary provider as the fallback. The deployment patterns post covers the canary and blue-green approaches for deploying routing changes without disrupting live agent workflows.
- Consider domain routing last — only when you have high-volume, domain-specific tasks where specialised models demonstrably outperform your current stack on your own benchmarks.
Semantic cache first — costs nothing and saves immediately. Complexity-tier routing second — requires the most calibration but delivers the biggest ongoing saving. Failover third — protects what you have built. Domain routing last, only when the benchmark evidence justifies it.
What to Measure in Production
Routing adds complexity to your agent stack, which means it adds monitoring requirements. The metrics that matter in production:
- Cost per task type by model tier. If your simple-tier tasks have the same cost distribution as your complex-tier tasks, the router is misclassifying. The goal is a clear step-down in average cost between tiers.
- Cache hit rate by query category. A hit rate below 15% suggests the threshold is too tight or the query distribution is too varied for semantic caching to pay. A hit rate above 50% may indicate the threshold is too loose and you are serving stale responses.
- Routing accuracy. The percentage of routed tasks that produced outputs at or above quality threshold. Sample and human-review a weekly subset. If accuracy drops on the cheap tier, your complexity classifier needs recalibration.
- Failover frequency by provider. A high failover rate from one provider is a signal to review your primary provider choice or negotiate SLAs before it becomes a production reliability issue.
The AI agent observability post covers the broader observability layer for production agent stacks — routing metrics sit inside that framework. Do not add routing without also adding the measurement layer for it. A routing layer you cannot observe is a routing layer you cannot trust.
If you are building an AI operating system for your UK service business and want to understand how model routing fits into the overall architecture — including the agent design, tool connections, and production deployment — speak to the Quantum Flow team. We build, test, and deploy AI operating systems for UK service businesses: cost-optimised, production-tested, and built to compound.