Summary of findings
Model prices are falling quickly, and enterprise AI budgets keep climbing. Both are true at once. Published prices fall for a fixed level of capability, but actual spend depends on two other things: how much computation each task uses, and which capability tier it runs on. Both are design decisions rather than market conditions, which makes AI cost more controllable than the coverage of runaway spending suggests, provided the controls go in early.
Capability got cheaper while the top tier got more expensive.
Between November 2022 and October 2024, the price of a fixed level of model capability fell more than 280-fold, according to Stanford HAI's AI Index. Over the same period, each new model generation opened a pricing tier above the last. The most capable generally available models in July 2026 cost roughly $10 per million input tokens and $50 per million output tokens, about double the top tier's price a year earlier. An organization that settles on a level of capability sees its costs fall. One that always moves to the newest model sees them rise.
What drives cost is computation per task, not price per token.
Reasoning models produce more billable output for the same answer. Retrieval systems bill for retrieved context on every call. Agent workflows resubmit accumulated context at every step, so cost compounds as steps increase, and Gartner estimates agentic workloads use five to thirty times the tokens of a single-turn interaction. A fivefold fall in price against a sevenfold rise in tokens leaves a task costing 40 percent more than it did at the start. That is how unit prices fall while invoices rise.
Choosing the right tier saves more than any price change will.
Published pricing spans more than a hundredfold between the cheapest and most expensive tiers. At 10 million documents a year, the same extraction task costs about £2,630 at the budget tier and about £300,750 at the frontier (Appendix B), a factor of 114. No price change in the past three years comes close to that range, and the buyer decides which tier to use.
Self-hosting cost turns on staffing and on how busy the hardware is.
Dedicated inference carries two fixed annual costs: the hardware, and the engineers who run it. At a single accelerator unit, engineers are about 90 percent of the total. Because both costs are annual and throughput is not, the rate per token depends directly on utilization, and halving utilization doubles it. At 80 percent utilization, dedicated infrastructure beats a small purchased tier past about 530 billion tokens a year. Below about 37 percent it never beats it at any scale, because the hardware alone costs more than the tier does (Exhibit 5).
Fragmented systems make every AI initiative repeat the same cost.
Organizations built by acquisition, and those that have left technology decisions to individual divisions, usually run several operational systems with different data models and different definitions of core business terms. Every new AI initiative rebuilds the same integration and validation work rather than reusing it, which makes future cost hard to predict. A shared data foundation reverses this: the first initiative funds the setup, and every initiative after it costs less.
Inference is a small cost when governed and a large one when it is not.
Modelled across 24 months, inference is about 11 percent of annual program cost once routing, caching, and capacity planning are applied. Without those controls it rises to about a third of a total that is itself 2.8 times larger, and that comparison is conservative: it assumes the uncontrolled program settled on the high tier rather than the frontier (Section 6).
The fragmentation problem: many systems, no shared definitions
Most sectors have consolidated over the past decade, one acquisition at a time. Manufacturing, insurance, financial services, healthcare, distribution, and retail have all followed the same path, and the outcome looks the same in each: one group running a single strategy across several operational estates it did not design.
Acquisition is not the only route there. Organic growth across regions, divisional autonomy over technology, or a decade of departmental systems added to a core platform all end in the same place. What follows applies to any organization running more than one system of record for the same kind of business event.
Acquirers usually keep the operational systems of the businesses they buy, at least at first, and that is usually right. Those systems run day-to-day operations, and replacing them immediately after a deal adds risk the business does not need. Divisional systems survive for the same reason: they work, and there is rarely an urgent case for replacing them.
Keeping them still costs something, and the cost builds up. The problem is not the number of systems. It is that they disagree with each other in five specific ways.
- Event definitions differ. Systems disagree on when an order counts as complete, when a claim counts as settled, what tolerance decides on-time, and how cancelled transactions are treated. Group figures built from these sources have to be reconciled rather than added up.
- Master data differs. The same customer, supplier, product, or policyholder is identified differently in each system, so joining records becomes probabilistic matching instead of a lookup, and someone maintains that matching logic indefinitely.
- Units and hierarchies differ. Unit, packaging, and product hierarchy conventions are commonly inconsistent, which affects any measure expressed per unit of throughput, per case, or per account.
- Integration work does not transfer. Event models, status codes, and interfaces differ by source, so connecting one system teaches you almost nothing about connecting the next. The work repeats in full.
- Validation effort repeats. Each source behaves differently, so each new application needs its own evaluation before anyone can trust it.
This is why AI cost is hard to predict in organizations like this. A model works from the data it is given, so inconsistent sources produce inconsistent answers and the model cannot detect the problem. What people see is an AI-generated answer that disagrees with an established report. Someone spends time working out which one is right, and trust in the tool drops a little each time.
Tools that let business users query data directly are the most exposed. Turning a question into a query requires agreed definitions of the things being asked about. Where a term means different things in different systems, the answer comes back looking reasonable and wrong, which is far harder to catch than an obvious error.
Data Hatch illustration, drawn from observed delivery patterns rather than measured data. No figure elsewhere in this article comes from this exhibit. The unified path costs 35 percent more on the first use case, less from the second onward, and about a third as much by the tenth. On cumulative spend the two lines cross at the fourth use case, which is the number that matters to whoever funds the foundation.
Without a shared foundation, every initiative repeats the integration and validation work from the start, so cost per initiative stays flat or rises as more sources come into scope. With one, the first initiative pays the setup cost and each one after it builds on that work. Group reporting, network optimization, customer-facing data services, and any project spanning divisions all draw on the same foundation once it exists. The business pays that cost once instead of paying it with every project. Section 7 sets out how to build it.
How AI cost actually moves
Total AI spend is the product of three factors: how many tasks are performed, how much computation each task consumes, and what that computation costs per unit. Most attention goes to the third, the price per token. The first two cause nearly all of the variation organizations actually see, and both can be designed for.
Stanford HAI's AI Index tracks the cost of querying a model at a fixed benchmark score. That cost fell from $20 per million tokens in November 2022 to $0.07 by October 2024, a drop of more than 280-fold. Successive model generations delivered more capability at equal or lower prices, and open-weight providers pushed prices down further.
Two different prices move in two different directions. The price of a fixed level of capability falls. The price of the best available model rises, because each new generation adds a tier above the previous top rate. In July 2026 the most capable generally available models cost roughly $10 per million input tokens and $50 per million output tokens, up from $5 and $25 a year earlier. This article calls the $10 and $50 tier the frontier, and the $5 and $25 tier below it the high tier, which is what the frontier was a year ago. Output is priced at roughly five times input across providers, and tier-by-tier rates appear in Appendix B.
| Mechanism | Effect on expenditure | Direction | In your control |
|---|---|---|---|
| Published rate for a fixed capability | Price per million tokens at a stated performance level | Falling | No |
| Position of the frontier | Each generation adds a tier above the last, at about twice the prior top rate | Rising | No |
| Capability tier selected | Routine work at the frontier can cost over 100 times the same work at the smallest sufficient tier | Sharply up if unmanaged | Yes |
| Reasoning output | Models that generate intermediate reasoning produce far more billable output per answer | Up | Yes |
| Multi-step agent architecture | Accumulated context is resubmitted at every step, so cost grows faster than the number of steps | Steeply up | Yes |
| Adoption and volume growth | Extending a proven application across the business multiplies the number of calls made | Up, by design | Yes |
Data Hatch analysis. Tier multiples are derived from published provider pricing as of July 2026 (Appendix B). The five-to-thirty-times estimate for agentic workloads is Gartner analysis published March 2026 (Appendix D, reference 2).
Data Hatch illustration, indexed to 100 at the start of the period. The third series is the product of the first two: 0.20 × 7.00 = 1.40. The gradients show a mechanism rather than a forecast and represent no specific provider. The exhibit holds the capability tier constant, so it shows consumption growth on its own. Moving work to a more expensive tier is a separate effect, and a larger one.
On its own that is a modest result: a fivefold fall in price against a sevenfold rise in tokens leaves the cost 40 percent above where it started. Consumption growth alone does not produce a runaway bill. Tier does. Moving a task from the high tier to the frontier doubles its cost again, and moving it from the small tier to the frontier multiplies it by ten. That is why the rest of this article spends more time on routing than on consumption.
A commitment expressed as a rate protects against only one of the three factors. A commitment expressed per unit of business work, such as cost per document processed or per claim assessed, constrains what the organization actually pays and holds as volume grows. Sections 5 and 8 set out how to build one.
Self-hosted versus purchased inference
Self-hosting is often proposed as the way to make AI cost predictable. The argument is that infrastructure cost is fixed, known in advance, and unaffected by a provider changing its prices. It holds only at a certain scale and a certain level of utilization, and most operations reach neither.
Dedicated inference carries two costs. One is the hardware. The other is the engineers who run it, and at small scale they are much the larger of the two: at a single accelerator unit, engineering is about 90 percent of the total and hardware about 10 percent. Both are annual costs that do not change with how much work the deployment does.
That gives two rules. Scale helps, because more throughput spreads the same fixed cost further. This is why engineering falls from 90 percent of the total at one accelerator unit to 42 percent at thirty-two: the hardware bill grows with each unit added, while the headcount barely moves. Utilization helps just as directly, because the fixed cost is the same whether the hardware is busy or idle. Cost per million tokens is therefore inversely proportional to utilization: halve it, and the rate doubles, at any scale. Most business cases work through the scale question carefully and then assume a utilization figure the workload cannot sustain.
At 80 percent utilization, hardware alone costs about $0.92 per million tokens. Adding 1.5 to 4 engineers, depending on the size of the estate, gives the all-in figures below. Those engineers are additional to the platform team a purchased-inference program already needs, not a replacement for it (Appendix A).
| Accelerator units | Annual throughput at 80% | All-in cost per million tokens | Against small tier, $2.00 blended | Against high tier, $10.00 blended | Against frontier tier, $20.00 blended |
|---|---|---|---|---|---|
| 1 | 37.8 billion | $8.83 | 4.4× more expensive | 12% cheaper | 56% cheaper |
| 4 | 151 billion | $3.56 | 78% more expensive | 64% cheaper | 82% cheaper |
| 8 | 303 billion | $2.57 | 29% more expensive | 74% cheaper | 87% cheaper |
| 16 | 605 billion | $1.91 | 5% cheaper | 81% cheaper | 90% cheaper |
| 32 | 1,211 billion | $1.58 | 21% cheaper | 84% cheaper | 92% cheaper |
Data Hatch model. Effective cost at 80 percent sustained utilization. All-in cost includes infrastructure at published specialist-provider rates and engineering at UK fully loaded cost. Purchased tiers are blended rates at the article's standard task profile of 1,500 input and 500 output tokens. Full derivation and sensitivities at Appendix A.
Against a small purchased tier, dedicated infrastructure becomes cheaper only past about 530 billion tokens a year, a little over 14 accelerator units. Below that, the engineering overlay dominates and purchasing wins, in some cases by a factor of four.
Against the frontier tier the comparison is not close: one accelerator unit costs 56 percent less, and the gap widens with scale. That says more about frontier pricing at $20.00 blended than about self-hosting.
What happens when utilization is not 80 percent
Eighty percent is a favorable assumption. Batch processing can reach it, because work can be queued and run overnight. Interactive workloads cannot, because demand follows the working day and the hardware sits idle the rest of the time. Expect 20 to 40 percent for an interactive workload with no batch work filling the gaps.
| Accelerator units | Engineering assumed | 80% utilization | 50% utilization | 30% utilization | 20% utilization |
|---|---|---|---|---|---|
| 1 | 1.5 FTE | $8.83 | $14.13 | $23.55 | $35.32 |
| 4 | 2.0 FTE | $3.56 | $5.70 | $9.49 | $14.24 |
| 8 | 2.5 FTE | $2.57 | $4.11 | $6.86 | $10.28 |
| 16 | 3.0 FTE | $1.91 | $3.06 | $5.10 | $7.65 |
| 32 | 4.0 FTE | $1.58 | $2.53 | $4.22 | $6.33 |
| Annual throughput, 1 unit | 37.8bn | 23.7bn | 14.2bn | 9.5bn | |
| Annual throughput, 32 units | 1,211bn | 757bn | 454bn | 303bn |
All-in cost, USD per million tokens, including infrastructure and incremental engineering. Purchased reference rates, blended: small $2.00, mid $6.00, high $10.00, frontier $20.00. Bold figures undercut the mid tier. Data Hatch model; derivation and limitations at Appendix A. The engineering ladder is the same at every utilization level, because headcount depends on the size of the estate rather than on how busy the hardware is.
Reading down a column, scale improves the figures. Reading across a row, lower utilization worsens them in exact proportion: the 50 percent column is 1.6 times the 80 percent column, the 30 percent column 2.67 times, the 20 percent column 4 times. One accelerator unit is 12 percent cheaper than the high tier at 80 percent utilization and 2.4 times more expensive at 30 percent.
| Sustained utilization | Beats frontier, $20.00 | Beats high, $10.00 | Beats mid, $6.00 | Beats small, $2.00 |
|---|---|---|---|---|
| 80% | 1 unit | 1 unit | 2 units | a little over 14 units, about 530bn tokens a year |
| 50% | 1 unit | 2 units | 3 units | about 65 units, about 1.5tn tokens a year, and only if headcount stops growing at 4 FTE |
| 30% | 2 units | 3 to 4 units | 8 to 16 units | unreachable at any scale |
| 20% | 2 units | 8 to 16 units | beyond 32 units | unreachable at any scale |
Data Hatch model, same inputs as Exhibit 5. The last column is explained below.
Below about 37 percent utilization, no amount of scale helps
At 30 percent utilization the hardware alone costs $2.47 per million tokens, and at 20 percent it costs $3.70. Both are above the $2.00 small-tier rate before a single engineer is paid, so adding hardware cannot close the gap. Below that point, dedicated inference cannot beat a small purchased tier at any scale.
The same calculation gives a floor for every tier. Hardware-only cost is $0.74 per million tokens divided by utilization.
| Purchased tier, blended | Utilization at which hardware alone equals the tier rate | Below that point |
|---|---|---|
| Small, $2.00 | 37% | No scale brings dedicated inference under the small tier |
| Mid, $6.00 | 12% | No scale brings it under the mid tier |
| High, $10.00 | 7% | Effectively always beatable |
| Frontier, $20.00 | 4% | Effectively always beatable |
Three conclusions follow.
First, a self-hosting case built on 80 percent utilization should name the workload that will deliver it. That means batch work large enough to fill the gaps around interactive demand, which is an operational commitment rather than a planning assumption. Test the plan against 37 percent: below it, no amount of scale makes the case.
Second, mixing workloads beats adding hardware. Putting batch work on the same infrastructure that serves interactive queries raises utilization directly. Buying more accelerators for the same demand lowers it.
Third, beating the frontier tier proves little, since dedicated infrastructure comes in under $20.00 blended at one or two units in every case modelled here. The comparison that decides the question is against the smallest tier that can do the work.
Data Hatch model based on published provider pricing as of July 2026, converted at $1.33 to £1. Per-document costs and full arithmetic at Appendix B.
A structured extraction using about 1,500 input and 500 output tokens shows the same pattern per document. At 10 million documents a year the gap between cheapest and most expensive tier is about 114-fold, close to £300,000 for identical work.
That volume is about 20 billion tokens a year: roughly twenty-six times below the 530 billion where dedicated infrastructure starts to win, and about half the annual capacity of one accelerator unit at high utilization. Purchasing costs about £30,100 a year at that volume, against £225,000 or more for the extra engineering self-hosting would need. The saving comes from routing work to the right tier, not from hosting it.
Two conditions still justify self-hosting below the break-even point. Dedicated infrastructure satisfies data residency requirements by design, and it removes exposure to a provider's future price changes entirely. Against a frontier-tier alternative, both come free. Against a small purchased tier, both come at a real premium, and Exhibit 5 gives its size at whatever utilization the deployment can sustain.
Four controls that hold a cost envelope
Four controls account for most of the cost reduction that is actually achievable, and they follow the practices the FinOps Foundation has set out for AI cost governance. Published research quantifies each one, but the figures are specific to the benchmark tested, so the saving on a real workload has to be measured rather than assumed.
Route work to the smallest capability tier that can do the job.
Most enterprise workloads do not need the most capable model available. RouteLLM, from LMSYS and UC Berkeley, cut cost by more than 85 percent on one benchmark while keeping about 95 percent of the reference model's performance, but by only 45 percent on a second and 35 percent on a third. FrugalGPT, from Stanford, reports up to 98 percent by passing requests through progressively more capable models and stopping once an answer is good enough, though on a single dataset of financial news headlines. Neither figure works as a planning number on its own. Plan against the whole 35 to 85 percent range, and expect the mix of easy and hard requests to decide where a given workload lands in it. Build the control into the platform as policy: check each request against a quality threshold, and escalate only when it falls short.
Cache repeated context, and shift deferrable work to batch processing.
Context that repeats across calls is billed every time unless it is cached, and providers discount cached reads by about 90 percent. The discount applies to reads only. Writes carry a premium and entries expire, so the blended saving depends on the ratio between them: workloads with stable instructions and frequent calls approach the headline figure, and workloads with varied context see little benefit. Separately, work that does not need an immediate response can move to batch execution at about half rate, with no similar caveat. Document extraction, classification, compliance review, case triage, and reconciliation can all be scheduled this way.
Purchase committed capacity sized to measured base load.
Reserved capacity turns variable billing into a fixed periodic charge at a substantial discount. The risk is over-commitment, since reserved capacity that goes unused is spend with no return. Size the reservation to measured base load and leave peak demand on consumption pricing.
Negotiate contractual protection proportionate to the size of the commitment.
At enterprise scale, several protections are usually available: rate protection for the term, so published increases do not apply to committed volume; advance notice of model withdrawal, so migration can be planned rather than forced; protection against changes in how usage is measured, since a metering change has the same effect as a price rise; data and workload portability, to preserve negotiating position at renewal; and periodic benchmarking against market rates.
| Control | Mechanism | Published effect, and its qualification |
|---|---|---|
| Capability tier routing | Policy-based selection with escalation by exception | 35 to 85 percent across the three benchmarks in the routing study; up to 98 percent on a single dataset for cascading. Workload-specific; requires measurement. |
| Context caching | Retention and reuse of repeated prompt context between calls | About 90 percent on cache reads. Blended saving is lower, because writes carry a premium and entries expire. |
| Batch execution | Deferred processing for work not requiring an immediate response | About 50 percent against standard rates, applied consistently by major providers. |
| Committed capacity | Reserved or provisioned throughput sized to measured base load | Substantial discount against on-demand pricing. Realized benefit depends on the accuracy of the base-load measurement. |
Routing figures from Appendix D, reference 3; cascading figure from reference 4. Caching, batch, and committed-capacity terms as published by major providers as of July 2026, and should be confirmed at contracting.
All four are far cheaper to design in than to add later, since retrofitting means reworking prompts, pipelines, and evaluation systems built without them in mind. Together they support a commitment expressed as a maximum cost per unit of business work, held across an agreed volume range and reviewed on a set schedule. A rate-only commitment does neither.
A twenty-four-month cost model
This model applies the mechanisms above to a representative program: 10 million processed documents and 2 million interactive queries a year by month 24. The three scenarios reach the same adoption and deliver the same business outcomes, and differ only in the controls applied. The difference between them is built into the model by construction, so it shows the potential size of the effect rather than proving it happened in practice.
Identical outcomes do not mean identical effort, and the model separates the two. The uncontrolled case carries higher observability and support costs because inconsistent outputs have to be reconciled before anyone can use them, not because the business accepts less. Routing has a trade-off in the other direction: the study behind control 1 kept about 95 percent of reference performance, so a workload that needs the remaining margin should set a higher threshold and plan for a smaller saving.
| Component | Optimized | Routed | Without controls |
|---|---|---|---|
| Document workload inference | £12,000 | £30,000 | £150,000 |
| Interactive workload inference | £14,000 | £26,000 | £301,000 |
| Data platform capacity and storage | £58,000 | £73,000 | £219,000 |
| Observability, evaluation, and compliance | £30,000 | £35,000 | £70,000 |
| Platform operation and support | £225,000 | £300,000 | £525,000 |
| Residual integration effort | £15,000 | £25,000 | £120,000 |
| Total annualised run rate | £354,000 | £489,000 | £1,385,000 |
| Inference as a share of total | 7% | 11% | 33% |
Data Hatch model. Composition of each case and derivation of every line at Appendix C. Modelled at an illustrative scale to establish relative magnitude and composition, not to forecast expenditure for any specific organization.
Inference is a minor cost once controls are applied.
In the routed case the two inference lines come to £56,000 between them, about 11 percent of the total: less than a fifth of platform operation, and less than data platform capacity on its own. Taken separately, as they appear in the budget, neither is among the three largest lines. An organization that focuses only on token pricing is optimizing the fourth-largest line in its own budget. Remove the controls and inference becomes the largest line after staffing.
Engineering capacity is the largest line in every scenario.
One additional engineer adds about £150,000 a year, roughly five times the entire document-inference line in the routed case. An approach needing fewer people to run beats one with lower licensing or inference costs and higher staffing. That favors reusing a shared foundation over repeating point-to-point integration work. It also favors purchased inference at these volumes: self-hosting the document workload would add at least 1.5 engineers, about £225,000, to replace an inference line of £30,075.
The model is more sensitive to staffing than to any pricing assumption.
Removing an accelerator unit or changing the document-processing tier moves the total far less than adding one person does. Look at the operating model before the rate card.
The 2.8× gap is a conservative figure.
The uncontrolled case assumes the high tier, one step below the frontier. An organization defaulting to the frontier itself would roughly double both inference lines, taking the total to about £1,836,000 and the ratio to about 3.8 times.
Fixing the root cause: a common data foundation
Section 2 described how fragmentation compounds: every new AI initiative pays the same integration cost again when systems and definitions are not shared. Four approaches address this, and they differ sharply in cost, timeline, and disruption.
Operational consolidation
Migrating divisions and acquired businesses onto a single operational platform gives the most complete result, but it is a multi-year program in its own right and means running new and legacy systems in parallel for a year or more before any benefit appears.
Coexistence through middleware
Point-to-point interfaces between existing systems keep disturbance low and solve transactional problems well: a record raised in one system appears correctly in another. They do not solve the analytical problem, because the same data still means different things in each system, so reporting still needs reconciliation.
Data foundation unification
A governed set of shared definitions over data that stays in its source systems is the fastest route to a consistent group-wide view, usually delivering value in three to nine months. It does not solve operational duplication, and it does not replace consolidation where consolidation is justified on its own terms.
Selective consolidation
Consolidate only where the business case stands on its own, and unify everything else at the definition layer. Once shared definitions exist in one place, the cost of maintaining any remaining divergence becomes visible, which makes each future consolidation decision measurable rather than assumed.
For most multi-system groups, data foundation unification is the practical starting point. It provides what reliable group reporting and predictable AI economics both depend on, without disturbing the systems the business uses daily, and it can run alongside a longer-term consolidation decision instead of waiting for one.
Platform selection, and the build-versus-buy question
Once the layer is settled, platform selection is straightforward. A well-specified definition layer runs equally well on a major vendor's data platform, an open lakehouse, or a managed analytical service. A poorly specified one is expensive on all three, because the cost sits in agreeing what the definitions are, not in the technology that enforces them.
AI-assisted development has narrowed the cost gap between building this layer and buying it. That gap alone should not decide the question. Section 6 showed that engineering capacity, not licensing or inference cost, is the largest line in every scenario. The questions that matter are which option needs fewer people at steady state, and how far each locks in pricing that cannot be renegotiated later. Apply that test to a custom build as rigorously as to a vendor platform.
Putting this into practice
Everything above is a model. Turning it into a commitment for a specific organization needs three things the model cannot supply: measurements from the actual operation, a way of stating the commitment that holds as volume grows, and someone accountable for it.
Measure your own baseline before fixing a cost ceiling
The figures here show the relative size and behavior of each cost component. They do not forecast any specific organization's spend. A defensible commitment depends on measured volumes, the token profile of real documents, and the routing distribution achievable on real workloads, none of which can be known from outside the operation. A number quoted before that work is done is either padded to cover the uncertainty or wrong. A padded number loses the work or pays for capacity nobody needs. A wrong one gets renegotiated after delivery has started and the alternatives have gone.
Start with a short, bounded baseline stage
Four to six weeks spent inventorying source systems, measuring volumes, and costing the processes being replaced produces the inputs a real commitment depends on. Keep it short enough, and priced separately enough, to require no commitment to the wider program. The output is a set of numbers that replace the assumptions in this article with measurements.
| Measurement | Method | What it determines |
|---|---|---|
| Source systems and definition conflicts | Schema and terminology review, limited to the terms the first workload uses | Scope of the definition layer, and the residual integration line |
| Document and transaction volumes | Counts from operational systems across a full seasonal cycle where the data allows | The volume base under every cost line in the model |
| Token profile of real documents | Tokenize a representative sample by document type and length | Cost per document at each tier, and whether the 1,500 input and 500 output assumption holds |
| Achievable routing distribution | Run a sample through a small tier against a quality threshold, and measure the escalation rate | Where inside the 35 to 85 percent range your workload falls |
| Cache-eligible share of context | Measure the repeated prefix across real calls, and the ratio of reads to writes | The blended caching saving, always lower than the headline read discount |
| Deferrable share of the workload | Classify by response-time requirement with the business owner, not the platform team | The share eligible for batch execution, and the utilization any self-hosting case could sustain |
| Base load and peak ratio | Instrument request rates across the day, the week, and month-end | Committed capacity sizing, and the platform capacity line |
| Agent step counts and token multiplier | Instrument a pilot workflow end to end, counting resubmitted context | Exposure to the five-to-thirty-times multiplier, and where step budgets sit |
| Current cost and effort of the process being replaced | Time and cost of the process as it runs today, including rework | The benefit side of the case, and the ceiling worth paying |
Data Hatch. Four to six weeks covers all nine if source access is arranged in advance. Access is the usual constraint, not analysis time.
Express the commitment as cost per unit of business work, inside a volume band
State the commitment in the unit the business recognizes: cost per document processed, per claim assessed, per case closed, per query answered. It should specify five things.
- The unit, defined precisely enough that both parties count it the same way, including partial, cancelled, and reprocessed items.
- The volume band over which the unit cost holds, and what happens above and below it. Unit cost falls with volume through caching and committed capacity, so the band should reflect that.
- The review cadence, usually quarterly, and what evidence is presented at each review.
- What is excluded: changes in scope, new document types, and material changes to the quality threshold, each of which changes the routing distribution and the cost.
- What triggers renegotiation either way, including sustained volume outside the band and material movement in published rates.
Instrument before you govern
None of the four controls in Section 5 can be managed from an invoice. A monthly total tells you spend went up, not whether that is adoption working or a routing policy that has stopped working. Attribute cost per request from the first production workload: the tier that served it, input tokens split by cached and uncached, output tokens, agent steps, whether it escalated, and the unit of business work it belonged to. Two numbers fall out of those fields, cost per completed unit and the escalation rate, and every other governance question depends on them.
Sequence the foundation ahead of or alongside the first workload
The first definition domain, covering the terms used most widely across the business, takes eight to twelve weeks and is what makes group figures reconcile without manual adjustment. The first production workload should be the highest-volume document or transaction process, built with routing, caching, and batch execution from day one. The two run in parallel rather than in sequence.
| Period | Focus | Outcome |
|---|---|---|
| Weeks 1–6 | Baseline stage, priced separately, carrying no commitment to the wider program | Measured inputs for a defensible unit cost |
| Weeks 4–16 | First definition domain: the terms used most widely across the business | Group figures reconcile without manual adjustment |
| Weeks 8–20 | First production workload, with routing, caching, and batch execution built in | Measured cost per unit at production volume |
| Months 5–8 | Second and third workloads on the same foundation | Evidence of the reuse crossover in Exhibit 1, from your own data |
| Months 6–9 | Commercial stage: commit capacity to measured base load, negotiate the protections in control 4 | A unit-cost commitment across an agreed volume band |
| Months 9–12 | Extend adoption; review unit cost monthly and the rate card quarterly | The envelope holds while volume grows |
Data Hatch. The definition domain and the first workload overlap because each improves the other: the workload shows which definitions matter, and the definitions make its output reconcilable.
Give the envelope an owner, and review unit cost rather than total spend
Total AI spend should rise if adoption is working, and treating that rise as an overrun puts pressure on the work creating the value. The number to govern is cost per unit of business work, which should fall as caching matures, routing improves, and committed capacity is sized against real base load. Make one person accountable for it, with engineering, finance, and the business owner in the same review rather than negotiating through it.
Watch for the failure modes that recur
- The pilot that never gets re-routed. Pilots are built on the frontier tier because it removes a variable, then move into production unchanged. This is the most expensive default in the field: up to a hundredfold against the smallest sufficient tier, and a factor of ten against a small tier that would have done the job.
- Prompts written without a cache-stable prefix. Caching depends on repeated content sitting at the front of the request. Prompts assembled in a different order each time cannot be cached, and reordering them after evaluation is built is expensive.
- Agents without step and budget limits. Because context is resubmitted at each step, cost grows faster than the step count. No ceiling on steps means no ceiling on cost.
- Reserved capacity sized to peak. Committed capacity is discounted because it is committed. Sizing it to peak turns the discount into permanent overpayment on idle capacity.
- A self-hosting case decided on hardware cost alone. Hardware is about 10 percent of all-in cost at a single accelerator unit. A case that cannot show sustained utilization above about 37 percent is not competing with a small purchased tier at any scale.
- A cost ceiling set from a vendor's figures. Published savings belong to the benchmark they came from. Yours are not knowable until measured.
Cost predictability is not a separate project from the AI program. It is what you get when the four controls in Section 5 and the shared foundation in Section 7 are built in from the start rather than added as a cleanup exercise later.
Frequently asked questions
Why do AI bills rise while model prices fall?
Published prices fall for a fixed level of capability, but actual spend depends on how much computation each task uses and which capability tier it runs on. Reasoning models, retrieval, and agent workflows multiply the tokens billed per task, and each new model generation opens a pricing tier above the last. A fivefold fall in price against a sevenfold rise in tokens leaves a task costing 40 percent more than it did at the start.
How much does the choice of model tier affect AI cost?
More than any price change will. Published pricing spans more than a hundredfold between the cheapest and most expensive tiers. At 10 million documents a year, the same extraction task costs about £2,630 at the budget tier and about £300,750 at the frontier, a factor of 114, and the buyer decides which tier to use.
When does self-hosting AI inference cost less than buying tokens?
Only at high sustained utilization and very large volume. At 80 percent utilization, dedicated infrastructure beats a small purchased tier past about 530 billion tokens a year. Below about 37 percent utilization it never wins at any scale, because the fixed hardware and engineering costs exceed what the purchased tier would charge for the same throughput.
What share of an AI program's cost is inference?
Modelled across 24 months, inference is about 11 percent of annual program cost once routing, caching, and capacity planning are applied. Without those controls it rises to about a third of a total that is itself 2.8 times larger.
How do you make AI spend predictable?
Apply four controls from the start: route each request to the smallest sufficient model, cache repeated context, plan capacity against measured base load, and negotiate rate protections. Build workloads on a shared data foundation so integration work is reused rather than repeated, and govern cost per unit of business work inside a volume band rather than total spend.
Appendices
ABreak-even model for dedicated inferenceOpen +Close −
This model computes an effective cost per million tokens for inference on dedicated infrastructure, including engineering capacity, and compares it against published per-token pricing on the same blended basis.
Infrastructure cost per million tokens equals the accelerator's hourly rate, divided by throughput multiplied by 3,600 and by utilization, multiplied by 1,000,000.
All-in cost per million tokens equals annual infrastructure cost plus annual engineering cost, divided by annual throughput in millions of tokens.
Inputs
| Input | Value applied | Basis |
|---|---|---|
| Accelerator hourly rate | $3.99 | Published list pricing from specialist infrastructure providers, July 2026 (Appendix D, reference 7). Rates in this class span roughly $1.50 to $6.00 by generation and provider, and major cloud providers price equivalent capacity higher. |
| Sustained aggregate throughput | 1,500 tokens per second | Conservative planning figure for a mid-size open-weight model on a single accelerator, from published open-serving benchmarks (Appendix D, reference 6). A frontier-scale open-weight model needs several accelerators, multiplying infrastructure cost accordingly. |
| Hours per year | 8,760 | Continuous availability. |
| Engineering capacity | 1.5 to 4 full-time employees | £150,000 each at UK fully loaded cost. Scales with the size of the estate rather than with volume. |
What the engineering figure covers
This assumption affects the comparison more than any other. The 1.5 to 4 full-time employees are the incremental capacity needed to run models on dedicated infrastructure: capacity planning, provisioning and failover, model upgrades and re-serving, security maintenance of the serving stack, evaluation of self-served models, and incident response for the serving layer.
They sit on top of the platform operation and support line in the Section 6 model, which covers work required whether inference is purchased or self-hosted: pipelines, orchestration, prompt and evaluation maintenance, access control, and support. Nothing is counted twice. An organization in the routed case carries 2 full-time employees for platform operation; the same organization self-hosting its document workload on one accelerator unit would carry 2 plus 1.5, not 2 in total.
The comparison in Exhibits 4 and 5 is therefore like for like. Purchased per-token pricing carries no engineering, and the dedicated figure carries only what self-hosting adds. Both sit on the same platform-operation base.
| Accelerator units | Incremental engineering FTE | Annual engineering cost | Annual infrastructure cost | Total annual fixed cost |
|---|---|---|---|---|
| 1 | 1.5 | $299,250 | $34,952 | $334,202 |
| 4 | 2.0 | $399,000 | $139,810 | $538,810 |
| 8 | 2.5 | $498,750 | $279,619 | $778,369 |
| 16 | 3.0 | $598,500 | $559,238 | $1,157,738 |
| 32 | 4.0 | $798,000 | $1,118,477 | $1,916,477 |
Engineering converted at $1.33 to £1. Annual infrastructure cost per accelerator is $3.99 × 8,760 = $34,952, and does not vary with utilization.
Known limitations
- Throughput is applied uniformly to input and output tokens. Processing input is faster than generating output, so applying a generation-speed figure to input overstates compute time, and therefore cost, for dedicated infrastructure. This makes the model conservative against self-hosting.
- Several infrastructure costs are left out: networking, storage, redundancy and failover, idle capacity held for peak demand, model licensing, and infrastructure for fine-tuning, distillation, and evaluation. Including them would raise the dedicated-hosting figures, working against the point above.
- Utilization of 80 percent is favorable, achievable for continuous batch processing but not for interactive workloads. Exhibit 5 gives the position at lower levels rather than treating 80 percent as the only case.
- The engineering ladder is a planning assumption rather than a measured relationship. An organization already running comparable infrastructure may carry the function at lower marginal headcount; one starting from nothing will carry it higher for the first year. Where the ladder is extended beyond 32 units, the assumption used is stated at that point, because the answer depends on it.
Computed results
Infrastructure alone. About $0.92 per million tokens at 80 percent utilization, $1.48 at 50 percent, $2.47 at 30 percent, and $3.70 at 20 percent. The general form is $0.74 divided by utilization.
All-in cost and utilization. Both components are fixed annual charges while throughput varies with utilization, so the all-in figure is inversely proportional to utilization. The 50 percent column of Exhibit 5 is 1.6 times the 80 percent column, the 30 percent column 2.67 times, and the 20 percent column 4 times.
Composition. At one accelerator unit and 80 percent utilization, all-in cost is about $8.83 per million tokens: infrastructure about 10 percent, incremental engineering about 90 percent. At 32 units it falls to about $1.58, of which infrastructure is about 58 percent and engineering 42 percent.
Break-even against the small tier at $2.00 blended. At 80 percent utilization the crossover falls a little above 14 accelerator units: 14 units gives $2.006 and 15 gives $1.956, so the line is crossed at about 530 billion tokens a year. The first point in the modelled table below the line is 16 units at $1.91.
At 50 percent utilization the crossover moves to about 65 units, roughly 1.5 trillion tokens a year, and only if engineering headcount stops growing at 4 full-time employees. If the ladder keeps rising at the rate applied between 8 and 32 units, all-in cost at 50 percent utilization approaches $2.005 and never crosses $2.00 at any scale. The break-even table in Section 4 uses the flat-headcount assumption, the more favorable of the two to self-hosting.
The utilization floor. At 30 percent utilization and below, infrastructure alone costs more than the small tier's blended rate, so no quantity of hardware closes the gap. Setting bare infrastructure cost equal to each tier rate gives the utilization below which self-hosting cannot win: 37 percent against small at $2.00, 12 percent against mid at $6.00, 7 percent against high at $10.00, and 4 percent against frontier at $20.00. This result does not depend on the engineering assumption at all.
BMethod for estimating document workload costOpen +Close −
The figure of 10 million documents a year used throughout is illustrative, representing the order of magnitude of document flow in a large, transaction-heavy operation, whether the records are shipments, claims, orders, cases, or accounts. Substituting measured volumes reproduces the calculation for any organization.
Per-document cost at each capability tier
| Capability tier | Indicative published rate per million tokens | Cost per document | At 10 million documents |
|---|---|---|---|
| Budget | $0.10 input, $0.40 output | $0.00035 | £2,632 |
| Small | $1.00 input, $5.00 output | $0.00400 | £30,075 |
| Mid | $3.00 input, $15.00 output | $0.01200 | £90,226 |
| High | $5.00 input, $25.00 output | $0.02000 | £150,376 |
| Frontier | $10.00 input, $50.00 output | $0.04000 | £300,752 |
Assumes 1,500 input and 500 output tokens per document, representative of a structured extraction on a typical business document, converted at $1.33 to £1. Rates are indicative of published tier pricing as of July 2026 and vary by provider (Appendix D, reference 8). Tier labels describe position in the market at a given moment, not vintage: the high tier here was the frontier a year earlier, and on the pattern in Exhibit 2 today's frontier will sit there a year from now.
Worked calculation
At the small tier, $1.00 per million on 1,500 input tokens gives $0.0015, and $5.00 per million on 500 output tokens gives $0.0025, for $0.0040 per document, or $40,000 at 10 million documents, about £30,075. The same calculation at the frontier gives $0.0400 per document and $400,000 annually, about £300,752, a factor of about 114 between the two ends.
Caching the input portion, where the extraction instruction is common across documents, and running the work in batch reduce these figures further. Both apply at every tier, so neither changes the relationship between them.
Blended rates used in the hosting comparison
A self-hosted deployment is billed for total tokens rather than separately for input and output. At the 1,500-and-500 profile, the blended rate is $0.175 per million tokens at budget, $2.00 at small, $6.00 at mid, $10.00 at high, and $20.00 at frontier. These are what Exhibits 4 and 5 compare against, and they are not the same figures as the input rates above.
Volume estimation where measured counts are unavailable
Volume can be estimated from operational throughput. A transaction-heavy operation generates several records per operational unit: an instruction, a confirmation, an evidence or completion record, and a financial document, plus compliance records on a portion of activity. The pattern holds across sectors even though the artifacts differ.
| Sector | Operational unit | Documents typically generated | Indicative multiplier |
|---|---|---|---|
| Distribution and logistics | Shipment | Dispatch instruction, delivery note, proof of delivery, invoice, customs documentation on cross-border movements | 4 to 6 |
| Manufacturing | Production or sales order | Purchase order, works order, certificate of conformity, despatch note, invoice | 4 to 6 |
| Insurance | Claim | First notification, supporting evidence, assessment record, correspondence, settlement statement | 5 to 8 |
| Financial services | Account or facility onboarding | Application, identity evidence, affordability and source-of-funds documents, agreement, periodic review record | 4 to 6 |
| Healthcare | Episode of care | Referral, consent, clinical notes, diagnostic results, discharge summary and coding record | 5 to 8 |
| Professional and business services | Engagement | Proposal, contract, milestone or timesheet record, deliverable, invoice | 3 to 5 |
| Retail and consumer | Supplier order or return | Purchase order, goods received note, invoice, credit note, returns record | 3 to 5 |
| Public sector | Case | Application, supporting evidence, assessment, decision notice, appeal record on a portion of cases | 4 to 6 |
The multiplier spans three to eight, with four to six a reasonable default until measured counts are available. On a multiplier of five, an operation handling around 2 million transactions, claims, or cases a year reaches the 10 million document volume used here. The estimate is sensitive to how the operational unit is defined, which is the first thing to pin down and the most common source of a materially wrong answer.
CComposition of the scenario casesOpen +Close −
Three cases model an identical program reaching identical adoption at month 24, differing only in the controls applied, processing 10 million documents and 2 million interactive queries annually at that point. The interactive workload is modelled at 8 billion input and 1.6 billion output tokens a year before any agent multiplier, about 4,000 input and 800 output tokens per query.
Governance variables applied
| Variable | Optimized | Routed | Without controls |
|---|---|---|---|
| Tier used for document work | Small, distilled | Small | High |
| Tier used for interactive work | Small and mid | Mid | High |
| Context caching | Applied by default | Applied to major workloads | Not implemented |
| Batch execution | Consolidated windows | Applied where available | Not applied |
| Agent step and budget limits | Enforced with budgets | Enforced | Not enforced |
| Platform capacity sizing | Right-sized to base load | Reserved to base load | Provisioned to peak |
The uncontrolled case is modelled at the high tier rather than the frontier; the sensitivity is given at the end of this appendix.
Derivation of each line
| Line | Optimized | Routed | Without controls |
|---|---|---|---|
| Document workload inference | Batch execution takes the small-tier figure from £30,075 to £15,038 at about half rate; a distilled model tuned for the extraction shortens output and removes about a fifth again, to £12,030: £12,000 | Small tier at 10 million documents (Appendix B): £30,075 | High tier: £150,376 |
| Interactive workload inference | Tokens split 55/45 between small and mid tiers at an 85% cache hit. Small: 3.74bn cached input at $0.10 plus 0.66bn uncached at $1.00 plus 0.88bn output at $5.00 equals $5,434. Mid: 3.06bn cached input at $0.30 plus 0.54bn uncached at $3.00 plus 0.72bn output at $15.00 equals $13,338. Total $18,772: £14,000 | Mid tier, 60% cache hit. 4.8bn cached input at $0.30 plus 3.2bn uncached at $3.00 equals $11,040; 1.6bn output at $15.00 equals $24,000; total $35,040: £26,000 | High tier, no caching, 5× token multiplier for agents without step limits (bottom of the range in reference 2). 40bn input at $5.00 plus 8bn output at $25.00 equals $400,000: £301,000 |
| Data platform capacity and storage | £35,000 production reserved against measured base load, £12,000 non-production, £11,000 storage (about 40TB): £58,000 | £45,000 production under annual reservation, £15,000 non-production, £13,000 storage (about 50TB): £73,000 | Provisioned to observed peak at about 3× base load: £219,000 |
| Observability, evaluation, and compliance | Same components as the routed case with fewer distinct models to evaluate: £30,000 | Monitoring, evaluation harnesses, guardrails, logging, and record-keeping: £35,000 | £70,000, reflecting reconciliation of inconsistent outputs |
| Platform operation and support | 1.5 FTE: £225,000 | 2 FTE at £150,000 fully loaded: £300,000 | 3.5 FTE: £525,000, reflecting incident load and reconciliation effort |
| Residual integration effort | Never zero: source systems change, interfaces are revised, new sources enter scope: £15,000 | £25,000 | No shared layer, so integration is re-incurred across sources: £120,000 |
Capacity pricing varies widely between platforms, so the platform capacity line is the most sensitive to platform choice and should be confirmed against the actual vendor rate card. The platform operation line excludes any incremental engineering for dedicated inference, because all three cases purchase inference; Appendix A sets out what self-hosting would add.
Totals and sensitivity
The three cases total about £354,000, £489,000, and £1,385,000, a ratio of about 2.8 times between the routed case and the case without controls.
Engineering headcount is the most influential assumption, since one full-time employee moves any case by £150,000. Platform capacity is second, varying threefold across the cases and depending on a rate card that should be confirmed before use.
Tier selection in the uncontrolled case is third. Modelled at the frontier rather than the high tier, document inference becomes £300,752 and interactive inference £601,504, taking that case to about £1,836,000 and the ratio against the routed case to about 3.8 times.
DReferencesOpen +Close −
Listed in the order first relied upon. Pricing and infrastructure references are point-in-time, established July 2026.
- Stanford Institute for Human-Centered AI, AI Index Report 2025, research and development chapter, for the recorded fall in inference cost at fixed benchmark performance between November 2022 and October 2024.
- Gartner, analysis published March 2026, estimating that agentic workloads consume 5 to 30 times the tokens of an equivalent single-turn interaction. Widely reported; the primary Gartner publication should be obtained before the figure is relied upon contractually.
- Ong, I., Almahairi, A., Wu, V., Chiang, W., Wu, T., Gonzalez, J. E., Kadous, M. W., and Stoica, I., RouteLLM: Learning to Route LLMs with Preference Data, arXiv preprint 2406.18665, June 2024. Reported reductions differ substantially across the three benchmarks evaluated, spanning 35 to above 85 percent.
- Chen, L., Zaharia, M., and Zou, J., FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance, arXiv preprint 2305.05176, May 2023. The headline reduction is reported on a financial news headline dataset.
- FinOps Foundation, FinOps Framework, including the FinOps for AI technology category and the FinOps Open Cost and Usage Specification.
- vLLM project, published performance benchmarks for open-weight model serving, used in Appendix A as the basis for sustained throughput per accelerator.
- Published list pricing for accelerated compute from specialist cloud infrastructure providers, retrieved July 2026.
- Published application programming interface pricing from major model providers, including tier rates, cached input discounts, and batch execution discounts, retrieved July 2026.
References 7 and 8 are point-in-time observations. Figures derived from them establish relative magnitude, between tiers and between hosting options, which has been stable. They do not fix absolute cost, which has not been, so absolute figures should be confirmed before any commitment. Reference 2 is an analyst estimate reported at second hand, used only to indicate the direction and approximate scale of agentic token consumption. Appendix C applies the bottom of the reported range.
ETerms usedOpen +Close −
| Term | Meaning as used in this article |
|---|---|
| Capability tier | A band of model capability and its associated price. This article refers to tiers rather than named products, because product names change faster than the underlying economics. |
| Frontier | The most capable tier generally available at a given moment, priced at about $10 per million input tokens and $50 per million output tokens in July 2026. Its position moves upward with each generation, and its price has risen accordingly. |
| High tier | The tier immediately below the frontier, at about $5 and $25 in July 2026. It was the frontier a year earlier, which is how capability gets cheaper while the top of the market gets more expensive. |
| Blended rate | A single per-million-token rate combining input and output pricing at a stated task profile, used for comparison against self-hosting, which is billed on total tokens. At the 1,500-and-500 profile: $2.00 small, $6.00 mid, $10.00 high, $20.00 frontier. |
| Dedicated inference | Running models on owned or reserved infrastructure instead of buying output from a provider per token. |
| Utilization | The proportion of available capacity actually doing work. Engineering sets the level of dedicated inference cost, and utilization multiplies that level, in inverse proportion. |
| Base load | Demand sustained across normal operation, excluding peaks. The correct basis for sizing committed capacity. |
| Routing | Sending each request to the least capable model sufficient to handle it, escalating only where a quality threshold is not met. |
| Escalation rate | The proportion of requests that fail the threshold at the first tier and pass to a more capable one. This is the number that shows whether a routing policy is working. |
| Context caching | Retaining repeated prompt content between calls so it is not billed at the full input rate each time. Reads are discounted, writes generally carry a premium. |
| Definition layer | A governed set of business definitions applied over data that stays in its source systems, so a term means the same thing across an organization. |
| Unit cost | Cost per unit of business work, such as per document processed or per claim assessed, rather than per token consumed. |
| Volume band | The range of volume over which a unit-cost commitment holds, outside which a stated adjustment mechanism applies. |