Most European IT teams integrating AI tools in 2025 made the same architectural decision by default: call a model API, ship a feature, move on. The per-token cost was visible, the deployment was fast, and the infrastructure looked unchanged. What changed quietly was everything underneath — storage growth, egress bills, network latency, and the eventual reckoning when usage scaled. AI infrastructure capacity planning is not a problem you encounter at the prototype stage; it is the problem that arrives six months into production.
Global cloud infrastructure spending reached US$399.6 billion in full-year 2025, up 24% year over year, and Omdia explicitly noted that AI demand is no longer confined to GPU compute. It is driving broader infrastructure load across CPUs, storage, and networking. Enterprise generative-AI spending jumped from an estimated US$11.5 billion in 2024 to US$37 billion in 2025. Model API costs alone doubled to US$8.4 billion. The money is moving fast, but the measurement is not keeping up: only 51% of organisations can confidently evaluate whether their AI investments are delivering returns.
That gap between spending and visibility is where the real infrastructure risk lives.
The inference bill is not the infrastructure bill
Budgeting for AI based on per-token inference pricing is like budgeting for a vehicle based on the sticker price. The inference cost is real, but for most production setups it represents only a fraction of total AI infrastructure cost. GPU compute itself represents only 30 to 40% of true infrastructure investment. The rest is vector database retrieval fees, embedding generation, storage growth, cross-region egress, agent retries, idle GPU capacity, and developer tooling that nobody is tracking centrally.
Average enterprise AI budgets reached US$85,521 per month in 2025, a 36% increase from the year before. Yet the hidden costs that compound that number rarely appear on the LLM pricing page. Managed RAG pipelines carry high minimum monthly fees for dedicated storage and vector databases. Query costs scale with index size, so the same search can cost ten times more as data grows from 10GB to 100GB. Embeddings, reranking, backups, and reindexing can double real production spend before anyone has noticed.
Token pricing is what gets quoted in every architecture decision. It is often not the dominant cost once you are running at scale.
Agentic amplification: when a workflow becomes a cost event
Inference costs now account for 80 to 90% of AI spend overall, but agentic flows cost 5 to 25 times more than standard chat interactions. The mechanism is subagent fan-out: an AI agent spawns multiple sub-tasks, each consuming tokens and compute, often without a hard ceiling. One documented enterprise incident produced a US$47,000 bill from a single agentic workflow that expanded beyond its intended scope. That is not a billing anomaly; it is the expected behaviour of an architecture without spend controls.
Agentic workloads also change the network profile of your deployment. A single user request that triggers a chain of tool calls, retrieval steps, and sub-agent invocations generates far more round-trip traffic than a simple chat completion. If your inference endpoint and your vector store sit in different availability zones or different cloud regions, you are paying data transfer fees on every hop of that chain, independently of internet egress.
The networking layer nobody is watching
Data movement is where AI infrastructure costs become genuinely difficult to predict. Moving large context windows across cloud regions incurs steep egress fees, typically US$0.08 to US$0.12 per GB on hyperscaler networks. Cross-AZ data transfer on AWS charges US$0.01 per GB in each direction; for a high-traffic multi-AZ inference deployment, this accumulates independently of any external traffic. Storage-to-compute throughput mismatches leave GPUs idle during data loading, inflating cost per token without any increase in useful output.
This is where network architecture matters for measuring AI infrastructure impact, not just for performance. We operate 44 points of presence across Europe and the Middle East, with direct peering to AWS, Azure, GCP, Oracle, and Cloudflare at every site. For workloads that span a hyperscaler API and on-premises or colo infrastructure, that peering topology shortens the physical path and reduces the egress surface where charges accumulate. The consequence is measurable: fewer hops means lower latency on agentic chains and a smaller cross-region transfer bill.
GPU workload vs API calls: the self-hosting threshold
For teams weighing LLM API costs against self-hosted inference, the economics are now reasonably clear. Amortised self-hosted inference on modern GPUs has fallen to roughly US$0.02 to US$0.11 per million tokens. Mid-tier and frontier cloud APIs run US$3 to US$30 per million tokens. The break-even point for self-hosting occurs around one billion tokens per month; at that scale, the 10 to 30 times API markup becomes prohibitive.
But self-hosted inference is not just a GPU purchase. Hardware requirements scale with model size: 7B parameter models need roughly 3.5GB VRAM at 4-bit quantisation, while 70B models require 35GB or a multi-GPU setup. The H100 SXM is the current production standard for LLM inference, with 80GB HBM3 and 3.35 TB/s memory bandwidth. The GPU is one component; the rest of the stack — orchestration, API gateway, load balancer, monitoring, and autoscaling logic — carries its own operational overhead.
There is also a user-count threshold between roughly 500 and 5,000 active users where shared cloud inference APIs stop being viable and self-hosted infrastructure becomes necessary. At that inflection point, infrastructure costs can jump 3 to 5 times overnight if the team has not planned for it. GPU utilisation inefficiency is the biggest cost leak on the self-hosted side: a GPU running at 10% load inflates cost per token by 10 times, and self-hosted capacity must be provisioned for peak demand, leaving expensive resources idle during quiet periods.
What capacity planning for AI actually looks like
AI infrastructure capacity planning requires treating the AI stack as a distributed system, not a third-party service line item. That means instrumenting token throughput and latency at the application layer, tracking egress separately by destination (same-region, cross-AZ, cross-region, internet), monitoring GPU utilisation against a target band rather than just watching for errors, and setting hard spend limits on agentic workflows before they reach production.
It also means deciding early whether your workload trajectory points toward self-hosted inference. If you are approaching a billion tokens per month, the API economics no longer work in your favour. If your agentic workflows are generating unpredictable fan-out, you need rate limiting and cost guardrails before the architecture is locked in. Neither of these decisions is primarily about the AI model; both are infrastructure decisions that need to be made with the same rigour you would apply to a database migration or a storage tier change.
The teams that will manage AI cloud costs effectively in 2026 are not the ones with the best model selection. They are the ones who treated AI adoption as a capacity planning problem from the first production deployment, not after the first surprise invoice.
