Dive Brief:
- Spending on AI-optimized infrastructure as a service — the compute that supports large language model training and operation — will nearly double through the end of the year, reaching $42 billion, a Gartner report released Monday found.
- The rise of agentic AI and inference models has amplified the need for the infrastructure and compute power. This year, the $23.3 billion of global spending on inference is projected to surpass the $19 billion of spending on training models, according to the report.
- Inference workloads represent the operational side of AI, where trained models generate responses, recommendations and decisions in real time, Hardeep Singh, senior principal analyst at Gartner, told CIO Dive in an email. “The fact that inference spending will exceed training spending in 2026 indicates that AI adoption is becoming more mainstream and production-oriented,” he said.
Dive Insight:
As companies mature their AI deployment plans and lean more heavily on agentic systems, their need for AI infrastructure is growing.
Enterprises are moving past building and training models to focus on deploying them and operating them at scale throughout their organizations, Singh said. Gartner forecasts that global spend on infrastructure as a service will continue to rise, reaching $66 billion in 2027.
“For the last few years, AI infrastructure demand was largely driven by model providers training the large foundation models,” Singh said. “Now, enterprises are embedding AI into applications, business processes and customer experiences, which requires continuous inference rather than periodic training.”
Forrester projected early this year that global technology spending would grow to reach $5.6 trillion in 2026, up from $5.2 trillion in 2025, amid rising demand for AI services.
CIOs and other tech decision-makers should see AI infrastructure is a strategic investment to operationalize AI across the business, instead of an experimental budget line item, Singh said. Enterprises now see demand for AI-optimized compute, storage, networking and orchestration capabilities that traditional infrastructure was not designed to support efficiently, he added.
Hyperscalers and frontier model developers continue to account for a large share of AI infrastructure investment. Top hyperscalers Google Cloud, Microsoft Azure and AWS plan to invest more than $500 billion in capital expenditures for AI infrastructure this year.
Enterprise demand has also accelerated rapidly — they will more than double their spending on generative AI models and AI agents this year, according to May Gartner data. Vendor-driven AI infrastructure that supports AI work, including AI-optimized IaaS, AI-optimized servers, AI network fabric, AI processing semiconductors and devices, accounted for more than 45% of spending, the report said.
As AI becomes embedded in enterprise workflows, they require infrastructure that’s capable of supporting higher performance, lower latency and greater scalability than before, Singh said. CIOs who are focused on scaling AI from pilots to business-critical production need to address compute capacity, data gravity, governance, operational resilience, security and cost management.
With all of these considerations in mind, many enterprises are reassessing their approaches to cloud, seeking a hybrid approach that combines public cloud, private cloud, colocation, edge and sovereign environments — or all of the above.
AI infrastructure is bigger than just a technology consideration for CIOs, Singh said.
“It is increasingly becoming a business capability that determines how quickly and effectively enterprises can scale AI across the enterprise,” he said.
