Dive Brief:
- Agentic AI workloads are placing new strain on enterprise architecture, with one prompt triggering hundreds of actions that makes legacy IT systems financially unsustainable, according to Google’s 2026 State of AI Infrastructure report.
- Based on a survey of more than 1,400 senior IT leaders, 83% said their infrastructure needs upgrades in order to support agentic AI systems. Only 17% have full confidence in their tech stack’s ability to support mission-critical AI agents. Costs are also rising as legacy tech struggles to support agentic workloads.
- “Traditional architectures are cracking under this pressure,” Nirav Mehta, VP of product management at Google Cloud, said in the report’s foreword. Mehta said 62% of IT leaders are seeing high inference costs driven by “data egress, storage bloat, and idle specialized hardware” in legacy systems.
Dive Insight:
Rising costs are top of mind for senior executives as AI adoption speeds ahead.
Amid rising concerns, Oracle and AWS rolled out new features in June in an attempt to help enterprises alleviate and manage AI costs. Meanwhile, the Linux Foundation launched the Tokenomics Foundation to develop best practices for enterprises using AI at scale.
The shift toward autonomous agents has meant a shift in AI compute demands, with inference accounting for nearly half of all workloads compared with just 28% for training workloads, according to Google’s infrastructure report.
Despite predictions that inference costs are expected to drop over the next four years, enterprises aren’t expected to see direct savings as demand rises for frontier capabilities such as agentic AI, which uses more tokens than generative AI, according to Gartner.
As AI agents propel an inference boom, 96% of senior IT leaders said cost efficiency is imperative in guiding decisions related to AI infrastructure, according to Google’s infrastructure report.
In the report, the tech giant concludes that agentic AI merits a new infrastructure standard, with governance playing a critical role in scaling inference and hybrid multicloud architecture — which Google defines as a mix of on-premises and multicloud — setting the bar for AI deployment, especially amid a push for digital sovereignty.
The report concluded that 75% of enterprises outside of the U.S. will have adopted a sovereignty strategy by 2030.
Efficiency is also becoming an important requirement with 91% of senior IT leaders factoring in power consumption when selecting hardware.
“In the agentic era, energy has moved from a technology concern to a boardroom priority,” the report said.
