Dive Brief:
- Unexpected AI costs are forcing enterprises to rethink implementation plans, with nearly half of organizations reporting AI spending surprises have escalated to the board, according to Mavvrik’s 2026 State of AI Cost Governance Report. The cost management software maker surveyed 396 enterprise organizations across industries in April and May.
- Fragmentation and a disconnect between AI adoption and governance are having a knock-on effect on projects, with one-quarter of respondents delaying or canceling AI initiatives due to unforeseen costs.
- Unexpected AI costs had a material impact on at least one business decision for two-thirds of businesses, while one-third of companies deployed emergency spending freezes as bills mounted.
Dive Insight:
Rather than stemming from model costs alone, AI spending is increasingly widespread, spanning developer tools, data platforms, infrastructure and agentic workloads. That fragmentation, coupled with a lack of oversight, is making it harder for CIOs and finance leaders to understand the full cost of AI initiatives and accurately measure return on investment.
“AI is fundamentally changing how infrastructure is consumed and how costs accumulate,” said Mavvrik CEO Sundeep Goel in the report. “What used to be predictable IT spending is now dynamic, distributed, and increasingly difficult to attribute.”
The challenge is becoming more pronounced as organizations move beyond simple generative AI deployments toward autonomous agents, according to Rita Sallam, chief of research for data and analytics at Gartner.
“While raw model unit costs (e.g. cost per token) might seem to be dropping, the actual cost per completed task is rising as agentic workflows become more complex and require more advanced reasoning,” she told CIO Dive in an email. “Pricing is shifting to usage-based models, compounding the expense of running complex, autonomous agentic workflows that frequently eclipses the financial savings they were designed to generate.”
Visibility gaps are having a domino effect through the chain of command, ultimately influencing business decisions, triggering emergency spending reviews and altering AI projects as organizations reassess costs.
“Everyone is talking about AI to ROI, but ROI is math,” Benchmarkit CEO Ray Rike said in the report. “You can’t accurately calculate ROI if you don’t know your costs.”
Executives have long grappled with hidden costs. Data released last year from Wasabi Technologies found nearly two-thirds of organizations exceeded their cloud storage budgets due to unanticipated usage and egress fees piling up.
Cost management efforts have risen amid the growing expenditure gap. Cloud providers — such as Google Cloud, AWS and Microsoft — have responded to rising scrutiny and reduced some storage-related fees. Similarly, FinOps practices, once designed for cloud costs, are now being deployed for shielding businesses from whopping AI bills.
According to Sallam, CIOs should treat AI cost optimization as an architectural requirement rather than a “retrospective finance exercise.”
“Enterprises should also build cost optimization directly into agentic workflows,” she added. “Match the model to the task — avoiding the mistake of ‘using a Ferrari when a Kia will do.’”
Sallam also highlighted routing simple queries to cheaper models and using smaller, domain-specific language models as potential strategies.
Organizations should also invest in governance, security and data foundations alongside AI deployments, according to Sallam, arguing that those investments are essential to sustaining long-term ROI.
“AI users play a role too,” she added. “They must understand the financial consequences of their AI usage. Mandate AI literacy programs to teach users AI token financial literacy. You don’t want them using the most expensive reasoning models to check the weather.”
