Artificial intelligence (AI) inference costs are unlikely to follow Jevon’s paradox, where greater resource efficiency leads to higher demand. In a recent report, analyst Gartner disputes the theory, which – in the context of AI – would increasingly drive down token costs, leading to higher AI consumption and improved industry economics.
In the report, Gartner discusses the paradox where more efficient token economics leads to higher token consumption and the deployment of higher-cost tokens. The authors of the report warn IT decision-makers that navigating the paradox and achieving a return on investment (ROI) will require “a relentless pursuit of inference efficiency and optimised model orchestration”.
According to Gartner, the value of tokens is variable, and tokens become more expensive to generate based on model complexity. In the Inference paradox report, Gartner analysts note that as AI workflows become more sophisticated, token consumption escalates exponentially.
The authors of the report point out that enterprises building multistep workflows powered by autonomous agents will need to take into account the need for exponentially greater token consumption, often from relatively more expensive models, which also means they require more memory, reasoning and validation. From a cost management perspective, Gartner’s analysis suggests that using advanced AI agents with reasoning capabilities are 150 times more expensive to run than similarly sized basic AI chatbots for a single task.
Gartner said AI agents need to be trained on how to think and what to do if something goes wrong, noting: “They need to be able to validate their results for accuracy without necessarily having a human in the loop. They need to talk to other agents.
“Our modeling indicates that, using mainstream compute, the hardware costs to train a medium-sized agentic model with advanced reasoning capabilities would be 2.5x greater than those needed to train a simple chatbot of the same size. Inference costs for the agentic model would be approximately 5x greater. Then the agentic model would need 5x to 30x more tokens on average than a chatbot to solve an equivalent task.”
According to Gartner, this means that a task that would cost a simple chatbot $0.01 could cost an AI agent up to $1.50.
When looking at different types of agentic AI tasks, Gartner found that the choice of model has a significant impact on the cost to the provider of the AI system. “We estimate that the provider cost per token generated by models optimised for ‘planning and learning’ is presently about 8x to 10x that of models that are best suited to ‘basic linear workflows’,” the report’s authors said.
Will Sommer, senior director analyst at Gartner, said: “Each successive generation of AI capability will necessitate more, and often more expensive, tokens. There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”
The analyst firm urged IT decision-makers to avoid defaulting to generic autonomous intelligence, which it warned would result in unbounded costs orders of magnitude higher than those of optimised product ecosystems, where the people responsible for the development of AI capabilities in their organisation address inference tiering to optimise AI models against specific use cases.
