Artificial intelligence (AI) inference costs for each agentic AI workflow will increase more than fivefold through 2028, according to Gartner Inc., a business and technology insights company.
The forecast presents a growing cost problem for companies developing AI products. Although the price of processing individual units of AI data, known as tokens, is falling, newer AI agents need far more tokens to complete complex tasks.
Unlike basic chatbots that respond to individual questions, AI agents can plan, make decisions, use tools, and complete several steps. These capabilities require more computing power, increasing the total cost of running an AI service.
“Product leaders cannot rely on more efficient token economics to rationalize AI costs,” said Will Sommer, senior director analyst at Gartner. “Each successive generation of AI capability will necessitate more, and often more expensive, tokens. There is no reliable, economical one-size-fits-all model on the horizon. Producing competitive AI products will require developing and maintaining complex multimodel ecosystems.”
Gartner calls this trend the “Inference Paradox.” AI models are becoming cheaper to use per token, but those savings encourage companies to build more advanced and costly applications. As a result, the overall cost of AI continues to increase without a clear guarantee of matching business returns.
Gartner identified three factors behind the trend: falling costs for foundational AI models, the use of more powerful models for advanced applications, and the much higher token requirements of complex AI workflows.
“The harsh economics of the Inference Paradox are exemplified by the differences between a simple chatbot and an AI agent,” Sommer said. “Where a simple chatbot must read and interpret a query and quickly respond with a probabilistically reasonable answer, an AI agent must constantly reason, negotiate, and question itself.”
Sending a task to an AI agent that can reason increases the provider’s inference costs by at least five times compared with a basic chatbot interaction, Gartner said. Costs can increase further as tasks become more complicated.
Companies will need to make sure AI agents produce much greater returns or use cheaper models for simpler tasks. This may require systems that automatically choose the right model based on a task’s difficulty and cost.
“Defaulting to generic autonomous intelligence will result in unbounded costs, orders of magnitude higher than those of optimized product ecosystems,” Sommer said.