Why Enterprise AI ROI Is An Architecture Problem

Ragy Thomas is Chairman, Co-CEO and Co-Founder of UnifyApps, and Founder and Chairman at Sprinklr. Author - “The Enterprise Brain.”

​CIOs are continually asking me the same two questions: What is AI actually costing us, and is it delivering measurable ROI?

The numbers explain the anxiety. According to Gartner, “72% of CIOs reported that their organizations are breaking even or are losing money on their AI investments.” Gartner also forecasts that “over 40% of agentic AI projects will be canceled by the end of 2027, due to escalating costs.”

AI isn’t getting expensive because intelligence itself costs more. It is getting expensive because most businesses bolt AI onto fragmented architecture. Every new agent rebuilds context, integrations, controls and verification around itself, while workflows become hard-wired to individual models. That same fragmentation makes ROI harder to prove because costs, decisions and outcomes remain scattered across systems.

The problem is architectural.

Fix the architecture and you change both sides of the ROI equation: The marginal cost of deploying AI falls, while the economic contribution of each workflow becomes easier to measure. Below are four architectural moves that can improve the unit economics of enterprise AI.

​Mandate a shared context layer.

Most AI agents become expensive because they have to fetch context repeatedly from multiple systems just to answer one question.

One multinational beauty company’s team saw this with their first attempt at agentic automation. The models worked and demoed well. But the agents kept breaking the moment they hit production, because each of the six point solutions they’d stitched together kept their own knowledge and governance in a silo. Every agent started cold, forcing the team to reconnect, reconcile and rebuild that context for every single new project.

Context is already prepared and shared, which cuts redundant token usage, reduces API traffic and makes responses both faster and cheaper.