Beneath Tokenomics: Why $/FLOP Is Becoming A Critical AI Cost Metric

Vinod Bijlani is an AI practice leader at Hewlett Packard Enterprise.

One number comes up in almost every cost discussion I have with CIOs and AI leaders: price per million tokens.

This metric is useful for comparing inference services and forecasting application spend, but it only measures the price of consuming AI, not the productivity of the infrastructure producing it.

That distinction matters as AI usage shifts from occasional prompts to persistent agents, multimodal workflows and reasoning-intensive applications. A low token price can conceal underutilized accelerators, memory bottlenecks, network congestion and rising power costs.

Leaders, therefore, need to ask a second question: How efficiently are capital, energy and physical capacity being converted into computation?

This is where dollars per floating-point operation ($/FLOP) becomes strategically useful. A token is what an application consumes. A FLOP is one unit of the computation the infrastructure performs.

Tokenomics can help explain consumption economics, and datanomics addresses the economics of making data reliable and usable, as I have written about previously. $/FLOP adds the infrastructure lens without diminishing the other focuses, because each metric measures a different layer of the AI stack.

Two Curves Moving In AI’s Favor, And One That Isn’t

Epoch AI’s recent trend data indicates that the performance purchased per dollar of AI chip spending has improved by about 49% a year since 2023, which is encouraging. The organization’s research into algorithmic progress in language-model pretraining also found in 2024 that the compute required to reach a given performance level historically halved roughly every eight months.

Demand, however, is growing even faster. Another piece of Epoch AI estimates that training compute for frontier language models has expanded by about five times a year since 2020.