Why Billion-Dollar AI Systems Are Only As Fast As The Data Feeding Them

It is easy to picture an AI data center as one vast, self-contained machine: a building packed with GPUs and servers churning through heavy computational work.

The reality is far more connected. Large AI operations tend to distribute workloads across multiple parts of a site and, in many cases, across separate facilities. Compute in one location constantly draws on data stored nearby and on other systems. The traffic also extends externally, linking distant data centers and cloud-based resources.

That dispersed architecture makes network performance both a technical and business issue. If data takes too long to reach a GPU, the processor can’t do the work companies have spent heavily to equip it for. “The GPU sits there waiting on data,” says William Kellogg, vice president of strategic markets at Spectrum Business®. Repeated throughout a major deployment, those delays add up, leaving costly computing capacity underused and lengthening training cycles, which can push back the point at which new services start generating revenue. For the operators Spectrum Business works with, connectivity can therefore affect how quickly an AI investment begins to pay off.

“If you can’t access data from adjacent environments, now all of a sudden your competitors have a leg up.”

Kellogg, who works with hyperscalers and operators on these networks, sees competitive stakes too. “If you can’t access data from adjacent environments, now all of a sudden your competitors have a leg up,” he says. “Look at the AI models in the market today. Some are leading, some are catching up. A lot of that comes down to the connectivity layer.”

The Reason AI Puts So Much Pressure On The Network

The data volumes are enormous, and they’re set to rise sharply. Research suggests the computing power used to train leading AI models is growing four to five times a year¹. A single GPU rack or cluster can handle terabits of data, Kellogg says, with requirements potentially rising to millions of gigabytes.