Power, Memory And Packaging, Not Transistors, Now Limit AI Chips

The transistor is no longer the bottleneck for AI silicon; power, memory bandwidth and packaging are. That was the premise I opened with on Tuesday, September 22, when I moderated the closing executive panel at the Global Semiconductor Alliance’s U.S. Executive Forum in Menlo Park, California in front of more than 150 semiconductor executives.

The session was titled “Silicon at the Breaking Point: Designing Chips for a World That Can’t Wait.” On stage with me were Paul Cho, president of Samsung Semiconductor; Mark Papermaster, AMD’s chief technology officer; Charlie Kawwas, president of Broadcom’s Semiconductor Solutions Group; and Richard Ho, OpenAI’s vice president of hardware. I’ve spent 35 years in this industry and constructed the premise of the session specifically to start an argument. I didn’t get one. All four of the panelists built on the premise from their first answers onward, and where they took the discussion is the point: The constraint has moved, and so has the job of the chip designer.

(Disclosure: AMD, Broadcom and Samsung are clients of my firm, Moor Insights & Strategy, as are Nvidia and Qualcomm mentioned below.)

Custom Silicon Went From Hyperscaler Hedge To Frontier-Lab Strategy

The Unit Of Purchase Is Now The Fleet, Which Changes What Chip Design Means

When I put forward this fleet-centered thesis, the four panelists converged on it. Ho said OpenAI’s unit of purchase isn’t a chip or a rack. It’s the entire fleet across multiple campuses, with power allocated by workload. Kawwas said that a gigawatt to train a cluster is no longer enough; the number is now two to four, and he expects single campuses of five to 10 gigawatts by 2031. When buyers plan in terms of gigawatts, chip design stops being a die decision and becomes a project of system co-design across memory, logic, packaging, networking and power, settled at the level of architecture definition rather than in procurement.