The Chip Industry’s Emergent Challenge: Tackling The Memory Wall

Nandan Nayampally is Chief Commercial Officer at semiconductor IP company Baya Systems.

​The IPO market has made a massive comeback recently, first with Cerebras’ IPO giving it a market cap of nearly $100 billion, followed by SpaceX securing the largest IPO in history a week after announcing its deal to sell compute to Google. It’s no coincidence that semiconductor connections strengthened both companies’ IPOs; Cerebras takes a unique approach with its wafer-scale products while SpaceX is specifically driving a strong semiconductor supply chain message with its TeraFab Initiative and selling access to its AI chip stockpile.​

Cerebras and SpaceX exemplify the two main paths being pursued. SpaceX is, of course, trying to solve the supply side by rapidly building fabs derived from Tesla’s successful GigaFactory approach to vertical integration under the same roof. On the other hand, Cerebras is taking the memory wall head-on by collocating memory on-die close to the processors themselves.

These successes reflect, in two different ways, the “memory wall” problem: how we can get to a place where compute doesn’t have to wait for memory to supply the necessary data to the right place, and what we’ve seen manifested as an acute supply capacity problem with high-bandwidth memory (HBM).

Overcoming the memory wall problem is what’s standing between us and the next generation of AI compute. And right now, the chip industry is taking a number of different approaches to solve this problem.

Wafer-Scale: The Radical Approach​

Generally, a semiconductor wafer (up to 12 inches in diameter) is diced up into chips or dies, and any defective parts of the wafer can be discarded. This often produces hundreds of high-performance chips from one wafer.

In the more conventional approach, data has to move through interconnects (thin wires) from the memory where it’s stored to the compute chip that’s asking for it. That adds time to any computing task.