NVIDIA rack Vera Rubin is bearing the brunt of high memory prices. Estimated costs for assembling the astronomical quantities of RAM and VRAM are up by 435%. The BOM (Bill of Material) for a complete rack with 72 GPUs and 36 CPUs and all that goes with them is… $7.8 million! Estimates from Morgan Stanley.
Vera Rubin: memory costs impact professionals too!
How is a rack organized?
With Vera Rubin, the total hardware bill has risen sharply, with the cost of assembling a complete rack coming in at $7.8 million. As a reminder, an NVL72 rack is made up of 72 Rubin GPUs and 36 Vera CPUs. Each GPU is backed by 288 GB of HBM4 memory, while each CPU manages 1.5 TB of LPDDR5X memory. To this must be added the network, cooling systems, power supply, etc., etc.
In terms of organization, it’s simple: a complete rack is made up of 36 superchips, each displaying two GPUs (2×288 GB HBM4) and one CPU (1.5 TB LPDDR5X). Finally, there are four superchips per tray.
Memory, the second largest item of expenditure, accounts for 26% of the total cost:
On the expense side, the element that weighs most heavily on the bill is the GPUs. For a complete rack, GPUs alone are estimated to cost no less than $4 million. In contrast, a current-generation NVL72 Blackwell rack costs $2.5 million for GPUs. Then, in second place, we have memory, which represents a 435% increase in expenditure. In total, estimated memory costs(HBM and LPDDR) are over $2 million, compared with “only” $373,939 for Blackwell. Finally, the third item of expenditure is CPUs, with an estimated unit cost of $5,000, i.e. $180,000 in total.
With these three components, we arrive at $6.14 million, the last two million being allocated to network hardware, PCBs, etc., etc..
A bill that professionals will also have to pay!
We’re only talking about the BOM here, just the bill for the hardware. Then there are the assembly and delivery costs, the margin NVIDIA takes, taxes, etc., etc., etc. Clearly, businesses will need to set aside a hefty budget to afford an NVL72 rack . What’s more, it’s a safe bet that these racks are destined for very large companies such as Amazon, Meta, OpenAI or Microsoft.
In any case, the price of GPUs has risen by 57% compared to Blackwell, while the price of memory has skyrocketed: +435%, and alone represents 26% of the hardware cost of a rack.
