NVIDIA Unveils Next-Generation GH200 Grace Hopper Superchip Platform

In the early hours of today at SIGGRAPH 2023, NVIDIA’s founder and CEO, Jensen Huang, announced the launch of the next-generation GH200 Grace Hopper Superchip. This marks the first time in the world that a chip has been equipped with HBM3e. The newly constructed dual-processor system offers a 3.5-fold increase in memory capacity and three times the bandwidth compared to the current generation of products. NVIDIA did not disclose the supplier of the HBM3e, but it is speculated to originate from SK Hynix.

Compared to the currently used HBM3, the new generation HBM3e offers a 50% speed enhancement, with the dual-processor system providing up to 10TB/s of bandwidth, meaning each stack can supply 5TB/s of bandwidth. The GH200 Grace Hopper Superchip is a fusion of the Hopper architecture GPU and the Arm architecture Grace CPU interconnected using NVLink-C2C. It possesses 72 Arm v9 architecture CPU cores, and the GPU should align with the H100 computing card, namely 16,896 FP32 CUDA cores. The newly crafted dual-processor system will feature 144 Arm v9 architecture CPU cores and 282GB of HBM3e, delivering 8 Petaflops of AI computing performance.

NVIDIA stated that the new platform is designed to handle the world’s most complex generative artificial intelligence workloads, encompassing large language models, recommendation systems, and vector databases, suitable for various configurations. NVIDIA plans to sell two versions: one version containing two chips available for customer integration into systems, and another version comprising a complete server system designed with two Grace Hopper components.

Furthermore, NVIDIA will also employ HBM3e in the Hopper architecture’s GH200 computing card, expected to arrive in the second quarter of 2024, slightly later than its competitor AMD’s Instinct MI300X, which delivers 5.2TB/s of bandwidth and a capacity of 192GB through HBM3.