Nvidia is preparing a higher-capacity H100 computing card

In the previous year’s GTC 2022, NVIDIA unveiled its new generation H100, based on the Hopper architecture, for use in next-generation accelerated computing platforms. Possessing 80 billion transistors, it utilizes a CoWoS 2.5D wafer-level packaging in a single-chip design, manufactured using Taiwan Semiconductor Manufacturing Company’s (TSMC) bespoke 4N process for NVIDIA.

At present, NVIDIA proffers several variants of the H100 computing card, namely H100 PCIe, H100 SXM, and H100 NVL (dual-GPU version). The standard H100 is equipped with 80GB of HBM2E, available in both PCIe and SXM5 versions, while the H100 NVL boasts 188GB of HBM3, corresponding to 94GB per GPU. The recently updated PCI ID list indicates that NVIDIA will soon launch 94GB and 64GB SXM versions of the H100, with the former also featuring a version tailored for the Chinese market, designated as H800L 94GB.

The 64GB is self-explanatory, but the 94GB is not a common figure, similar to the H100 NVL, and NVIDIA has not elucidated how this capacity is configured. Last year, there were reports that NVIDIA was also preparing a 120GB PCIe version, though it has not yet been officially confirmed and may merely be an engineering sample being tested in the laboratory, ultimately not appearing in any market.

The complete GH100 chip comprises 8 GPC clusters, 72 TPCs, 144 SMs, and a total of 18,432 FP32 CUDA cores. It employs the fourth generation Tensor Core, totaling 576, accompanied by 60MB of L2 cache. However, not all of these are activated in the actual product, with the SXM5 version enabling 132 SMs, a total of 16,896 FP32 CUDA cores, 528 Tensor Cores, and 50MB of L2 cache. The PCIe 5.0 version, on the other hand, enables 114 SMs, with only 14,592 FP32 CUDA cores. The specifications for the H100 with larger video memory are not currently clear, but it is expected to perpetuate the existing configuration.