NVIDIA H100 SXM: 80GB HBM3 video memory
At GTC 2022, NVIDIA released a new generation of H100 based on the Hopper architecture for the next generation of accelerated computing platforms. It has 80 billion transistors, is a CoWoS 2.5D wafer-level package, has a single-chip design, and is manufactured using a 4nm process tailored by TSMC for Nvidia.
Recently, ServeTheHome released a recent photo of the NVIDIA H100 SXM. You can see the new design of the SXM shape. The PCB model is PG520. It is understood that the equipped GH100 chip has an area of about 814 mm² and is located in the middle, surrounded by six HBM3 memory with a capacity of 80GB. Compared to the previous generation A100, the connection layout of the H100 has also changed, making it a bit shorter. The TDP of the NVIDIA H100 SXM is as high as 700W, which is 250W to 300W higher than similar products based on the Ampere and Volta architectures, but the PCIe version of the H100 is only 350W.
The full GH100 chip is equipped with 8 sets of GPC, 72 sets of TPC, 144 sets of SM, and a total of 18432 FP32 CUDA cores. It uses the fourth-generation Tensor Core, a total of 576, and is equipped with a 60MB L2 cache. However, not all of them are opened in the actual product. Among them, 132 groups of SMs are enabled in the SXM version, with a total of 16896 FP32 CUDA cores, 528 Tensor Cores, and 50MB of L2 cache, while the PCIe 5.0 version enables 114 groups of SMs, and the number of FP32 CUDA cores is only 14592. In addition, the GH100 supports NVIDIA’s fourth-generation NVLink interface, which can provide up to 900 GB/s of bandwidth. At the same time, the GH100 is the first GPU to support the PCIe 5.0 standard and the first to use HBM3. It supports up to six HBM3s with a bandwidth of 3TB/s, which is 1.5 times that of the A100 using HBM2E.
Recently, some retailers in Japan have listed the NVIDIA H100 PCIe, and the displayed price is 4,745,950 yen (about $36,567.5). Since the NVIDIA H100 SXM has a higher specification and has more CUDA cores, the price may be more expensive.
