Intel’s Generative AI Breakthrough: Gaudi 3 Unveiled
At the Vision 2024 event, Intel announced the launch of the Gaudi 3 accelerator, offering a generative AI solution for global enterprises, building upon the performance and scalability of its predecessor, the Gaudi 2. Intel claims that, compared to NVIDIA’s H100, the Gaudi 3 boasts a training performance improvement of 70%, an inference capability increase of 50%, and a 40% enhancement in efficiency, all while consuming less power.
The new Gaudi 3 generation offers double the FP8 and quadruple the FP16 performance compared to the Gaudi 2; its manufacturing process has been upgraded from 7nm to 5nm, fabricated by TSMC (Taiwan Semiconductor Manufacturing Company); it incorporates the fifth generation of tensor processing cores, with the number of cores increased from 24 to 64, in addition to eight matrix compute engines; the memory and cache have been upgraded from 96GB HBM2E / 48MB to 128GB HBM2E / 96MB, and the memory bandwidth has been enhanced from 2.45TB/s to 3.7TB/s; network connectivity has evolved from 24x100GbE to 24x200GbE; it provides a 600W TDP PCIe 5.0 dual-slot expansion card and a 900W TDP OAM module.
Although the TDP of the PCIe expansion card is 300W less than that of the OAM module, both offer the same FP8 peak performance of 1835 TeraFLOPS, albeit with potentially lower sustained performance. The PCIe version can operate in groups of four per system, whereas the OAM version can be configured in groups of eight per server. Intel also provides a targeted end-to-end full-stack AI software solution for generative AI for the Gaudi 3 accelerators, including embedded software, software suites, AI software, and AI applications.
Intel plans to ship the Gaudi 3 accelerators to customers in the third quarter of this year, with OEM manufacturers such as Lenovo, HP, Dell, and Supermicro utilizing this new product to build systems. Intel has not disclosed the pricing for the Gaudi 3 accelerators, but rumors suggest it will be significantly lower than that of the H100.
