Transforming Deep Learning: Google’s Powerful 4th-Gen TPU Accelerators

Google announced the launch of its fourth-generation Tensor Processing Unit (TPU) accelerators at Google I/O 2021, and in 2022, the company unveiled the largest machine learning center constructed with these accelerators. Operating with 90% carbon-free energy, Google emphasized that the fourth-generation TPU accelerators would power over 90% of its artificial intelligence training, and hinted at the introduction of a new generation of TPU accelerators.

In a recent announcement, Google stated that compared to the third-generation TPU accelerators, the fourth-generation model enhances deep learning performance by more than tenfold. The new accelerators also reduce energy consumption by two to three times during deep learning domain specific accelerators (DSA) algorithms and cut carbon dioxide emissions by over 20 times when operating in traditional data centers.

Leveraging the performance, scalability, and energy efficiency of the fourth-generation TPU accelerators, Google aims to accelerate the training of large-scale natural language models while significantly reducing overall data center energy consumption.

Google’s largest-scale natural language model, PaLM, with over 540 billion parameters, was trained using a supercomputer powered by fourth-generation TPU accelerators in a span of more than 50 days. Additionally, the AI service provided by startup Midjourney, which automatically generates images from text, also employs a supercomputer designed by Google with fourth-generation TPU accelerators.

Google emphasizes that compared to NVIDIA’s A100 accelerators built with the Ampere architecture, the fourth-generation TPU accelerators deliver a 1.7 times performance improvement and a 1.9 times reduction in energy consumption under the same configuration.

However, Google has not compared its accelerators to NVIDIA’s H100 accelerators based on the Hopper architecture. This is because the two products were not launched simultaneously. Nonetheless, Google hinted at the upcoming release of a new generation of TPU accelerators to compete with the H100 but has not disclosed any specific details. The company may reveal more information during this year’s Google I/O 2023 event.