At the Google I/O developer conference, Google on Tuesday was announced his next generation Tensor Processing Units (TPU), the artificial intelligence chips of the data center. This sixth generation chip, called Trillium, will be released later this year.
“Google was built for this moment. We’ve been a pioneer in GPUs for more than a decade,” Google CEO Sundar Pichai said in a pre-conference press briefing.
Announcing the next generation of TPUs is something of a tradition at I/O, even as the chips are released later in the year. When they arrive, however, they will have a 4.7x performance boost in computing performance per chip compared to the fifth generation, according to Pichai.
In part, Google achieved this by expanding the chip’s matrix multiplication units (MXUs) and pushing the overall clock speed. In addition, Google also doubled the memory bandwidth for its Trillium chips.
What’s perhaps even more important, though, is that Trillium features the third generation of SparseCore, which Google describes as “a specialized accelerator for processing extremely large integrations common to advanced ranking and recommendation workloads.” This, the company claims, will allow Trillium TPUs to train models faster and serve them with lower latency.
Pichai also described the new chips as Google’s “most power-efficient” TPUs to date, which is especially important as demand for AI chips continues to grow exponentially. “Industry demand for ML computing has grown by 1 million over the past six years, roughly tenfold each year,” he said. This is not sustainable without investment in reducing the power requirements of these chips. Google promises that the new TPUs are 67% more energy efficient than fifth-generation chips.
Google’s TPUs have recently tended to come in several variants. So far, Google hasn’t provided additional details about the new chips or how much it will cost to use them in Google Cloud.
Earlier this year, Google also announced that it would be among the first cloud providers to offer access to Nvidia’s next-generation Blackwell processors. That means developers will have to wait until early 2025 to get access to these chips.
“We will continue to invest in infrastructure to power our advances in artificial intelligence and continue to break new ground,” Pichai said.
We’re launching an AI newsletter! Sign up here to start getting it in your inbox on June 5th.
