Google today announced the release of its new Gemini Long-Language (LLM) model, and with it, the company also released its new Cloud TPU v5p, an updated version of the Cloud TPU v5e, which launched in general availability earlier this year. A v5p pod consists of a total of 8,960 chips and is supported by Google’s fastest interconnect to date, with up to 4,800 Gpbs per chip.
Not surprisingly, Google promises that these chips are significantly faster than the v4 TPUs. The team claims that v5p features a 2x improvement in FLOPS and a 3x improvement in high-bandwidth memory. However, this is a bit like comparing the new Gemini model to the older OpenAI GPT 3.5 model. Google itself, after all, has already moved the latest technology beyond TPU v4. In many ways, however, the v5e pods were a bit of a downgrade from the v4 pod, with only 256 v5e chips per pod versus 4096 in the v4 pods, and a total of 197 TFLOPs of 16-bit floating-point performance per v5e chip versus 275 for the v4 chips. For the new v5p, Google promises up to 459 TFLOPs of 16-bit floating-point performance, supported by the fastest interface.
Image Credits: Google
Google says that all this means that TPU v5p can train a large language model like GPT3-175B 2.8 times faster than TPU v4 — and do it more cost-effectively (though TPU v5e, though slower, actually offers more relative return per dollar than v5p).
Image Credits: Google
“In our early usage, Google DeepMind and Google Research observed a 2x speedup for LLM training workloads using TPU v5p chips compared to performance on our TPU v4 generation,” writes Jeff Dean, Chief Scientist, Google DeepMind and Google Research. “Strong support of ML Frameworks (JAX, PyTorch, TensorFlow) and orchestration tools enables us to scale v5p even more efficiently. With the 2nd generation of SparseCores we also see a significant improvement in the performance of integrations-heavy workloads. TPUs are critical to enabling our larger-scale research and engineering effort on cutting-edge models like Gemini.”
The new TPU v5p is not yet generally available, so developers should contact their Google account manager to get listed.
