TPU hardware versions
Tensor Processing Units are Google's custom accelerators for the compute-heavy math of machine learning. This page walks the Cloud TPU family from the newest generation back to the foundational one - what distinguishes each version and how to choose the right accelerator for a demanding AI workload.
Cloud TPU family: a quick overview
| Version | What defines it |
|---|---|
| TPU v2 | The foundational public Cloud TPU - introduced scalable TPU slices with a 2D torus interconnect. |
| TPU v3 | A step up from v2 - more compute per TensorCore and greater memory, for larger models that didn't fit on v2. |
| TPU v4 | Flexible and efficient - 3D mesh topology, improved memory system, advanced options like twisted tori. |
| TPU v5p | Powerful and scalable - large-scale 3D interconnect topology and high HBM capacity, for very large models. |
| TPU v6e (Trillium) | Highest value and performance for transformer, text-to-image, and CNN models; adds SparseCore for embeddings. |
| TPU v7e (Ironwood) | Builds on Trillium and optimizes custom AI accelerators primarily for inference workloads. |
Ironwood (v7e): inference acceleration
Ironwood is designed to be highly performant, scalable, and energy-efficient, reflecting a strategic shift toward accelerating inference at scale. That focus supports the move from reactive AI systems (which surface data for humans to interpret) to proactive AI agents that autonomously retrieve, generate, and interpret data. Ironwood is purpose-built for the computational and communication demands of this "age of inference" - large language models, Mixture of Experts (MoEs), and reasoning tasks - minimizing on-chip data movement and latency while running extensive tensor manipulations.
- Peak compute per chip - 4,614 TFLOPs (TeraFLOPs).
- HBM capacity per chip - 192 GB of High Bandwidth Memory.
- HBM bandwidth per chip - 7.37 TB/s.
- Inter-Chip Interconnect (ICI) bandwidth - 1.2 TBps bidirectional, for coordinated, synchronous communication across the full pod.
- Pod scale - up to 9,216 liquid-cooled chips.
- Aggregate compute - a 9,216-chip pod delivers 42.5 exaflops.
- Power efficiency - 2x performance per watt vs. Trillium, and ~30x more power efficient than the first Cloud TPU (v2, 2018), supported by advanced liquid cooling.
Trillium (v6e)
Trillium (v6e) is Cloud TPU's newest-generation accelerator before Ironwood. Each Trillium chip contains a single TensorCore housing two Matrix-Multiply Units (MXUs), a vector unit, and a scalar unit. A Trillium VM can contain 1, 4, or 8 chips; slices of 4 chips and smaller share the same NUMA node. Trillium shares much of its architecture with v5e but brings significant advancements.
Specify it: TPU API - use AcceleratorType, e.g. v6e-8 for an 8-TensorCore
slice. GKE API - use --machine-type, e.g. t6e-standard-8t.
- Massive performance boost - significantly higher peak compute and bandwidth per chip.
- Increased memory - double the HBM capacity and bandwidth per chip.
- SparseCore - v6e adds SparseCores to accelerate embedding-heavy models (common in recommendation systems).
- Optimized for - transformer, text-to-image, and convolutional neural network (CNN) training, fine-tuning, and serving.
TPU v5p
TPU v5p is a high-performance accelerator distinguished by its scale and 3D interconnect topology. Each v5p chip contains a TensorCore with four MXUs, a vector unit, and a scalar unit. All 4x4x4 (one cube) and larger v5p slices have full 3D torus connectivity; slices smaller than a full cube are 3D connected but lack the wrap-around links that define a true 3D torus.
- Pod size - 8,960 chips with flexible, high-speed networking, configurable into various slice shapes.
- Single-slice training - supported up to 6,144 chips.
- Multislice - scale further, up to 18,432 chips.
Specify it: TPU API - use AcceleratorType as v$VERSION_NUMBERp-$CORES_COUNT,
e.g. v5p-256 is a slice with 256 TensorCores (128 chips). GKE API - all v5p TPU
VMs use the ct5p-hightpu-4t machine type.
Like TPU v4, ICI resiliency is enabled by default for v5p slices that are one
cube or larger (e.g. v5p-128, a 4x4x4 config). It improves fault tolerance and
scheduling availability.
TPU v4
TPU v4 provides a powerful, flexible platform, noted for its 3D mesh interconnect topology. Each TPU v4 chip contains two TensorCores; each TensorCore has four MXUs, a vector unit, and a scalar unit.
3D mesh vs. 3D torus - v4 chips connect directly to their nearest neighbors in
three dimensions (a 3D mesh). For topologies where dimensions are 2A=B=C or
2A=2B=C, the connections can be configured as a 3D torus, which generally
gives better performance thanks to improved communication paths.
- NUMA locality - v4 supports non-uniform memory access; bind your training script to NUMA node 0 to raise memory throughput, especially for CPU-heavy workloads.
- Unified HBM - a unified 32-GiB HBM space across both TensorCores improves coordination.
- Enhanced HBM and DMA - latest HBM standards and better direct memory access for high-performance striding.
- TensorCore improvements - doubled MXUs, higher clock rates, more transposition/permutation bandwidth, and faster MXU weight loading.
Recap
A dense comparison of the generally available versions:
| Chip metric | v5e | Trillium (v6e)* | v4 | v5p |
|---|---|---|---|---|
| Chips per pod | 256 | 256 | 4,096 | 8,960 |
| Bf16 TFLOPs | 197 | 918 (4.7x) | 275 | 459 |
| Int8 / FP8 TOPs | 394 | 1,836 (4.7x) | N/A | 918 |
| HBM per chip (GB) | 16 | 32 (2x) | 32 | 96 |
| HBM BW (GB/s) | 820 | 1,640 (2x) | 1,228 | 2,765 |
| ICI BW per chip (GB/s) | 400 bi-dir | 800 bi-dir (2x) | 600 bi-dir | 1,200 bi-dir |
| DCN BW per chip (Gb/s) | 25 | 100 (4x) | 50 | 50 |
| SparseCore (embeddings) | No | Yes (new) | Yes | Yes |
*Trillium multipliers are relative to v5e.
For exam recall, tie each milestone to its version:
| Milestone | Version |
|---|---|
| First public Cloud TPU with scalable slices and a 2D torus interconnect | v2 |
| First to deploy optical circuit switches (OCS) for dynamic interconnect reconfiguration | v4 |
| Large-scale 3D interconnect topology with high HBM capacity | v5p |