Cloud TPUs cheat sheet

One-page review guide for the Cloud TPUs course. Each block links to the full page.

TPU (Tensor Processing Unit) - a custom application-specific integrated circuit (ASIC) built by Google to accelerate the matrix-heavy computations of machine learning.

When a TPU is the best bet

≡FactsTPU sweet spots

• Training massive deep-learning models - large-scale, compute-bound training.
• Models relying on embeddings - sparse recommendation / ranking workloads (SparseCore).
• Scientific and healthcare AI - high-throughput scientific computing.

See Use cases and case studies for the scaling architectures behind each.

Google TPU evolution

TPU generations, per-chip and per-pod performance relative to v2

#NumbersGeneration highlights

• v1 (2015) - internal inference-only accelerator.
• v2 (2018) - 1x/chip, 1x/pod baseline; distributed shared memory.
• v3 (2020) - 3x/chip, 12x/pod; liquid cooled.
• v4 (2022) - 6.6x/chip, 100x/pod; optically reconfigurable.
• v5p (2023) - 21x/chip, 750x/pod; most flexible AI accelerator.
• Trillium (2024) - ~100x v2 performance; next frontier of AI models.
• Ironwood (2025) - cutting-edge chip, largest pod: TPU 7 = 256 chips/pod, TPU 7x = 9,216 chips/pod.

Full family and specs: TPU hardware versions.

Physical grouping

#NumbersPod, slice, cube

• Pod - a collection of TPUs physically grouped and connected by a specialized high-speed network.
• Slice - a subset of chips within a single pod, connected by fast Inter-Chip Interconnects (ICI).
• Cube - a 4x4x4 topology of interconnected TPU chips.

Deep dive: TPU system architecture.

Consumption options

≡FactsGetting TPU capacity

• Long-term reservation - request and reserve TPU resources in advance for an extended period.
• On-demand - request resources as soon as possible, keep them as long as you want.
• Spot - cheapest; can be preempted by Google Cloud at any time if capacity is needed elsewhere.

#NumbersDynamic Workload Scheduler (DWS) modes

• Flex mode - request hardware for a specified period, from 1 minute to 7 days.
• Calendar mode - create future reservations for hardware you know you'll need in advance.

Full detail: Consumption options.

GPU/TPU interoperability

≡FactsPortable accelerator deployment

• Use a dual-container approach within a single pod for seamless GPU/TPU switching.
• The container whose accelerator (GPU or TPU) is present starts its vLLM server; the other sleeps - only the correct server for the underlying hardware runs.

Full detail: GPU/TPU interoperability.

Model development best practices

#NumbersThree principles

• Principle 1 - Layout for efficiency - tile-friendly tensor layouts that keep the MXU busy.
• Principle 2 - Fixed shapes for predictable performance - stable tensor shapes avoid recompiles.
• Principle 3 - Avoiding unnecessary padding - TPU-friendly dimensions minimize wasted compute.

Full detail: Model development best practices.

Recap

Exam guide§2.1

Consolidated review guide for the AI Infrastructure: Cloud TPUs course.