Consumption options
Consumption options are the different "rental agreements" for your TPU compute power. Each option sets its own terms for how quickly you get capacity, how long you can keep it, and what it costs. Picking the right one is as much a financial and logistical decision as a technical one.
Factors to weigh
Before choosing an option, size up the workload against five questions:
How quickly do you need the TPU capacity to be available?
How long will you need the capacity for your workload?
Do you need a precise, defined period, or can you be flexible?
Can the workload handle being unexpectedly stopped (preempted)?
What is your budget?
Quota: your compute allowance
Quota is your pre-approved allowance for using Cloud TPU cores. Google Cloud uses quotas to ensure fair resource distribution, prevent usage spikes, and help customers avoid cost overruns. A quota limits how much of a specific resource a project can consume, and it applies to hardware, software, and network components alike (API call volumes, concurrent load balancers, total projects, and so on).
Your quota requirements depend on how you consume TPUs:
- Cloud TPU APIs directly - you need either on-demand or preemptible quota for the number of TPU cores you want. Different TPU versions have different default quotas.
- TPUs through GKE - you use Compute Engine API quota instead, a different quota system.
Managing TPUs through GKE consumes Compute Engine API quota, not Cloud TPU quota. If you plan capacity around TPU quota alone, GKE workloads can fail to schedule. See the Cloud TPU quotas documentation for the specifics.
Dynamic Workload Scheduler (DWS)
Dynamic Workload Scheduler (DWS) optimizes the allocation of high-demand, finite resources such as GPUs and TPUs, so workloads can get the hardware they need when they need it. It directly addresses the supply-and-demand imbalance by balancing customer demand against the cloud's finite resources.
- Built on Google Borg technology.
- Improves access to AI/ML resources, helps optimize spending, and schedules all needed accelerators simultaneously.
- Supports TPUs and NVIDIA GPUs.
- Integrated with Compute Engine Managed Instance Groups, GKE, Agent Platform, and Batch.
DWS offers two provisioning modes. For simplicity: flex start mode is like waiting in line, and calendar mode is like booking a hotel.
The three consumption options
Beyond DWS, TPU capacity is requested through three main methods that trade off assurance, flexibility, and price.
Recap
Trade assurance and price against your workload's tolerance for interruption. Long-term reservations give the highest assurance of dedicated capacity plus a committed use discount - the go-to for large, long-running training. On demand maximizes flexibility with instant access and no fixed end. Spot is the cheapest, best for work that can survive being preempted.