What is a tensor processing unit (TPU)?
A tensor processing unit (TPU) is an application-specific chip that Google designed to run the matrix multiplications at the heart of neural networks. A tensor is the multi-dimensional array of numbers that deep learning frameworks pass between layers, and the TPU is built around units that multiply such arrays at high speed and low power. Unlike a GPU, it has no graphics heritage and no general-purpose role.
TPUs are not sold as cards for an organization’s own servers. They are available as instances in Google’s cloud, where thousands can be wired together into pods for training very large models. Software reaches them through frameworks such as TensorFlow and JAX, and PyTorch support comes through an additional library.
Why TPUs matter
For most mid-sized companies, TPUs matter mainly as a point of comparison. They show that GPUs are one of several ways to run neural network math, and that cloud providers compete on custom silicon. A company that trains large models in Google’s cloud may find TPUs the better option for that job. A company that runs open-weight models on its own hardware will use GPUs, because that is what it can buy and what most inference servers support.
What to watch for
- Code tuned for TPUs ties the workload to one cloud, so weigh the gain against the loss of portability.
- Many open-weight models and inference servers are tested on GPUs first, and TPU support may lag or be missing.
- On-premises or air-gapped deployments cannot use TPUs at all.