How GPUs, Google's TPUs and the custom chips of AWS and Meta differ in design, software and measured results, and what a compute buyer should ask.
NVIDIA describes its H100, the company's ninth generation of data centre GPU, as built mainly for AI, high-performance computing and data analytics. Only two of its processing clusters can run graphics shaders at all.
Open one up and you find a grid of identical processors, which NVIDIA calls streaming multiprocessors. The H100 in its SXM5 form has 132. Each carries 128 FP32 CUDA cores for general arithmetic and four Tensor Cores for the matrix multiplication that neural networks run on, and all of them are kept busy by thousands of threads running the same instructions on different data, a style that Google's TPU v4 paper, describing the A100 that preceded the H100, labels single instruction, multiple threads.
A GPU still does high-precision maths: the H100 handles the 64-bit floating point of scientific simulation as well as the 8-bit formats used for AI. Its code also carries over. NVIDIA's Blackwell tuning guide says the architecture keeps the CUDA model of Ampere and Hopper, and that software written to their best practices should usually run faster on Blackwell without changes.
A Blackwell GPU holds 208 billion transistors on two dies, joined by a 10 TB/s link and seen by software as one GPU, and NVIDIA's GB200 NVL72 rack puts 72 of them with 36 Grace CPUs into one liquid-cooled NVLink domain.
Google's TPU dates from a 2013 forecast: if people used voice search for three minutes a day, the speech recognition behind it would force the company to double its data centres. It started a high-priority project for a custom inference chip, and bought off-the-shelf GPUs for training. Fifteen months later the first TPU was in service, and it has run in Google's data centres since 2015.
At its centre is a matrix unit of 65,536 8-bit multiply-accumulate cells, with a peak of 92 trillion operations per second and 28 MiB of on-chip memory. The unit is a systolic array. Data flows in from the left and weights load from the top, meeting in the cells at regular intervals, so every number fetched from memory is used many times over. That matters because reading a large on-chip memory costs far more power than the arithmetic does.
The design drops the caches, out-of-order execution and multithreading that lift a CPU or GPU's average throughput. The authors saw those features as a poor fit for strict response times, and leaving them out kept the TPU small and low-power. They called it "closer in spirit to an FPU (floating-point unit) coprocessor than it is to a GPU."
Google Cloud's documentation, updated on 29 October 2026, lists TPU7x as the latest TPU: the first chip of Ironwood, the seventh generation. Each chip has two TensorCores and four SparseCores, peaks at 2,307 TFLOPs in BF16 and 4,614 in FP8, and carries 192 GiB of high-bandwidth memory at 7,380 GBps. SparseCores serve embeddings, the large lookup tables behind ranking and recommendation. On TPU v4 they sped such models up 5 to 7 times for 5% of die area and power.
A TPU7x pod links up to 9,216 chips, each wired directly to its neighbours in three dimensions, in cubes of 4x4x4. For TPU v4, Google's paper describes optical switches that can reconfigure that network at under 5% of system cost and under 3% of its power. It also reports that over 90% of training at Google then ran on TPUs.
Amazon Web Services designs its own accelerator too, and calls Trainium3 its fourth-generation purpose-built machine learning chip. Eight NeuronCore-v4 cores deliver 2,517 TFLOPS in the MXFP8 and MXFP4 formats and 671 in BF16, with 144 GiB of memory at 4.9 TB/s. AWS sells it in UltraServers of up to 144 chips. Its documentation lists support for dynamic shapes and control flow, two areas where Google's guidance steers work away from TPUs.
Meta builds MTIA for its own workloads. The second generation, described in April 2024, was designed to serve ranking and recommendation models. It is an 8x8 grid of processing elements on TSMC's 5nm process, with more on-chip SRAM than typical GPUs so that it stays busy when inference batches are small. Meta presents it as a complement to commercially available GPUs.
In both cases the chip is tied to one company's data centres or cloud, and to that company's compiler.
Code for a TPU must pass through XLA, a just-in-time compiler that turns the model graph into TPU machine code. On TPU7x that means JAX or PyTorch. TensorFlow, the framework the first TPU ran, is not supported. The compiler also fixes tensor shapes at the first batch, which is why Google's documentation calls models with changing shapes a poor fit and asks for dense maths that tiles into 128x128 blocks to fill the matrix unit.
Trainium runs through AWS's Neuron SDK, which AWS says runs PyTorch, vLLM, Hugging Face and Ray code without modification. AMD's GPUs ran PyTorch on its ROCm stack in MLPerf's September 2026 inference round, and MTIA uses PyTorch 2.0 with a Triton compiler back end.
Google's own guidance sends models with many custom PyTorch or JAX operations to GPUs and keeps TPUs for work dominated by matrix arithmetic. Teams with many hand-written CUDA kernels have the most to port.
Most published comparisons come from the chip makers themselves. Google's 2017 paper set its TPU against an Intel Haswell CPU and an NVIDIA K80 GPU on production inference and found it 15 to 30 times faster on average, with 30 to 80 times better performance per watt. Six years later the TPU v4 paper reported 1.2 to 1.7 times the speed of the NVIDIA A100 in systems of similar size, on 1.3 to 1.9 times less power. The authors left out the H100. When they did the work in 2022, they note, it was not yet on offer from AWS, Azure or Google Cloud.
MLPerf, run by the MLCommons consortium, is an open, peer-reviewed benchmark. Every submission must reach a set model accuracy, and chip makers, cloud providers and server builders all take part. Its record on TPUs has gaps:
Neither 2026 round includes Ironwood, Trainium3 or MTIA, so none of them has a neutral, current comparison with Blackwell. Peak figures from data sheets are a weak substitute, since they use different number formats and real speed depends on how much of the peak a workload reaches. The TPU v4 paper reports a large language model training at about 60% of peak.
A TPU is an application-specific chip that Google designs for machine learning and rents through Google Cloud. A GPU is a programmable parallel processor, sold by NVIDIA and AMD, that runs AI alongside scientific and data work. Both spend most of their effort multiplying large matrices. Google's first TPU left out caches, out-of-order execution and multithreading to stay small and low-power, and current TPUs still work best on code that compiles into large matrix operations of fixed shape.
Tensor Processing Unit. Google describes TPUs as its custom-developed application-specific integrated circuits (ASICs) for machine learning. They have run in its data centres since 2015, and Ironwood is the seventh generation.
Google's guidance favours TPUs for models dominated by matrix maths, with no custom operations in the main training loop, large batch sizes, runs of weeks or months, or very large embeddings. It points models with many custom PyTorch or JAX operations to GPUs, and lists frequent branching and high-precision arithmetic as poor fits for TPUs.
AMD and Intel both entered accelerators in MLPerf's September 2026 inference round, and AMD's Instinct GPUs appeared in the June 2026 training round. Google (TPU), Amazon Web Services (Trainium) and Meta (MTIA) design their own accelerators for their clouds and data centres.
AI chips and compute are one of the subjects of Panel 3, on AI infrastructure, at the Davos Super Intelligence Summit 2027, held on 19 January 2027 at the Grandhotel Belvedere in Davos. The same panel covers data centres, energy and scaling, and sovereign AI infrastructure, and the programme sets out all four panels.