What is it made of?
A datacenter GPU is not one chip but a package: a processor die, towers of memory, a silicon baseplate wiring them together, and a power-and-cooling system built to remove a kilowatt of heat. Follow a single multiply through it, then open any layer.
- What are the parts of an AI accelerator, and what does each one do?
- Why does the memory sit on the package instead of on the motherboard?
- Where does a kilowatt of heat go?
Follow one multiply through the package.
A model’s answer is billions of multiply-adds. Each one makes the same six-stage trip, and each stage is a part you can see. Step through it and the model lights up the part doing the work.
The weights wait in memory
Before anything is multiplied, the model’s numbers (its weights) sit in the HBM stacks beside the processor: 80 GB of them on an H100. No arithmetic happens here.
How HBM is built →Every layer, one at a time.
Open any part for what it does, why it matters, and where it’s headed.
Step through all six stages and count how many do arithmetic. One does. The other five move bytes or heat.
Pull the slider to EXPLODED and rotate the package. The memory stacks sit beside the die, not on top of it: every weight has to cross the baseplate between them.
Stage 2 is the slow one. The cores can multiply far faster than the interposer can deliver weights, which is why the next stop is about memory.