NVIDIA H100
NVIDIA’s data center GPU from the Hopper generation (2022), and the workhorse of cloud inference for years.
- Memory size: 80 GB HBM3 (SXM version)
- Memory bandwidth: about 3,350 GB/s, roughly twice the RTX 5090
- Compute capability:
sm_90
HBM (“high bandwidth memory”) is stacked right next to the GPU chip, which is how data center GPUs get their much higher bandwidth than consumer cards that use GDDR.