GPU Specifications / NVIDIA
NVIDIA H100 SXM5
Hopper datacenter GPU with 80 GB of HBM3 memory, 3350 GB/s of bandwidth and up to 1,979 TFLOPS of FP16 tensor compute.
Key Specifications
Memory
80 GB HBM3
Memory Bandwidth
3,350 GB/s
TDP
700 W
Architecture
Hopper
Interconnect
NVLink 4.0 · 900 GB/s
Est. On-demand Price
~$8.00/h
Hourly rates are indicative on-demand estimates; actual pricing varies by provider and commitment.
Compute Performance
| Precision | Peak throughput |
|---|---|
| FP64 (double precision) | 34 TFLOPS |
| FP32 (single precision) | 67 TFLOPS |
| FP24 | 134 TFLOPS |
| FP16 (tensor) | 1,979 TFLOPS |
| INT8 (tensor) | 3,958 TFLOPS |
| INT4 (tensor) | 7,916 TFLOPS |
Tensor figures use the vendor's peak numbers (with structured sparsity where supported).
System Requirements
Recommended CPU
AMD EPYC 9454 or Intel Xeon Platinum 8480+
Max VRAM per node (8 GPUs)
640 GB
System RAM (min / recommended)
384 / 768 GB
Minimum PSU
1600 W
LLMs on the H100 SXM5
Number of H100 SXM5 GPUs needed to serve popular models at 8-bit quantization (including 20% overhead for activations and KV cache).
| Model | Params | VRAM (8-bit) | GPUs needed |
|---|---|---|---|
| GPT-5.6 Sol | 2400B | 2682 GB | 34x H100 SXM5 |
| GPT-5 Flagship | 2100B | 2347 GB | 30x H100 SXM5 |
| GPT-5.6 Luna | 1400B | 1565 GB | 20x H100 SXM5 |
| Kimi K3 (1.2T) | 1200B | 1341 GB | 17x H100 SXM5 |
| Kimi K2.6 (1T) | 1000B | 1118 GB | 14x H100 SXM5 |
| GPT-5.6 Terra | 800B | 894 GB | 12x H100 SXM5 |
| DeepSeek V4 Pro (671B) | 671B | 750 GB | 10x H100 SXM5 |
| Llama 4 Behemoth (500B) | 500B | 559 GB | 7x H100 SXM5 |
| Claude 5 Fable (480B) | 480B | 536 GB | 7x H100 SXM5 |
| GLM 5.2 (400B) | 400B | 447 GB | 6x H100 SXM5 |
| Grok 4.5 | 350B | 391 GB | 5x H100 SXM5 |
| Claude 4.8 Opus (300B) | 300B | 335 GB | 5x H100 SXM5 |
| Muse Spark 1.1 | 300B | 335 GB | 5x H100 SXM5 |
| Grok 4 | 270B | 302 GB | 4x H100 SXM5 |
| Gemini 3.1 Pro | 250B | 279 GB | 4x H100 SXM5 |
| Qwen 3.7 Max (235B) | 235B | 263 GB | 4x H100 SXM5 |
| Mistral Large 3 (200B) | 200B | 224 GB | 3x H100 SXM5 |
| Grok 3 Mini | 190B | 212 GB | 3x H100 SXM5 |
| Claude 5 Sonnet (175B) | 175B | 196 GB | 3x H100 SXM5 |
| Gemini 3.5 Flash | 150B | 168 GB | 3x H100 SXM5 |
| Gemini 2.5 Flash | 140B | 156 GB | 2x H100 SXM5 |
| Llama 4 Maverick (128B) | 128B | 143 GB | 2x H100 SXM5 |
| DeepSeek V4 Flash (120B) | 120B | 134 GB | 2x H100 SXM5 |
| Qwen 3.6 Plus (110B) | 110B | 123 GB | 2x H100 SXM5 |
| Nova Premier (80B) | 80B | 89 GB | 2x H100 SXM5 |
| Qwen 3 Coder-Next (80B) | 80B | 89 GB | 2x H100 SXM5 |
| Claude 4.5 Haiku (70B) | 70B | 78 GB | 1x H100 SXM5 |
| Llama 3.3 Instruct (70B) | 70B | 78 GB | 1x H100 SXM5 |
| Mistral Medium 3.5 (70B) | 70B | 78 GB | 1x H100 SXM5 |
| Yi 1.5 (40B) | 40B | 45 GB | 1x H100 SXM5 |
| Nova Core (34B) | 34B | 38 GB | 1x H100 SXM5 |
| DeepSeek V3.1 (32B) | 32B | 36 GB | 1x H100 SXM5 |
| Gemma 3 (27B) | 27B | 30 GB | 1x H100 SXM5 |
| Mistral Small 4 (24B) | 24B | 27 GB | 1x H100 SXM5 |
| Yi 1.5 (15B) | 15B | 17 GB | 1x H100 SXM5 |
| Phi 4 (14B) | 14B | 16 GB | 1x H100 SXM5 |
| Nova Lite (12B) | 12B | 13 GB | 1x H100 SXM5 |
| Llama 3.2 Instruct (11B) | 11B | 12 GB | 1x H100 SXM5 |
| Gemma 3 (9B) | 9B | 10 GB | 1x H100 SXM5 |
| Yi 1.5 Lite (9B) | 9B | 10 GB | 1x H100 SXM5 |
| Phi 4 Mini (7B) | 7B | 8 GB | 1x H100 SXM5 |
| Phi 3.5 (3.8B) | 3.8B | 4 GB | 1x H100 SXM5 |
Frequently Asked Questions
How much VRAM does the NVIDIA H100 SXM5 have?
The NVIDIA H100 SXM5 has 80 GB of HBM3 memory with 3350 GB/s of memory bandwidth.
Which LLMs can run on a single H100 SXM5?
At 8-bit quantization, a single H100 SXM5 (80 GB) can serve models up to roughly 70B parameters, such as Claude 4.5 Haiku (70B). Larger models require multiple GPUs or more aggressive quantization.
How much does it cost to rent a NVIDIA H100 SXM5?
On-demand cloud pricing for the H100 SXM5 is around $8.00/hour, i.e. about $5,840/month running 24/7. Actual prices vary by provider, region, and commitment.
What are the power and system requirements of the NVIDIA H100 SXM5?
The H100 SXM5 has a TDP of 700W. A power supply of at least 1600W per GPU is recommended. Recommended host CPUs: AMD EPYC 9454 or Intel Xeon Platinum 8480+.
Deploy on a GPU cloud
Rent the NVIDIA H100 SXM5 by the hour instead of buying hardware.
