GPU Specifications / NVIDIA
NVIDIA A100 80GB
Ampere datacenter GPU with 80 GB of HBM2e memory, 2039 GB/s of bandwidth and up to 624 TFLOPS of FP16 tensor compute.
Key Specifications
Memory
80 GB HBM2e
Memory Bandwidth
2,039 GB/s
TDP
400 W
Architecture
Ampere
Interconnect
NVLink 3.0 · 600 GB/s
Est. On-demand Price
~$4.50/h
Hourly rates are indicative on-demand estimates; actual pricing varies by provider and commitment.
Compute Performance
| Precision | Peak throughput |
|---|---|
| FP64 (double precision) | 9.7 TFLOPS |
| FP32 (single precision) | 19.5 TFLOPS |
| FP24 | 39 TFLOPS |
| FP16 (tensor) | 624 TFLOPS |
| INT8 (tensor) | 1,248 TFLOPS |
| INT4 (tensor) | 2,496 TFLOPS |
Tensor figures use the vendor's peak numbers (with structured sparsity where supported).
System Requirements
Recommended CPU
AMD EPYC 7763 or Intel Xeon Platinum 8380
Max VRAM per node (8 GPUs)
640 GB
System RAM (min / recommended)
256 / 512 GB
Minimum PSU
1000 W
LLMs on the A100 80GB
Number of A100 80GB GPUs needed to serve popular models at 8-bit quantization (including 20% overhead for activations and KV cache).
| Model | Params | VRAM (8-bit) | GPUs needed |
|---|---|---|---|
| GPT-5.6 Sol | 2400B | 2682 GB | 34x A100 80GB |
| GPT-5 Flagship | 2100B | 2347 GB | 30x A100 80GB |
| GPT-5.6 Luna | 1400B | 1565 GB | 20x A100 80GB |
| Kimi K3 (1.2T) | 1200B | 1341 GB | 17x A100 80GB |
| Kimi K2.6 (1T) | 1000B | 1118 GB | 14x A100 80GB |
| GPT-5.6 Terra | 800B | 894 GB | 12x A100 80GB |
| DeepSeek V4 Pro (671B) | 671B | 750 GB | 10x A100 80GB |
| Llama 4 Behemoth (500B) | 500B | 559 GB | 7x A100 80GB |
| Claude 5 Fable (480B) | 480B | 536 GB | 7x A100 80GB |
| GLM 5.2 (400B) | 400B | 447 GB | 6x A100 80GB |
| Grok 4.5 | 350B | 391 GB | 5x A100 80GB |
| Claude 4.8 Opus (300B) | 300B | 335 GB | 5x A100 80GB |
| Muse Spark 1.1 | 300B | 335 GB | 5x A100 80GB |
| Grok 4 | 270B | 302 GB | 4x A100 80GB |
| Gemini 3.1 Pro | 250B | 279 GB | 4x A100 80GB |
| Qwen 3.7 Max (235B) | 235B | 263 GB | 4x A100 80GB |
| Mistral Large 3 (200B) | 200B | 224 GB | 3x A100 80GB |
| Grok 3 Mini | 190B | 212 GB | 3x A100 80GB |
| Claude 5 Sonnet (175B) | 175B | 196 GB | 3x A100 80GB |
| Gemini 3.5 Flash | 150B | 168 GB | 3x A100 80GB |
| Gemini 2.5 Flash | 140B | 156 GB | 2x A100 80GB |
| Llama 4 Maverick (128B) | 128B | 143 GB | 2x A100 80GB |
| DeepSeek V4 Flash (120B) | 120B | 134 GB | 2x A100 80GB |
| Qwen 3.6 Plus (110B) | 110B | 123 GB | 2x A100 80GB |
| Nova Premier (80B) | 80B | 89 GB | 2x A100 80GB |
| Qwen 3 Coder-Next (80B) | 80B | 89 GB | 2x A100 80GB |
| Claude 4.5 Haiku (70B) | 70B | 78 GB | 1x A100 80GB |
| Llama 3.3 Instruct (70B) | 70B | 78 GB | 1x A100 80GB |
| Mistral Medium 3.5 (70B) | 70B | 78 GB | 1x A100 80GB |
| Yi 1.5 (40B) | 40B | 45 GB | 1x A100 80GB |
| Nova Core (34B) | 34B | 38 GB | 1x A100 80GB |
| DeepSeek V3.1 (32B) | 32B | 36 GB | 1x A100 80GB |
| Gemma 3 (27B) | 27B | 30 GB | 1x A100 80GB |
| Mistral Small 4 (24B) | 24B | 27 GB | 1x A100 80GB |
| Yi 1.5 (15B) | 15B | 17 GB | 1x A100 80GB |
| Phi 4 (14B) | 14B | 16 GB | 1x A100 80GB |
| Nova Lite (12B) | 12B | 13 GB | 1x A100 80GB |
| Llama 3.2 Instruct (11B) | 11B | 12 GB | 1x A100 80GB |
| Gemma 3 (9B) | 9B | 10 GB | 1x A100 80GB |
| Yi 1.5 Lite (9B) | 9B | 10 GB | 1x A100 80GB |
| Phi 4 Mini (7B) | 7B | 8 GB | 1x A100 80GB |
| Phi 3.5 (3.8B) | 3.8B | 4 GB | 1x A100 80GB |
Frequently Asked Questions
How much VRAM does the NVIDIA A100 80GB have?
The NVIDIA A100 80GB has 80 GB of HBM2e memory with 2039 GB/s of memory bandwidth.
Which LLMs can run on a single A100 80GB?
At 8-bit quantization, a single A100 80GB (80 GB) can serve models up to roughly 70B parameters, such as Claude 4.5 Haiku (70B). Larger models require multiple GPUs or more aggressive quantization.
How much does it cost to rent a NVIDIA A100 80GB?
On-demand cloud pricing for the A100 80GB is around $4.50/hour, i.e. about $3,285/month running 24/7. Actual prices vary by provider, region, and commitment.
What are the power and system requirements of the NVIDIA A100 80GB?
The A100 80GB has a TDP of 400W. A power supply of at least 1000W per GPU is recommended. Recommended host CPUs: AMD EPYC 7763 or Intel Xeon Platinum 8380.
Deploy on a GPU cloud
Rent the NVIDIA A100 80GB by the hour instead of buying hardware.
