GPU Specifications / NVIDIA
NVIDIA A40
Ampere datacenter GPU with 48 GB of GDDR6 memory, 696 GB/s of bandwidth and up to 299.4 TFLOPS of FP16 tensor compute.
Key Specifications
Memory
48 GB GDDR6
Memory Bandwidth
696 GB/s
TDP
300 W
Architecture
Ampere
Interconnect
NVLink 3.0 · 112.5 GB/s
Est. On-demand Price
~$1.80/h
Hourly rates are indicative on-demand estimates; actual pricing varies by provider and commitment.
Compute Performance
| Precision | Peak throughput |
|---|---|
| FP64 (double precision) | 0.6 TFLOPS |
| FP32 (single precision) | 37.4 TFLOPS |
| FP24 | 74.8 TFLOPS |
| FP16 (tensor) | 299.4 TFLOPS |
| INT8 (tensor) | 598.7 TFLOPS |
| INT4 (tensor) | 1,197.4 TFLOPS |
Tensor figures use the vendor's peak numbers (with structured sparsity where supported).
System Requirements
Recommended CPU
AMD EPYC 7543 or Intel Xeon Gold 6342
Max VRAM per node (8 GPUs)
384 GB
System RAM (min / recommended)
256 / 512 GB
Minimum PSU
850 W
LLMs on the A40
Number of A40 GPUs needed to serve popular models at 8-bit quantization (including 20% overhead for activations and KV cache).
Frequently Asked Questions
How much VRAM does the NVIDIA A40 have?
The NVIDIA A40 has 48 GB of GDDR6 memory with 696 GB/s of memory bandwidth.
Which LLMs can run on a single A40?
At 8-bit quantization, a single A40 (48 GB) can serve models up to roughly 40B parameters, such as Yi 1.5 (40B). Larger models require multiple GPUs or more aggressive quantization.
How much does it cost to rent a NVIDIA A40?
On-demand cloud pricing for the A40 is around $1.80/hour, i.e. about $1,314/month running 24/7. Actual prices vary by provider, region, and commitment.
What are the power and system requirements of the NVIDIA A40?
The A40 has a TDP of 300W. A power supply of at least 850W per GPU is recommended. Recommended host CPUs: AMD EPYC 7543 or Intel Xeon Gold 6342.
Deploy on a GPU cloud
Rent the NVIDIA A40 by the hour instead of buying hardware.
