Mistral AI
Mistral Medium 3.5 (70B)
Balanced model tuned for agentic and coding tasks
Model Summary
Family
Mistral
Version
3.5
Parameters
70B (est.)
Parameter counts for closed models are estimates; vendors rarely publish exact sizes.
VRAM Requirements by Quantization
Memory needed to serve Mistral Medium 3.5 (70B) for inference, including a 20% overhead for activations and KV cache.
| Precision | VRAM needed | Smallest single GPU that fits |
|---|---|---|
| INT4 (4-bit) | 39.12 GB | NVIDIA L40S (48 GB) |
| INT8 (8-bit) | 78.23 GB | NVIDIA H100 SXM5 (80 GB) |
| FP16 (16-bit) | 156.46 GB | NVIDIA B200 SXM (192 GB) |
| FP32 (32-bit) | 312.92 GB | Multi-GPU required |
Recommended GPU Configurations
Cheapest on-demand configurations to serve Mistral Medium 3.5 (70B) at 8-bit (78 GB VRAM).
1x NVIDIA A100 80GB
80 GB total VRAM · Ampere
~$4.50/h
1x AMD Instinct MI300X
192 GB total VRAM · CDNA 3
~$6.00/h
2x NVIDIA L40S
96 GB total VRAM · Ada Lovelace
~$7.00/h
Quick GPU Planning
Use the calculator pre-filled with this exact version to estimate memory, speed, and compute requirements in a few clicks.
Access Pre-filled CalculatorFrequently Asked Questions
How much VRAM do you need to run Mistral Medium 3.5 (70B)?
With an estimated 70B parameters, Mistral Medium 3.5 (70B) needs roughly 78 GB of VRAM in 8-bit (INT8), 39 GB in 4-bit, and 156 GB in FP16, including a 20% overhead for activations and KV cache.
Which GPUs can run Mistral Medium 3.5 (70B)?
At 8-bit quantization, the most cost-effective option is 1x NVIDIA A100 80GB (80 GB combined VRAM, around $4.50/hour on-demand). Higher-end cards like the NVIDIA B200 or AMD MI355X reduce the GPU count needed.
Can Mistral Medium 3.5 (70B) run on a single GPU?
Yes. In 8-bit, a single NVIDIA H100 SXM5 (80 GB) fits the model.
How much does it cost to serve Mistral Medium 3.5 (70B) in the cloud?
Renting 1x A100 80GB costs on the order of $4.50/hour, i.e. about $3,285/month running 24/7. Actual prices vary by provider and commitment; spot and reserved capacity can be significantly cheaper.
Deploy on a GPU cloud
Rent 1x A100 80GB by the hour instead of buying hardware.
