Home / Models / Hermes 3 Llama 3.1 70B / NVIDIA RTX 3090 24 GB
Can Hermes 3 Llama 3.1 70B run on NVIDIA RTX 3090 24 GB?
Can it run? Yes, with CPU/RAM offload. Full GPU fit? No.
Can it run?
- Can it run?
- Yes, with CPU/RAM offload. This is not equivalent to full-GPU inference.
- Full GPU fit?
- No
- Model
- Hermes 3 Llama 3.1 70B · 71B
- GPU
- NVIDIA RTX 3090 24 GB · 24 GB advertised
Memory calculation
Breakdown for Q4 at 8K context, batch size 1. Calculator v1.0.0.
| Model weights | 41.4 GB |
|---|---|
| KV cache | 2.5 GB |
| Runtime reserve | 2.6 GB |
| Safety margin | 1.5 GB |
| Required (estimated) | 48.0 GB |
| GPU usable VRAM | 21.6 GB |
Result: CPU RAM OFFLOAD REQUIRED
Quantization table
| Quantization | 2K | 4K | 8K | 16K | 32K | 64K | 128K |
|---|---|---|---|---|---|---|---|
| BF16 | No | No | No | No | No | No | No |
| FP16 | No | No | No | No | No | No | No |
| Q3 | Offload | Offload | Offload | Offload | Offload | Offload | Offload |
| Q4 | Offload | Offload | Offload | Offload | Offload | Offload | Offload |
| Q5 | Offload | Offload | Offload | Offload | Offload | Offload | Offload |
| Q6 | No | No | No | No | No | No | No |
| Q8 | No | No | No | No | No | No | No |
What fits entirely in VRAM?
No calculated full-VRAM fit at the evaluated quantizations and context lengths.
What requires RAM offload?
Weights may run with CPU/RAM offload. That is not equivalent to full-GPU inference. RigForAI does not estimate tokens/sec in this release.
Available graphics cards
These SKUs use this GPU chip. Amazon CTAs appear only for EXACT/HIGH matches. Absence of a price does not change the compatibility result above.
MSI SUPRIM GeForce RTX 3090 X 24G NVIDIA 24 GB GDDR6X
24 GB · Amazon EXACT
$1489.97
Check price on Amazon
GIGABYTE GV-N3090TURBO-24GD graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1995.00
Check price on Amazon
MSI GAMING RTX 3090 X TRIO 24G graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$2069.00
Check price on Amazon
GIGABYTE GAMING GV-N3090GAMING OC-24GD graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1989.00
Check price on Amazon
MSI VENTUS GeForce RTX 3090 3X 24G OC NVIDIA 24 GB GDDR6X
24 GB · Amazon EXACT
$1995.00
Check price on Amazon
GIGABYTE GAMING RTX 3090 BOX NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1795.00
Check price on Amazon
GIGABYTE GeForce RTX 3090 VISION OC 24G NVIDIA 24 GB GDDR6X
24 GB · Amazon EXACT
$1479.99
Check price on Amazon
GIGABYTE AORUS GV-N3090AORUS M-24GD graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1899.00
Check price on Amazon
GIGABYTE AORUS GeForce RTX 3090 XTREME WATERFORCE 24G NVIDIA 24 GB GDDR6X
24 GB · Amazon EXACT
$2389.00
Check price on Amazon
GIGABYTE EAGLE GV-N3090EAGLE OC-24GD graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1347.22
Check price on Amazon
ASUS ROG -STRIX-RTX3090-O24G-GAMING graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1629.99
Check price on Amazon
GIGABYTE AORUS GV-N3090AORUS X-24GD graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
$1462.22
Check price on Amazon
ASUS TUF Gaming TUF-RTX3090-O24G-GAMING graphics card NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon EXACT
Check price on Amazon
ASUS TUF Gaming TUF-RTX3090-O24G-GAMING NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon unmatched
ASUS ROG -STRIX-RTX3090-O24G-WHITE NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon unmatched
ASUS ROG -STRIX-RTX3090-24G-GAMING NVIDIA GeForce RTX 3090 24 GB GDDR6X
24 GB · Amazon unmatched
Sources
Model: official config NousResearch/Hermes-3-Llama-3.1-70B (main). GPU/product specs: Icecat. Compatibility: RigForAI calculator v1.0.0 estimated VRAM · Q8 at 8K: DOES_NOT_FIT. Amazon: affiliate commerce match, not a spec source.