Home / Models / Code Llama 34B Instruct / NVIDIA RTX 5070 Ti 16 GB
Can Code Llama 34B Instruct run on NVIDIA RTX 5070 Ti 16 GB?
Can it run? Yes, with CPU/RAM offload. Full GPU fit? No.
Can it run?
- Can it run?
- Yes, with CPU/RAM offload. This is not equivalent to full-GPU inference.
- Full GPU fit?
- No
- Model
- Code Llama 34B Instruct · 34B
- GPU
- NVIDIA RTX 5070 Ti 16 GB · 16 GB advertised
Memory calculation
Breakdown for Q4 at 8K context, batch size 1. Calculator v1.0.0.
| Model weights | 19.8 GB |
|---|---|
| KV cache | 1.5 GB |
| Runtime reserve | 1.5 GB |
| Safety margin | 0.8 GB |
| Required (estimated) | 23.6 GB |
| GPU usable VRAM | 14.4 GB |
Result: CPU RAM OFFLOAD REQUIRED
Quantization table
| Quantization | 2K | 4K | 8K | 16K |
|---|---|---|---|---|
| BF16 | No | No | No | No |
| FP16 | No | No | No | No |
| Q3 | Offload | Offload | Offload | Offload |
| Q4 | Offload | Offload | Offload | Offload |
| Q5 | Offload | Offload | Offload | Offload |
| Q6 | Offload | Offload | Offload | Offload |
| Q8 | No | No | No | No |
What fits entirely in VRAM?
No calculated full-VRAM fit at the evaluated quantizations and context lengths.
What requires RAM offload?
Weights may run with CPU/RAM offload. That is not equivalent to full-GPU inference. RigForAI does not estimate tokens/sec in this release.
Available graphics cards
These SKUs use this GPU chip. Amazon CTAs appear only for EXACT/HIGH matches. Absence of a price does not change the compatibility result above.
ASUS TUF Gaming TUF-RTX5070TI-O16G-GAMING NVIDIA GeForce RTX 5070 Ti 16 GB GDDR7
16 GB · Amazon EXACT
$1099.99
Check price on Amazon
MSI GeForce RTX 5070 Ti 16G VENTUS 3X OC NVIDIA 16 GB GDDR7
16 GB · Amazon EXACT
$1179.99
Check price on Amazon
GIGABYTE GeForce RTX 5070 Ti GAMING OC 16G Graphics Card
16 GB · Amazon EXACT
$1088.24
Check price on Amazon
MSI GAMING GeForce RTX 5070 Ti 16G TRIO OC NVIDIA 16 GB GDDR7
16 GB · Amazon EXACT
$1049.99
Check price on Amazon
ASUS ROG -STRIX-RTX5070TI-O16G-GAMING NVIDIA GeForce RTX 5070 Ti 16 GB GDDR7
16 GB · Amazon EXACT
$1547.00
Check price on Amazon
GIGABYTE GeForce RTX 5070 Ti WINDFORCE SFF 16G Graphics Card - 16GB GDDR7, 256bit, PCI-E 5.0, 2452 MHz Core Clock, 3 x DP 2.1b, 1 x HDMI 2.1b, NVIDIA DLSS 4, GV-N507TWF3-16GD
16 GB · Amazon EXACT
$1129.99
Check price on Amazon
ASUS Prime -RTX5070TI-O16G NVIDIA GeForce RTX 5070 Ti 16 GB GDDR7
16 GB · Amazon EXACT
Check price on Amazon
Sources
Model: official config codellama/CodeLlama-34b-Instruct-hf (main). GPU/product specs: Icecat. Compatibility: RigForAI calculator v1.0.0 estimated VRAM · Q8 at 8K: DOES_NOT_FIT. Amazon: affiliate commerce match, not a spec source.