Home / Models / Mixtral 8x7B Instruct / NVIDIA RTX 4070 Ti 12 GB
Can Mixtral 8x7B Instruct run on NVIDIA RTX 4070 Ti 12 GB?
Can it run? Yes, with CPU/RAM offload. Full GPU fit? No.
Can it run?
- Can it run?
- Yes, with CPU/RAM offload. This is not equivalent to full-GPU inference.
- Full GPU fit?
- No
- Model
- Mixtral 8x7B Instruct · 47B
- GPU
- NVIDIA RTX 4070 Ti 12 GB · 12 GB advertised
Memory calculation
Breakdown for Q4 at 8K context, batch size 1. Calculator v1.0.0.
| Model weights | 27.4 GB |
|---|---|
| KV cache | 1.0 GB |
| Runtime reserve | 1.9 GB |
| Safety margin | 1.1 GB |
| Required (estimated) | 31.3 GB |
| GPU usable VRAM | 10.8 GB |
Result: DOES NOT FIT
Quantization table
| Quantization | 2K | 4K | 8K | 16K | 32K |
|---|---|---|---|---|---|
| BF16 | No | No | No | No | No |
| FP16 | No | No | No | No | No |
| Q3 | Offload | Offload | Offload | Offload | Offload |
| Q4 | No | No | No | No | No |
| Q5 | No | No | No | No | No |
| Q6 | No | No | No | No | No |
| Q8 | No | No | No | No | No |
What fits entirely in VRAM?
No calculated full-VRAM fit at the evaluated quantizations and context lengths.
What requires RAM offload?
Weights may run with CPU/RAM offload. That is not equivalent to full-GPU inference. RigForAI does not estimate tokens/sec in this release.
Available graphics cards
These SKUs use this GPU chip. Amazon CTAs appear only for EXACT/HIGH matches. Absence of a price does not change the compatibility result above.
MSI GAMING GeForce RTX 4070 Ti X TRIO 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon EXACT
$1295.00
Check price on Amazon
MSI VENTUS GeForce RTX 4070 Ti 3X 12G OC NVIDIA 12 GB GDDR6X
12 GB · Amazon EXACT
$968.88
Check price on Amazon
GIGABYTE GAMING GeForce RTX 4070 Ti OC 12G NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon EXACT
$1298.00
Check price on Amazon
ASUS ROG -STRIX-RTX4070TI-O12G-GAMING NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon EXACT
$819.99
Check price on Amazon
MSI SUPRIM RTX4070 Ti X NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon EXACT
$1299.00
Check price on Amazon
ASUS TUF Gaming TUF-RTX4070TI-12G-GAMING NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon EXACT
$777.00
Check price on Amazon
GIGABYTE EAGLE GeForce RTX 4070 Ti OC 12G (rev. 2.0) NVIDIA 12 GB GDDR6X
12 GB · Amazon EXACT
$1289.00
Check price on Amazon
GIGABYTE GeForce RTX 4070 Ti WINDFORCE OC 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon EXACT
$892.22
Check price on Amazon
MSI GAMING GEFORCE RTX 4070 TI X SLIM 12G graphics card NVIDIA 12 GB GDDR6X
12 GB · Amazon EXACT
$1195.00
Check price on Amazon
ASUS ROG -STRIX-RTX4070TI-12G-GAMING NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon EXACT
Check price on Amazon
GIGABYTE AERO GeForce RTX 4070 Ti OC 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon unmatched
GIGABYTE AORUS GeForce RTX 4070 Ti ELITE 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon unmatched
GIGABYTE EAGLE GeForce RTX 4070 Ti 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon unmatched
GIGABYTE EAGLE GeForce RTX 4070 Ti OC 12G (rev. 2.0) NVIDIA 12 GB GDDR6X
12 GB · Amazon unmatched
ASUS TUF Gaming TUF-RTX4070TI-O12G-GAMING NVIDIA GeForce RTX 4070 Ti 12 GB GDDR6X
12 GB · Amazon unmatched
GIGABYTE AERO GeForce RTX 4070 Ti OC V2 12G NVIDIA 12 GB GDDR6X
12 GB · Amazon unmatched
Sources
Model: official config mistralai/Mixtral-8x7B-Instruct-v0.1 (main). GPU/product specs: Icecat. Compatibility: RigForAI calculator v1.0.0 estimated VRAM · Q8 at 8K: DOES_NOT_FIT. Amazon: affiliate commerce match, not a spec source.