Home / Models / Falcon 7B Instruct
Falcon 7B Instruct
7.2B parameters · falcon
Estimated VRAM requirement
Weight memory uses parameter count × bits per weight, plus a quantization overhead factor. KV cache uses standard GQA formulas. Values are calculated estimates, not measured allocations.
| Quantization | Estimated weights | 2K total | 4K total | 8K total | 16K total | 32K total | 64K total | 128K total |
|---|---|---|---|---|---|---|---|---|
| FP16 | 13.7 GB | 16.7 GB | 17.8 GB | 20.0 GB | 24.4 GB | 33.3 GB | 51.1 GB | 86.6 GB |
| Q8 | 7.7 GB | 10.2 GB | 11.3 GB | 13.5 GB | 18.0 GB | 26.8 GB | 44.6 GB | 80.1 GB |
| Q6 | 6.0 GB | 8.3 GB | 9.5 GB | 11.7 GB | 16.1 GB | 25.0 GB | 42.7 GB | 78.2 GB |
| Q5 | 5.1 GB | 7.4 GB | 8.5 GB | 10.7 GB | 15.1 GB | 24.0 GB | 41.7 GB | 77.2 GB |
| Q4 | 4.2 GB | 6.4 GB | 7.5 GB | 9.8 GB | 14.2 GB | 23.1 GB | 40.8 GB | 76.3 GB |
| Q3 | 3.4 GB | 5.5 GB | 6.6 GB | 8.8 GB | 13.3 GB | 22.2 GB | 39.9 GB | 75.4 GB |
Find a GPU for this model
Run at the quantization and context selected below. Defaults are Q4 and 8K when you do not change the controls. Results are full-VRAM fits only — not a speed ranking.
- Run at
- Q4 · 8K context
- Required VRAM
- 9.8 GB
Lowest-cost full-VRAM option
Sorted by lowest current Amazon price among GPUs that fit entirely in VRAM. This is not a performance ranking.
NVIDIA Tesla K40 12 GB
- GPU VRAM
- 12 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 10.8 GB
- VRAM headroom
- 1.0 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 2 mapped · 1 Amazon-matched
- Amazon
- $262.48 current Amazon price
NVIDIA Quadro P5000 16 GB
- GPU VRAM
- 16 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 14.4 GB
- VRAM headroom
- 4.6 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 5 mapped · 1 Amazon-matched
- Amazon
- $330.99 current Amazon price
NVIDIA RTX 3060 12 GB
- GPU VRAM
- 12 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 10.8 GB
- VRAM headroom
- 1.0 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 28 mapped · 7 Amazon-matched
- Amazon
- $369.22 current Amazon price
More VRAM headroom
Sorted by leftover usable VRAM after the calculated requirement. Headroom is not tokens/sec.
NVIDIA RTX 6000 48 GB
- GPU VRAM
- 48 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 43.2 GB
- VRAM headroom
- 33.4 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 1 mapped · 0 Amazon-matched
- Amazon
- Check price on Amazon
NVIDIA RTX 8000 48 GB
- GPU VRAM
- 48 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 43.2 GB
- VRAM headroom
- 33.4 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 1 mapped · 0 Amazon-matched
- Amazon
- Check availability
NVIDIA RTX A6000 48 GB
- GPU VRAM
- 48 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 43.2 GB
- VRAM headroom
- 33.4 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 2 mapped · 0 Amazon-matched
- Amazon
- Check availability
Other compatible GPUs
NVIDIA GTX 1080 Ti 11 GB
- GPU VRAM
- 11 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 9.9 GB
- VRAM headroom
- 0.1 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 16 mapped · 0 Amazon-matched
- Amazon
- Check price on Amazon
NVIDIA RTX 2080 Ti 11 GB
- GPU VRAM
- 11 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 9.9 GB
- VRAM headroom
- 0.1 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 14 mapped · 0 Amazon-matched
- Amazon
- Check price on Amazon
AMD Radeon RX 6750 XT 12 GB
- GPU VRAM
- 12 GB advertised
- Required VRAM
- 9.8 GB
- Usable GPU VRAM
- 10.8 GB
- VRAM headroom
- 1.0 GB — leftover usable memory after the calculated requirement, not a speed ranking
- Compatible
- Yes — fits entirely in VRAM
- Product SKUs
- 6 mapped · 0 Amazon-matched
- Amazon
- Check price on Amazon
Compatible GPUs
Status is a calculated memory fit against usable VRAM (advertised × 0.9). Open a GPU to see product SKUs and Amazon listings.
Sources
Model specifications: official repository tiiuae/falcon-7b-instruct · revision main. Compatibility: RigForAI calculator v1.0.0 (estimated VRAM). GPU products: Icecat. Amazon is a separate affiliate match and is not used as a spec source.