Home / Multi-GPU

llama.cpp · homogeneous 2–4 GPUs · not a PC builder

Can this model run across multiple GPUs?

Phase 8 answers memory feasibility for identical cards under llama.cpp. Default split is layer (pipeline parallel: each GPU owns a slice of layers; KV stays with those layers). This is not 2 × VRAM = pool, and it is not a complete PC build.

2× AMD Radeon RX 6950 XT 16 GB

Does not fit on this configuration

Model
Mixtral 8x22B Instruct · Q4 · 8K
Runtime / split
llama.cpp · layer · calculator 1.0.0
Single-GPU status
DOES_NOT_FIT · required 91.8 GB vs usable 14.4 GB
Per card advertised / usable
16 GB / 14.4 GB
Aggregate advertised
32.0 GB — capacity, not effective model capacity
Nominal aggregate usable
28.8 GB (N × advertised × 0.90)
Effective per-GPU peak
46.3 GB vs usable 14.4 GB
Per-GPU split (approx.)
weights 41.4 GB · KV 0.9 GB · runtime 2.6 GB · safety 1.5 GB
Why
Even a 2K context exceeds per-GPU usable VRAM after partition and per-device reserves.
Topology
Requires space for 2 discrete GPUs. Exact cooler slot width is not in the RigForAI graph. layer split is pipeline parallel and can run over PCIe. KV stays with the layers on each GPU. NVLink is not treated as a 1× aggregate VRAM pool.
Performance
Phase 5 predictions are single-GPU. Multi-GPU tok/s is not published.
GPU-only current cost
$2380.00 = 2 × $1190.00 at the current per-card matched price. This does not mean 2 units are in stock, and it is not a complete build cost.
MSI GAMING Radeon RX 6950 XT X TRIO 16G AMD Radeon RX 6950XT 16 GB GDDR6 · ASIN B09YHXFD19
Amazon
Formula
per_gpu_peak = ceil(weight_bytes/N) + ceil(kv_bytes/N) + runtime_base + safety_base + weight_fractions×ceil(weight_bytes/N). Compare to usable = advertised×0.90. Not (single_gpu_required × N) and not advertised_vram × N as effective capacity.

AMD Radeon RX 6950 XT 16 GB · Single-GPU can-run · Find a single GPU · Build a machine for this GPU setup

Query combinations are not indexed. Source: llama.cpp multi-GPU documentation.