Home / Multi-GPU
llama.cpp · homogeneous 2–4 GPUs · not a PC builder
Can this model run across multiple GPUs?
Phase 8 answers memory feasibility for identical cards under llama.cpp. Default split is layer (pipeline parallel: each GPU owns a slice of layers; KV stays with those layers). This is not 2 × VRAM = pool, and it is not a complete PC build.
What is modeled
- Runtime: llama.cpp CUDA (same family as Phase 5). Calculator 1.0.0.
- Counts: 2, 3, or 4 of the same canonical GPU.
- Split modes from official llama.cpp multi-GPU docs:
layer(default) and experimentaltensor. - Heterogeneous mixes (4090 + 3090) are not modeled.
- Multi-GPU tok/s is not published. Phase 5 numbers stay single-GPU.
Query combinations are not indexed. Source: llama.cpp multi-GPU documentation.