Hardware for Local LLMs¶

Community hardware data tracked from Hugging Face Hardware.

GPU Landscape¶

:material-nvidia: NVIDIA · 45%

RTX 30xx 27% · RTX 40xx 27% · RTX 50xx 19% · Datacenter 13%
Apple Silicon · 35%

M1–M4 series with unified memory — the local LLM sweet spot.
:material-gpu: AMD · 5%

RX 7000 33% · RX 6000/5000 30% · RX 9000 19% · Instinct 5%

Model Size	Q4_K_M	Q8_0	FP16
7B–8B	5 GB	8 GB	16 GB
13B–14B	8 GB	14 GB	26 GB
34B	20 GB	34 GB	68 GB
70B–72B	40 GB	70 GB	140 GB
405B MoE	230 GB	—	—

8 GB VRAM

7B–8B models (Llama 4 8B, Mistral 7B, Gemma 3)
12–16 GB VRAM

7B–14B comfortably, 34B with CPU offloading
24 GB VRAM

34B easily, 70B at Q4. The enthusiast sweet spot.
32 GB+ VRAM

70B comfortably, small MoE models. Near-datacenter perf.
Apple Silicon 16 GB+

Excellent for 7B–14B via MLX. 20–30% faster than llama.cpp.
Apple Silicon 64 GB+

Runs 70B models using unified memory. No GPU offloading needed.