Guides
Buying guides by VRAM size, fixes for common out-of-memory errors, and VRAM for image and speech models.
Best GPU for local LLMs: 12GBThe current graphics cards with 12 GB of VRAM for running local LLMs, with real specs from NVIDIA and AMD and the models that fit in 12 GB.
Best GPU for local LLMs: 16GBThe current graphics cards with 16 GB of VRAM for running local LLMs, with real specs from NVIDIA and AMD and the models that fit in 16 GB.
Best GPU for local LLMs: 24GBThe current graphics cards with 24 GB of VRAM for running local LLMs, with real specs from NVIDIA and AMD and the models that fit in 24 GB.
Best GPU for local LLMs: 32GBThe current graphics cards with 32 GB of VRAM for running local LLMs, with real specs from NVIDIA and AMD and the models that fit in 32 GB.
Local LLMs on a Mac: how much unified memoryThe macOS GPU memory limit and how to raise it, what fits on 16 to 512 GB Macs, Llama 70B on a MacBook, and speed by Apple chip.
How to fix "CUDA out of memory"What "torch.cuda.OutOfMemoryError: CUDA out of memory" means, and the fixes to try in order, from batch size and context length to 4-bit quantization, expandable_segments and offloading.
Run a MoE model bigger than your VRAMKeep attention layers on the GPU and park MoE experts in system RAM: the llama.cpp, LM Studio and Ollama settings, and real reported setups for 12, 16 and 24 GB cards.
llama.cpp "failed to allocate CUDA0 buffer"Fix cudaMalloc out-of-memory when loading a GGUF: GPU layers, context, flash attention, batch size.
Ollama "model requires more system memory"Parallel slots, context size, and the Docker/LXC page-cache bug.
vLLM "No available memory for the cache blocks"gpu_memory_utilization, max-model-len and max-num-seqs explained.
Whisper VRAM by model sizeOpenAI's official figures: ~1 GB (tiny) to ~10 GB (large).
FLUX.1 VRAM~50 GB to load everything, and the documented ways to run it under 16 GB.
The buying guides contain Amazon affiliate links: we may earn a commission at no cost to you. Disclosure.