VRAMNeeded Calculator

Fix vLLM "No available memory for the cache blocks"

vLLM stops at startup with:

ValueError: No available memory for the cache blocks. Try increasing `gpu_memory_utilization` when initializing the engine.

(Quoted from vLLM issue #17979 and discussion #15842.)

What it means

vLLM reserves a fixed share of the GPU: total VRAM × gpu_memory_utilization. From that budget it takes the model weights and the activation peak measured during a test run. Whatever is left becomes the KV cache. If nothing is left, you get this error. vLLM's startup log shows the split, for example: model weights take 2.32GiB; non_torch_memory takes 0.15GiB; PyTorch activation peak memory takes 4.69GiB; the rest of the memory reserved for KV Cache is 20.78GiB (PR #19312).

Fixes, in order

1. Raise --gpu-memory-utilization (if the GPU is yours alone)

The error message says this directly. In issue #17431 a vLLM maintainer replied: "Your GPU memory utilization is too low, can you increase it?" Don't set it above what's actually free.

2. Lower --max-model-len

Some models default to a very long context. In issue #18124 (Qwen2.5-VL-3B on an A100) a maintainer suggested --max-model-len 8192, "because Qwen2.5-VL has a very large context length by default", and the user confirmed it worked with --max-model-len 8192 and --max-num-seqs 64.

3. Lower --max-num-seqs

In discussion #15842 a reply notes that the default --max-num-seqs went from 256 in the V0 engine to 1024 in V1, which uses more memory at startup.

4. Use a smaller or quantized checkpoint

An AWQ, GPTQ or FP8 checkpoint leaves more room for the KV cache, and --kv-cache-dtype fp8 shrinks the cache itself. Check sizes with the VRAM calculator.

5. Running two vLLM servers on one GPU?

The V1 engine used to count memory used by other processes against the new instance, so a second server failed even with room to spare (#17979). PR #19312 fixed this, so update vLLM. Give each instance a utilization share that fits what is left.

Still doesn't fit?

Use tensor parallelism across GPUs, or rent a bigger card. Our GPU prices page compares hourly rates for 48 and 80 GB GPUs (we may earn from some links, disclosure). See also Fix CUDA out of memory.

Sources

Last checked: