Best GPU for local LLMs: 24GB
24 GB is where 26B to 31B models start to fit at 4-bit, and 14B models move up to 8-bit. The model list below comes from the same data as our 24 GB tier page. Specs are copied from NVIDIA's and AMD's own product pages.
NVIDIA
GeForce RTX 4090
- Memory: 24 GB GDDR6X
- Memory bus: 384-bit
- Power: 450 W total graphics power
24 GB on a 384-bit bus. Needs an 850 W power supply per NVIDIA.
Check price on Amazon Affiliate link
Specs: NVIDIA product page, checked 2026-10-31.
NVIDIA
GeForce RTX 3090
- Memory: 24 GB GDDR6X
- Memory bus: 384-bit
- Power: 350 W graphics card power
The same 24 GB and 384-bit bus at lower power. Discontinued, so listings may be used or renewed.
Check price on Amazon Affiliate link
Specs: NVIDIA product page, checked 2026-10-31.
Amazon links are affiliate links: we may earn a commission at no cost to you. We don't show Amazon prices because they change often. Disclosure.
What fits in 24 GB
15 of the 30 language models in our calculator fit comfortably in 24 GB with an 8K-token conversation:
| Model | Best precision | Memory |
|---|---|---|
| Gemma 4 31B | 4-bit (Q4_K_M) | 19.6 GB |
| Qwen3.8 27B | 4-bit (Q4_K_M) | 18.0 GB |
| Gemma 4 26B A4B | 4-bit (Q4_K_M) | 17.0 GB |
| LFM2.5-24B-A2B | 4-bit (Q4_K_M) | 14.7 GB |
| gpt-oss-20b | 16-bit (FP16/BF16) | 14.0 GB |
| Qwen2.5 14B Instruct | 8-bit (Q8_0) | 17.3 GB |
| Qwen3 14B | 8-bit (Q8_0) | 17.0 GB |
| DeepSeek-R1-0528-Qwen3-8B | 16-bit (FP16/BF16) | 17.6 GB |
| Llama 3.1 8B Instruct | 16-bit (FP16/BF16) | 17.1 GB |
| Nemotron 3.5 Bolt E4B | 16-bit (FP16/BF16) | 15.1 GB |
| Qwen2.5 7B Instruct | 16-bit (FP16/BF16) | 15.7 GB |
| Mistral 7B Instruct v0.3 | 16-bit (FP16/BF16) | 15.6 GB |
| Spark-X2.6 4B | 16-bit (FP16/BF16) | 8.5 GB |
| Llama 3.2 3B Instruct | 16-bit (FP16/BF16) | 7.4 GB |
| LFM3 1.7B | 16-bit (FP16/BF16) | 3.6 GB |
Same data and rule as What can I run with 24 GB?: real Hugging Face download sizes plus FP16 KV cache, under 90% of 24 GB. Data checked 2026-10-30.
How to choose
- VRAM decides what fits. Every card here has 24 GB, so all of them hold the same models.
- A wider memory bus usually means faster tokens. Generating text mostly reads the weights from memory, so at the same VRAM a 256-bit card is usually faster than a 128-bit one. See the measured speeds we've collected.
- Check power and your app. Make sure your power supply can handle the card. NVIDIA cards work with nearly every local AI app. On AMD, check your app's support first.
- Renting may be cheaper. For a few hours a month, compare cloud GPU prices before you buy. In India, see renting a 24GB GPU in India (rupee prices), A100 and H100 rental in India and Runpod alternatives that accept UPI.