VRAMNeeded Calculator

What can an RTX 4090 run?

The GeForce RTX 4090 has 24 GB of GDDR6X memory, the same amount as the RTX 3090.[1] So the list of models that fit is the same as on a 3090: the table below checks every language model in our calculator at three quantizations with an 8K-token context, using the same file sizes and fit rule as the rest of the site. What the 4090 changes is speed, covered further down.

Best model for the RTX 4090 24GB: Gemma 4 31B. It is the largest model in our calculator (31.3B parameters) that fits comfortably at 4-bit: Q4_K_M plus an 8K-token FP16 KV cache needs 19.6 GB, under 90% of 24 GB. The highest precision it fits comfortably at is 4-bit (Q4_K_M). In total, 15 of the 30 language models we track fit comfortably at 4-bit on this card.

Model4-bit (Q4_K_M)8-bit (Q8_0)16-bit (FP16/BF16)
Gemma 4 31B*Google19.6 GB · Fits32.6 GB · Too big61.4 GB · Too big
Qwen3.8 27BAlibaba Qwen18.0 GB · Fits29.7 GB · Too big55.2 GB · Too big
Gemma 4 26B A4B*Google17.0 GB · Fits26.9 GB · Too big50.5 GB · Too big
LFM2.5-24B-A2BLiquid AI14.7 GB · Fits25.7 GB · Too big48.1 GB · Too big
gpt-oss-20bOpenAI11.9 GB · Fits12.3 GB · Fits14.0 GB · Fits
Qwen2.5 14B InstructAlibaba Qwen10.6 GB · Fits17.3 GB · Fits31.2 GB · Too big
Qwen3 14BAlibaba Qwen10.3 GB · Fits17.0 GB · Fits30.9 GB · Too big
DeepSeek-R1-0528-Qwen3-8BDeepSeek6.2 GB · Fits9.9 GB · Fits17.6 GB · Fits
Llama 3.1 8B InstructMeta6.0 GB · Fits9.6 GB · Fits17.1 GB · Fits
Nemotron 3.5 Bolt E4B*NVIDIA6.2 GB · Fits8.2 GB · Fits15.1 GB · Fits
Qwen2.5 7B InstructAlibaba Qwen5.2 GB · Fits8.6 GB · Fits15.7 GB · Fits
Mistral 7B Instruct v0.3Mistral AI5.4 GB · Fits8.8 GB · Fits15.6 GB · Fits
Spark-X2.6 4BXHToken3.8 GB · Fits4.7 GB · Fits8.5 GB · Fits
Llama 3.2 3B InstructMeta3.0 GB · Fits4.4 GB · Fits7.4 GB · Fits
LFM3 1.7BLiquid AI1.2 GB · Fits1.9 GB · Fits3.6 GB · Fits
Qwen3.6 35B-A3BAlibaba Qwen22.5 GB · Tight38.0 GB · Too big71.2 GB · Too big
Qwen3.8-35B-A3B (Empero Distill)Empero21.9 GB · Tight38.0 GB · Too big71.2 GB · Too big
Nanbeige4.3 33BNanbeige22.5 GB · Tight36.9 GB · Too big68.0 GB · Too big
Qwen2.5 32B InstructAlibaba Qwen22.0 GB · Tight37.0 GB · Too big67.7 GB · Too big
Qwen3 32BAlibaba Qwen21.9 GB · Tight37.0 GB · Too big67.7 GB · Too big
Kimi K3Moonshot AI1,508.9 GB · Too big1,561.4 GB · Too bign/a
Mistral Large 4 (Le Chonk)Mistral AI635.5 GB · Too big1,115.8 GB · Too big2,099.5 GB · Too big
DeepSeek-V4.1-Flash-1017DeepSeek500.7 GB · Too big508.4 GB · Too bign/a
Qwen3.8 Flash NextAlibaba Qwen119.8 GB · Too big188.5 GB · Too big354.2 GB · Too big
Ornith-1.5-122BOrnith79.9 GB · Too big132.8 GB · Too big249.6 GB · Too big
gpt-oss-120bOpenAI63.1 GB · Too big63.7 GB · Too big65.7 GB · Too big
Llama 3.1 70B InstructMeta45.2 GB · Too big77.7 GB · Too big143.8 GB · Too big
Llama 3.3 70B InstructMeta45.2 GB · Too big77.7 GB · Too big143.8 GB · Too big
Mixtral 8x7B Instruct v0.1Mistral AI27.5 GB · Too big50.7 GB · Too big94.5 GB · Too big
Recursive FlintRecursive25.3 GB · Too big42.5 GB · Too big79.8 GB · Too big

Each cell is the Hugging Face GGUF file size plus FP16 KV cache for an 8K-token context (or the model's maximum, if shorter). Fits = under 90% of 24 GB; Tight = 90 to 100%; Too big = over 24 GB. * = KV cache not included (no config data). Same data as the VRAM calculator and what can I run with 24 GB. Data checked 2026-10-30.

Check one model at your own context length in the VRAM calculator with the RTX 4090 selected.

Can an RTX 4090 run Llama 70B?

Not on its own. Llama 3.3 70B at 4-bit (Q4_K_M) is a 42.5 GB download, and an 8K-token FP16 KV cache adds about 2.7 GB, so it needs about 45.2 GB, nearly twice the card's 24 GB (see it in the calculator). Your options:

  • Two RTX 4090s (48 GB total). llama.cpp can split the layers across both cards. Under our rule that is tight (45.2 GB is over 90% of 48 GB), so keep the context short. One published benchmark ran Llama 3 70B Q4_K_M on 2× RTX 4090 at 19.06 tokens/s generation, with a 1,024-token prompt.[2]
  • Offload part of the model to system RAM. llama.cpp and Ollama can keep some layers on the CPU. It loads, but generation is much slower because those layers run from system memory. We don't publish a speed for it because we haven't found a sourced number for this setup.
  • Rent a bigger card for the hour. On Runpod Community Cloud, a 48 GB RTX A6000 listed at $0.33/hr (also tight for 70B at 8K) and an 80 GB A100 PCIe at $1.19/hr, which fits it comfortably (prices read 2026-10-31; see GPU cost per hour for Vast.ai prices on the same cards).
  • A smaller model that fits. The best pick in the table above runs fully on the card. A MoE model bigger than 24 GB can also work with expert offload to system RAM.

RTX 4090 vs RTX 3090 for local LLMs

Both have 24 GB, so pick on speed, power and price. Specs from NVIDIA:[1][3][4]

RTX 4090 RTX 3090
Memory 24 GB GDDR6X, 384-bit 24 GB GDDR6X, 384-bit
Memory bandwidth 1,008 GB/s 936 GB/s
L2 cache 72 MB 6 MB
CUDA cores 16,384 10,496
Power (NVIDIA spec) 450 W, 850 W PSU 350 W, 750 W PSU
NVLink No Yes

Generating text is mostly limited by memory bandwidth, and the 4090 has only about 8% more of it. Reading a long prompt is limited by compute, where the 4090 is far ahead. The published llama.cpp results match that:

Test (llama.cpp, CUDA) RTX 4090 RTX 3090 4090 lead
Llama 2 7B Q4_0, generation (tg128)[5] 186.21 tok/s 158.16 tok/s +18%
Llama 2 7B Q4_0, prompt (pp512)[5] 11,992.7 tok/s 5,174.69 tok/s 2.3×
Llama 3 8B Q4_K_M, generation[2] 127.74 tok/s 111.74 tok/s +14%
Llama 3 8B Q4_K_M, prompt[2] 6,898.71 tok/s 3,865.39 tok/s 1.8×
Llama 3 70B Q4_K_M on two cards, generation[2] 19.06 tok/s 16.29 tok/s +17%

Different builds and dates, measured by different people; the lead column is our arithmetic. More results on the measured speed page.

In short: for chat, a 4090 answers roughly 15 to 20% faster than a 3090 in these tests. For long documents, RAG and coding agents that send big prompts, it reads them about twice as fast. The model list doesn't change: the 4090 doesn't fit anything a 3090 can't.

More for 24 GB cards

Rent a GPU

An RTX 4090 listed at $0.34/hr on Runpod Community Cloud (read 2026-10-31). Prices for both providers are on the GPU cost per hour page.

Some links on this page are affiliate or referral links: we may get a commission or account credit if you sign up. See our disclosure.

Sources

All sources checked on .

  1. NVIDIA, GeForce RTX 4090 specs (24 GB GDDR6X, 384-bit, 16,384 CUDA cores, 450 W, 850 W PSU, NVLink: No). Checked 2026-11-02. https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
  2. XiongjieDai, GPU-Benchmarks-on-LLM-Inference (llama.cpp, Llama 3 8B and 70B tables, 1,024-token prompt / 1,024 generated). Checked 2026-11-02. https://github.com/XiongjieDai/GPU-Benchmarks-on-LLM-Inference
  3. NVIDIA, Ada GPU Architecture whitepaper, Appendix A: RTX 4090 memory bandwidth 1,008 GB/s, L2 cache 73,728 KB. Checked 2026-11-02. https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
  4. NVIDIA, GA102 (Ampere) whitepaper: RTX 3090 936 GB/s, L2 6,144 KB, 10,496 CUDA cores; and the RTX 3090 spec page (350 W, 750 W PSU, NVLink: Yes). Checked 2026-11-02. https://images.nvidia.com/aem-dam/en-zz/Solutions/geforce/ampere/pdf/NVIDIA-ampere-GA102-GPU-Architecture-Whitepaper-V1.pdf, https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
  5. llama.cpp GitHub discussion #15013, "Performance of llama.cpp on Nvidia CUDA" scoreboard (llama-bench pp512 / tg128). Checked 2026-11-02. https://github.com/ggml-org/llama.cpp/discussions/15013

Too big for your card? Rent the VRAM by the hour, or upgrade.

Run it on a cloud GPU

No upgrade needed

Rent a 24–80 GB GPU for the hour you need it.

Vast.ai Affiliate partner Runpod Affiliate partner E2E Networks Works with

Buy this GPU

Own the VRAM

Compare current GPU prices first on our GPU price table and measured speeds.

B&H Photo Works with

We may earn from links marked Affiliate partner. Disclosure.

Last checked: