VRAMNeeded Calculator

What can an RTX 5090 run?

The GeForce RTX 5090 has 32 GB of GDDR7 on a 512-bit bus, 8 GB more than an RTX 4090 or RTX 3090.[1] The table below checks every language model in our calculator at three quantizations with an 8K-token context, using the same file sizes and fit rule as the rest of the site.

Best model for the RTX 5090 32GB: Mixtral 8x7B Instruct v0.1. It is the largest model in our calculator (46.7B parameters) that fits comfortably at 4-bit: Q4_K_M plus an 8K-token FP16 KV cache needs 27.5 GB, under 90% of 32 GB. The highest precision it fits comfortably at is 4-bit (Q4_K_M). In total, 22 of the 33 language models we track fit comfortably at 4-bit on this card.

Model4-bit (Q4_K_M)8-bit (Q8_0)16-bit (FP16/BF16)
Mixtral 8x7B Instruct v0.1Mistral AI27.5 GB · Fits50.7 GB · Too big94.5 GB · Too big
Recursive FlintRecursive25.3 GB · Fits42.5 GB · Too big79.8 GB · Too big
Qwen3.6 35B-A3BAlibaba Qwen22.5 GB · Fits38.0 GB · Too big71.2 GB · Too big
Qwen3.8-35B-A3B (Empero Distill)Empero21.9 GB · Fits38.0 GB · Too big71.2 GB · Too big
Nanbeige4.3 33BNanbeige22.5 GB · Fits36.9 GB · Too big68.0 GB · Too big
Qwen2.5 32B InstructAlibaba Qwen22.0 GB · Fits37.0 GB · Too big67.7 GB · Too big
Qwen3 32BAlibaba Qwen21.9 GB · Fits37.0 GB · Too big67.7 GB · Too big
Gemma 4 31B*Google19.6 GB · Fits32.6 GB · Too big61.4 GB · Too big
Qwen3.8 27BAlibaba Qwen18.0 GB · Fits29.7 GB · Tight55.2 GB · Too big
Gemma 4 26B A4B*Google17.0 GB · Fits26.9 GB · Fits50.5 GB · Too big
LFM2.5-24B-A2BLiquid AI14.7 GB · Fits25.7 GB · Fits48.1 GB · Too big
gpt-oss-20bOpenAI11.9 GB · Fits12.3 GB · Fits14.0 GB · Fits
Qwen2.5 14B InstructAlibaba Qwen10.6 GB · Fits17.3 GB · Fits31.2 GB · Tight
Qwen3 14BAlibaba Qwen10.3 GB · Fits17.0 GB · Fits30.9 GB · Tight
DeepSeek-R1-0528-Qwen3-8BDeepSeek6.2 GB · Fits9.9 GB · Fits17.6 GB · Fits
Llama 3.1 8B InstructMeta6.0 GB · Fits9.6 GB · Fits17.1 GB · Fits
Nemotron 3.5 Bolt E4B*NVIDIA6.2 GB · Fits8.2 GB · Fits15.1 GB · Fits
Qwen2.5 7B InstructAlibaba Qwen5.2 GB · Fits8.6 GB · Fits15.7 GB · Fits
Mistral 7B Instruct v0.3Mistral AI5.4 GB · Fits8.8 GB · Fits15.6 GB · Fits
Spark-X2.6 4BXHToken3.8 GB · Fits4.7 GB · Fits8.5 GB · Fits
Llama 3.2 3B InstructMeta3.0 GB · Fits4.4 GB · Fits7.4 GB · Fits
LFM3 1.7BLiquid AI1.2 GB · Fits1.9 GB · Fits3.6 GB · Fits
Kimi K3Moonshot AI1,508.9 GB · Too big1,561.4 GB · Too bign/a
Mistral Large 4 (Le Chonk)Mistral AI635.5 GB · Too big1,115.8 GB · Too big2,099.5 GB · Too big
DeepSeek-V4.1-Flash-1017DeepSeek500.7 GB · Too big508.4 GB · Too bign/a
GLM-5.3-FlashZ.ai (Zhipu AI)181.5 GB · Too big341.1 GB · Too big641.8 GB · Too big
Qwen3-235B-A22BAlibaba Qwen143.7 GB · Too big251.5 GB · Too big471.9 GB · Too big
Qwen3.8 Flash NextAlibaba Qwen119.8 GB · Too big188.5 GB · Too big354.2 GB · Too big
Ornith-1.5-122BOrnith79.9 GB · Too big132.8 GB · Too big249.6 GB · Too big
Qwen3.5-122B-A10BAlibaba Qwen76.7 GB · Too big130.1 GB · Too big244.5 GB · Too big
gpt-oss-120bOpenAI63.1 GB · Too big63.7 GB · Too big65.7 GB · Too big
Llama 3.1 70B InstructMeta45.2 GB · Too big77.7 GB · Too big143.8 GB · Too big
Llama 3.3 70B InstructMeta45.2 GB · Too big77.7 GB · Too big143.8 GB · Too big

Each cell is the Hugging Face GGUF file size plus FP16 KV cache for an 8K-token context (or the model's maximum, if shorter). Fits = under 90% of 32 GB; Tight = 90 to 100%; Too big = over 32 GB. * = KV cache not included (no config data). Same data as the VRAM calculator and what can I run with 32 GB. Data checked 2026-11-02.

Check one model at your own context length in the VRAM calculator with the RTX 5090 selected.

What the extra 8 GB buys over a 24 GB card

Using the same data as our 24 GB and 32 GB tier pages (weights + 8K FP16 KV cache, 90% line):

  • Two models that don't fit 24 GB at all now fit: Mixtral 8x7B at Q4_K_M (27.5 GB) and Recursive Flint at Q4_K_M (25.3 GB).
  • The 32B–35B class moves from tight to comfortable: Qwen3 32B (21.9 GB), Qwen2.5 32B (22.0 GB), Nanbeige4.3 33B (22.5 GB) and Qwen3.6 35B-A3B (22.5 GB) are over 90% of a 24 GB card but under 71% of 32 GB, which leaves room for a longer context.
  • Two small MoE models step up to 8-bit: Gemma 4 26B A4B (26.9 GB at Q8_0) and LFM2.5-24B-A2B (25.7 GB at Q8_0), versus 4-bit on 24 GB.

The 70B class still doesn't fit (see below).

RTX 5090 vs RTX 4090: specs

Specs from NVIDIA's product pages and the 50 Series compare table on them:[1][2][3]

RTX 5090 RTX 4090
Memory 32 GB GDDR7, 512-bit 24 GB GDDR6X, 384-bit
Memory bandwidth 1,792 GB/s 1,008 GB/s
CUDA cores 21,760 16,384
AI TOPS (NVIDIA figure) 3,352 1,321
Power (NVIDIA spec) 575 W, 1,000 W PSU 450 W, 850 W PSU
NVLink No No

The bandwidth figures are not in the product pages' spec lists. They come from the "Memory Bandwidth" row of NVIDIA's RTX 50 Series compare table, which is loaded on the RTX 5090 page; the 4090's 1,008 GB/s matches NVIDIA's Ada whitepaper.[2][4]

Measured speed: RTX 5090 vs RTX 4090

Generating text is mostly limited by memory bandwidth, where the 5090 has about 78% more. Reading a prompt is limited by compute. Published llama.cpp results (CUDA, whole model on one GPU):

Test (llama.cpp, CUDA) RTX 5090 RTX 4090 5090 lead
Llama 2 7B Q4_0, generation (tg128)[5] 290.02 tok/s 186.21 tok/s +56%
Llama 2 7B Q4_0, prompt (pp512)[5] 14,073.41 tok/s 11,992.70 tok/s +17%
Llama 2 7B Q4_0, generation, flash attention on[5] 300.40 tok/s 188.96 tok/s +59%
gpt-oss-20b MXFP4, generation (tg128)[6] 282.51 tok/s 221.95 tok/s +27%
gpt-oss-20b MXFP4, prompt (pp2048)[6] 9,848.38 tok/s 8,022.33 tok/s +23%
gpt-oss-20b MXFP4, prompt (pp32768)[6] 6,290.76 tok/s 5,112.35 tok/s +23%

The Llama 2 rows are the scoreboard entries in llama.cpp discussion #15013 (different builds and testers for each card). The gpt-oss-20b rows come from the same guide, run by the same author with the same build (a094f38) and settings (-fa 1 -b 4096 -ub 2048). The lead column is our arithmetic.

We only list models with a published run on both cards. We don't give per-model speeds for the rest of the table above, because we haven't found sourced 5090 numbers for them. More results are on the measured speed page.

Power limit: one tester in the same discussion ran a 5090 at 400 W and at 600 W with the same build: generation was 236.73 vs 239.62 tok/s (about 1% apart), prompt reading 12,706.26 vs 14,751.98 tok/s (about 14% apart).[5] So for chat, capping the card's power costs little speed. That tester's numbers are lower than the scoreboard's, which shows how much systems and builds vary.

What doesn't fit on an RTX 5090

Model Needs (4-bit + 8K context) On one 5090 Cheapest single GPU that holds it (Runpod)[8]
Llama 3.3 70B 45.2 GB No RTX A6000 48 GB, $0.33/hr Community (tight); A100 PCIe 80 GB, $1.19/hr Community
gpt-oss-120b 63.1 GB No (expert offload works) A100 PCIe 80 GB, $1.19/hr Community
Qwen3.5-122B-A10B 76.7 GB No RTX Pro 6000 96 GB, $1.69/hr Community

Memory needs are from our calculator data (Llama 3.3 70B on a 5090). Runpod prices are listed per-GPU on-demand rates, checked November 2, 2026; Community Cloud runs on shared hosts. Our GPU cost per hour page also has Vast.ai prices for the same cards.

  • gpt-oss-120b with expert offload: llama.cpp's gpt-oss guide gives about 30 tok/s at zero context on a 5090 with --n-cpu-moe 21 (21 layers of experts kept in system RAM). It warns that with --n-cpu-moe 20 the model no longer fits in 32 GB, and that on Windows it can still load and then run very slowly, so check GPU memory in Task Manager.[6] See our MoE expert offload guide.
  • Two RTX 5090s (64 GB) hold Llama 3.3 70B at Q4_K_M with an 8K context at 71% of memory, which fits by our rule. We haven't found a published 2×5090 70B speed, so we don't quote one.
  • Dense 70B with CPU offload loads on one 5090 with enough system RAM, but the layers in system RAM run much slower.

Watch out: the RTX 5090 Laptop GPU has 24 GB

NVIDIA's RTX 5090 Laptop GPU has 24 GB of GDDR7 and 896 GB/s of bandwidth, not 32 GB.[7] For local LLMs it fits the same models as a 24 GB card. One laptop run in discussion #15013 (HP Omen Max 16, 140–160 W) measured 157.84 tok/s generation on the Llama 2 7B Q4_0 test, versus 290.02 for the desktop card.[5]

Rent an RTX 5090 instead

On Runpod, an RTX 5090 is listed at $0.69/hr on Community Cloud and $1.19/hr on Secure Cloud (checked November 2, 2026).[8]

Some links on this page are affiliate or referral links: we may get a commission or account credit if you sign up. See our disclosure.

More for 32 GB cards

Sources

All sources checked on .

  1. NVIDIA, GeForce RTX 5090 specs (32 GB GDDR7, 512-bit, 21,760 CUDA cores, 3,352 AI TOPS, 575 W total graphics power, 1,000 W required system power, NVLink: No, CUDA capability 12.0). Checked 2026-11-02. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
  2. NVIDIA, RTX 50 Series dynamic compare table loaded on the RTX 5090 page ("Memory Bandwidth": RTX 5090 1792 GB/sec, RTX 4090 1008 GB/sec; RTX 4090 1,321 AI TOPS). Checked 2026-11-02. https://images.nvidia.com/aem-dam/en-zz/Solutions/raw-html/dynamic-specs-table-50-series/v1.1.3.js
  3. NVIDIA, GeForce RTX 4090 specs (24 GB GDDR6X, 384-bit, 16,384 CUDA cores, 450 W, 850 W PSU, NVLink: No). Checked 2026-11-02. https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
  4. NVIDIA, Ada GPU Architecture whitepaper, Appendix A (RTX 4090 memory bandwidth 1,008 GB/s). https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf
  5. llama.cpp GitHub discussion #15013, "Performance of llama.cpp on Nvidia CUDA": scoreboard (RTX 5090 commit 8cf6b42, RTX 4090 commit 2241453; llama-bench pp512 / tg128), RodriMora's 400 W / 600 W runs (Aug 2, 2025, build 9c35706), henfiber's RTX 5090 Laptop run (Sep 16, 2026). Checked 2026-11-02. https://github.com/ggml-org/llama.cpp/discussions/15013
  6. llama.cpp GitHub discussion #15396, "guide : running gpt-oss with llama.cpp" (ggerganov): RTX 5090 and RTX 4090 gpt-oss-20b benchmarks (build a094f38) and the gpt-oss-120b --n-cpu-moe 21 tip. Checked 2026-11-02. https://github.com/ggml-org/llama.cpp/discussions/15396
  7. NVIDIA, GeForce RTX 50 Series laptops (RTX 5090 Laptop GPU: 24 GB GDDR7, 896 GB/s). Checked 2026-11-02. https://www.nvidia.com/en-us/geforce/laptops/50-series/
  8. Runpod, GPU pricing (per-GPU on-demand rates; checked November 2, 2026), via our GPU price data. https://www.runpod.io/pricing

Too big for your card? Rent the VRAM by the hour, or upgrade.

Run it on a cloud GPU

No upgrade needed

Rent a 24–80 GB GPU for the hour you need it.

Vast.ai Affiliate partner Runpod Affiliate partner E2E Networks Works with

Buy this GPU

Own the VRAM

Compare current GPU prices first on our GPU price table and measured speeds.

B&H Photo Works with

We may earn from links marked Affiliate partner. Disclosure.

Last checked: