VRAMNeeded Calculator

OpenAI · VRAM requirements

gpt-oss-20b VRAM requirements: can you run it locally?

  • Parameters20.9B (3.60B active)
  • Context128K
  • 4-bit file11.7 GB
  • Checked2026-10-28

Quick answer

At the popular 4-bit Q4_K_M quantization, gpt-oss-20b's weights are 11.7 GB, and an 8K-token context adds about 0.20 GB of KV cache, so plan for roughly 11.9 GB plus your engine's overhead.[4][3] That fits a 16 GB card; a 24 GB card is comfortable. Unquantized BF16 weights are about 13.8 GB.[4]

gpt-oss-20b is from OpenAI. Its Hugging Face repo was created on 2025-08-04, it is licensed Apache 2.0, and Hugging Face lists 20,914,757,184 parameters.[1] The model card reports about 3.6B active parameters per token.[1] Native context is up to 131,072 tokens where the card documents it.[2]

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files fit on smaller GPUs, at some cost in quality. Below are GGUF builds from the bartowski (or listed) repo using exact Hugging Face file sizes (1 GB = 1,000,000,000 bytes), largest first.[4] The MoE experts stay in MXFP4 in every GGUF, so Q4_K_M, Q8_0 and BF16 files are close in size.

QuantizationWeights
BF16 (original)13.8 GB
Q8_012.1 GB
Q6_K_L12.0 GB
Q6_K12.0 GB
Q5_K_L11.9 GB
Q4_K_L11.9 GB
Q2_K_L11.8 GB
Q3_K_XL11.8 GB
Q5_K_M11.7 GB
Q5_K_S11.7 GB
Q4_K_M Common default11.7 GB
Q4_K_S11.7 GB
Q4_111.6 GB
IQ4_NL11.6 GB
IQ4_XS11.6 GB
Q3_K_M11.6 GB
IQ3_M11.6 GB
IQ3_XS11.6 GB
IQ3_XXS11.6 GB
Q2_K11.6 GB
Q3_K_S11.6 GB
IQ2_M11.5 GB
IQ2_S11.5 GB
Q4_011.5 GB
IQ2_XS11.5 GB
IQ2_XXS11.5 GB
Q3_K_L11.5 GB

Ollama's gpt-oss:20b tag is the usual one-line install for the 20B build when the library lists it.[5][6]

Context length adds memory (the KV cache)

12 full-attention layers × 8 KV heads × 64 head dim × 2 × 2 bytes. The other 12 layers use a 128-token sliding window, so their cache stays tiny.[3] An FP16 cache takes 24,576 bytes per token:

ContextKV cache (FP16)
4,096 tokens0.10 GB
8,192 tokens0.20 GB
32,768 tokens0.81 GB
131,072 tokens3.22 GB

Some engines can store the cache in 8-bit to roughly halve it.

Which GPUs and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the device's memory.[7][8][9][10][11][12][13]

DeviceBF16 (original)14.0 GBQ8_012.3 GBQ6_K12.2 GBQ5_K_M11.9 GBQ4_K_M11.9 GBIQ4_XS11.8 GB
RTX 306012 GBNoNoNoTightTightTight
RTX 4060 Ti16 GBFitsFitsFitsFitsFitsFits
RTX 309024 GBFitsFitsFitsFitsFitsFits
RTX 409024 GBFitsFitsFitsFitsFitsFits
RTX 509032 GBFitsFitsFitsFitsFitsFits
RTX 6000 Ada48 GBFitsFitsFitsFitsFitsFits
Mac24 GB unifiedFitsFitsFitsFitsFitsFits
Mac32 GB unifiedFitsFitsFitsFitsFitsFits
Mac64 GB unifiedFitsFitsFitsFitsFitsFits

Column headers show weights + 8,192-token KV cache.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%. "No" means it won't fit on the GPU alone. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat Mac rows as optimistic.

How to run it

The easiest way is Ollama when the library carries this build: ollama run gpt-oss:20b.[5] With llama.cpp or LM Studio, download a GGUF such as the Q4_K_M file from bartowski/openai_gpt-oss-20b-GGUF.[4]

If the model is bigger than your VRAM, these tools can keep some layers in system RAM, but every token then waits on the slower memory, so expect much lower speed.

Cheapest GPU to rent for gpt-oss-20b

On-demand Secure Cloud prices listed on Runpod's pricing page (confirm before you rent — prices move):[14]

GPU (Runpod Secure Cloud (on-demand, as listed))VRAMPrice
RTX 409024 GB$0.89/hr
RTX 509032 GB$1.19/hr
RTX A600048 GB$0.59/hr
A100 PCIe80 GB$1.79/hr

Prices checked on 2026-10-28. They change often, so confirm on the provider's page. Rent on Runpod Referral link

Rent a GPU for this model
Affiliate partnerVast.aiConsumer GPU marketplace (4090/3090-class rentals)See GPUs →Referral linkRunpodPods and serverless GPUsSee GPUs →Works withE2E NetworksIndia GPU cloudSee GPUs →
Run it via API instead
Affiliate partnerNovita AI model APIsBrowse open models you can call through Novita's API instead of hosting them.Browse model APIs →Affiliate partnerNovita SandboxCloud runtimes for AI agents: browser access, filesystem, computer use, long-running tasks.See Sandbox →

Don't want to host it yourself? Compare AI APIs and Sandboxes.

See all GPU rental prices for more providers.

Where to rent or buy

Run it on a cloud GPU

  • Vast.ai Affiliate partner Consumer GPU marketplace (4090/3090-class rentals)
  • Runpod Affiliate partner Pods and serverless GPUs (referral link)
  • E2E Networks Works with India GPU cloud (official site)

Run it via API instead

  • Novita AI Affiliate partner Open-model APIs and Sandbox for AI agents (no GPU rental)

Buy this GPU (or a Mac)

  • B&H Photo Works with GPUs and computing gear (official site)

Which card? Read our best GPUs with 16 GB buying guide (the smallest size this model fits comfortably).

We may earn from links marked Affiliate partner. Disclosure.

Frequently asked questions

How much VRAM does gpt-oss-20b need at 4-bit?

About 11.7 GB for the Q4_K_M weights alone.[4] An 8K FP16 KV cache adds about 0.20 GB, for roughly 11.9 GB before engine overhead.[3]

Can I run it on a consumer GPU?

A 16–24 GB card is the practical target at Q4_K_M. Check the fit table above for your exact card.

Is BF16 practical?

BF16 is about 13.8 GB.[4] Prefer Q4_K_M or Q8_0 unless you have the memory.

How we calculated

  • Weights are the exact GGUF file sizes on Hugging Face (shards added together).[4]
  • KV cache is calculated from config.json at FP16.[3]
  • Device memory is from NVIDIA and Apple spec pages.
  • Not included: engine working buffers, the OS and display. Leave headroom.

Sources

All sources checked on .

  1. Hugging Face model page: openai/gpt-oss-20b (parameter count, license, repo date). https://huggingface.co/openai/gpt-oss-20b
  2. Hugging Face model card / README for architecture and context. https://huggingface.co/openai/gpt-oss-20b
  3. config.json (openai/gpt-oss-20b). https://huggingface.co/openai/gpt-oss-20b/blob/main/config.json
  4. Hugging Face GGUF file sizes: bartowski/openai_gpt-oss-20b-GGUF. https://huggingface.co/bartowski/openai_gpt-oss-20b-GGUF/tree/main
  5. Ollama library tags (where listed). https://ollama.com/library/gpt-oss
  6. Ollama pull / tag sizes where published for this family.
  7. NVIDIA GeForce RTX 4060 / 50-series 8 GB specs. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5060-family/
  8. NVIDIA GeForce RTX 3060 specs (12 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3060-3060ti/
  9. NVIDIA GeForce RTX 4060 Ti / 16 GB class specs. https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/
  10. NVIDIA GeForce RTX 3090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
  11. NVIDIA GeForce RTX 4090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
  12. NVIDIA GeForce RTX 5090 / RTX 6000 Ada specs. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
  13. Apple MacBook Pro / Mac Studio unified memory specs. https://www.apple.com/macbook-pro/specs/
  14. Runpod, GPU pricing. https://www.runpod.io/pricing

Last checked: