VRAMNeeded Calculator

OpenAI · VRAM requirements

gpt-oss-120b VRAM requirements: can you run it locally?

  • Parameters116.8B (5.1B active)
  • Context128K
  • 4-bit file62.8 GB
  • Checked2026-10-28

Quick answer

At the popular 4-bit Q4_K_M quantization, gpt-oss-120b's weights are 62.8 GB, and an 8K-token context adds about 0.30 GB of KV cache, so plan for roughly 63.1 GB plus your engine's overhead.[4][3] You will need a 48 GB+ workstation GPU, multi-GPU, or a high-memory Mac — or rent. Unquantized BF16 weights are about 65.4 GB.[4]

gpt-oss-120b is from OpenAI. Its Hugging Face repo was created on 2025-08-04, it is licensed Apache 2.0, and Hugging Face lists 116,829,156,672 parameters.[1] The model card reports about 5.1B active parameters per token.[1] Native context is up to 131,072 tokens where the card documents it.[2]

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files fit on smaller GPUs, at some cost in quality. Below are GGUF builds from the bartowski (or listed) repo using exact Hugging Face file sizes (1 GB = 1,000,000,000 bytes), largest first.[4] Experts stay in MXFP4, so quant sizes are close.

QuantizationWeights
BF16 (original)65.4 GB
Q8_063.4 GB
Q6_K63.3 GB
Q4_K_L63.1 GB
Q2_K_L63.0 GB
Q3_K_XL62.9 GB
Q4_K_M Common default62.8 GB
Q4_162.7 GB
IQ4_NL62.7 GB
IQ4_XS62.7 GB
Q3_K_M62.7 GB
IQ3_M62.7 GB
Q2_K62.7 GB
Q3_K_S62.7 GB
IQ2_M62.7 GB
Q4_062.6 GB
Q3_K_L62.6 GB

Ollama's gpt-oss:120b tag is the usual one-line install for the 120B build when the library lists it.[5][6]

Context length adds memory (the KV cache)

18 full-attention layers × 8 KV heads × 64 head dim × 2 × 2 bytes; sliding-window layers capped at 128 tokens.[3] An FP16 cache takes 36,864 bytes per token:

ContextKV cache (FP16)
4,096 tokens0.15 GB
8,192 tokens0.30 GB
32,768 tokens1.21 GB
131,072 tokens4.83 GB

Some engines can store the cache in 8-bit to roughly halve it.

Which GPUs and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the device's memory.[7][8][9][10][11][12][13]

DeviceBF16 (original)65.7 GBQ8_063.7 GBQ6_K63.6 GBQ4_K_M63.1 GBIQ4_XS63.0 GBQ3_K_M63.0 GB
RTX 409024 GBNoNoNoNoNoNo
RTX 509032 GBNoNoNoNoNoNo
RTX 6000 Ada48 GBNoNoNoNoNoNo
A10080 GBFitsFitsFitsFitsFitsFits
H10080 GBFitsFitsFitsFitsFitsFits
Mac64 GB unifiedNoTightTightTightTightTight
Mac128 GB unifiedFitsFitsFitsFitsFitsFits
Mac192 GB unifiedFitsFitsFitsFitsFitsFits

Column headers show weights + 8,192-token KV cache.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%. "No" means it won't fit on the GPU alone. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat Mac rows as optimistic.

How to run it

The easiest way is Ollama when the library carries this build: ollama run gpt-oss:120b.[5] With llama.cpp or LM Studio, download a GGUF such as the Q4_K_M file from bartowski/openai_gpt-oss-120b-GGUF.[4]

If the model is bigger than your VRAM, these tools can keep some layers in system RAM, but every token then waits on the slower memory, so expect much lower speed.

Cheapest GPU to rent for gpt-oss-120b

On-demand Secure Cloud prices listed on Runpod's pricing page (confirm before you rent — prices move):[14]

GPU (Runpod Secure Cloud (on-demand, as listed))VRAMPrice
RTX 409024 GB$0.89/hr
RTX 509032 GB$1.19/hr
RTX A600048 GB$0.59/hr
A100 PCIe80 GB$1.79/hr

Prices checked on 2026-10-28. They change often, so confirm on the provider's page. Rent on Runpod Referral link

Rent a GPU for this model
Affiliate partnerVast.aiConsumer GPU marketplace (4090/3090-class rentals)See GPUs →Referral linkRunpodPods and serverless GPUsSee GPUs →Works withE2E NetworksIndia GPU cloudSee GPUs →
Run it via API instead
Affiliate partnerGPT OSS 120B on Novita AICall this model through Novita's API or try it in their playground, no GPU needed.Open model page →Affiliate partnerNovita SandboxCloud runtimes for AI agents: browser access, filesystem, computer use, long-running tasks.See Sandbox →

Don't want to host it yourself? Compare AI APIs and Sandboxes.

Or call it by API: compare LLM API prices.

See all GPU rental prices for more providers.

Where to rent or buy

Run it on a cloud GPU

  • Vast.ai Affiliate partner Consumer GPU marketplace (4090/3090-class rentals)
  • Runpod Affiliate partner Pods and serverless GPUs (referral link)
  • E2E Networks Works with India GPU cloud (official site)

Run it via API instead

  • Novita AI Affiliate partner Open-model APIs and Sandbox for AI agents (no GPU rental)

Buy this GPU (or a Mac)

  • B&H Photo Works with GPUs and computing gear (official site)

We may earn from links marked Affiliate partner. Disclosure.

Frequently asked questions

How much VRAM does gpt-oss-120b need at 4-bit?

About 62.8 GB for the Q4_K_M weights alone.[4] An 8K FP16 KV cache adds about 0.30 GB, for roughly 63.1 GB before engine overhead.[3]

Can I run it on a consumer GPU?

You will likely need a 48 GB+ card, multi-GPU, heavy RAM offload, or a cloud rental. Check the fit table above for your exact card.

Is BF16 practical?

BF16 is about 65.4 GB.[4] Prefer Q4_K_M or Q8_0 unless you have the memory.

How we calculated

  • Weights are the exact GGUF file sizes on Hugging Face (shards added together).[4]
  • KV cache is calculated from config.json at FP16.[3]
  • Device memory is from NVIDIA and Apple spec pages.
  • Not included: engine working buffers, the OS and display. Leave headroom.

Sources

All sources checked on .

  1. Hugging Face model page: openai/gpt-oss-120b (parameter count, license, repo date). https://huggingface.co/openai/gpt-oss-120b
  2. Hugging Face model card / README for architecture and context. https://huggingface.co/openai/gpt-oss-120b
  3. config.json (openai/gpt-oss-120b). https://huggingface.co/openai/gpt-oss-120b/blob/main/config.json
  4. Hugging Face GGUF file sizes: bartowski/openai_gpt-oss-120b-GGUF. https://huggingface.co/bartowski/openai_gpt-oss-120b-GGUF/tree/main
  5. Ollama library tags (where listed). https://ollama.com/library/gpt-oss
  6. Ollama pull / tag sizes where published for this family.
  7. NVIDIA GeForce RTX 4060 / 50-series 8 GB specs. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5060-family/
  8. NVIDIA GeForce RTX 3060 specs (12 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3060-3060ti/
  9. NVIDIA GeForce RTX 4060 Ti / 16 GB class specs. https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/
  10. NVIDIA GeForce RTX 3090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
  11. NVIDIA GeForce RTX 4090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
  12. NVIDIA GeForce RTX 5090 / RTX 6000 Ada specs. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
  13. Apple MacBook Pro / Mac Studio unified memory specs. https://www.apple.com/macbook-pro/specs/
  14. Runpod, GPU pricing. https://www.runpod.io/pricing

Last checked: