Cheapest way to run Gemma 4 31B
We earn commissions when you shop through the links below.
Gemma 4 31B needs 21.5 GB at 4-bit — cheapest rental $0.16/hr.
Quick answer
At 4-bit with an 8K-token conversation, Gemma 4 31B needs about 21.5 GB. It fits a RTX 3090 (24 GB), so if you already own one it costs nothing per token; the cheapest rental big enough is RTX A5000 on Runpod at $0.16 an hour; Novita AI hosts it for $0.14 per 1M input tokens.
| Option | What you need | Cost | Notes |
|---|---|---|---|
| Your own GPU | RTX 3090 (24 GB) at 4-bit (Q4_K_M), needs 21.5 GB | No GPU purchase prices in our data. Check price on Amazon Amazon affiliate link | $0 per token or hour once you own the card (power not counted) |
| Mac (unified memory) | Mac, 24 GB at 4-bit (Q4_K_M), needs 21.5 GB | No Mac prices in our data | $0 per token once you own it |
| Rent a GPU | RTX A5000 (24 GB total) on Runpod Referral link, Community Cloud | $0.16 per hour | Pay only while it runs. Price checked 2026-11-09 |
| Hosted API | Novita AI: Gemma 4 31B Referral link | $0.14 in / $0.4 out per 1M tokens | Pay per token, nothing to set up. Price checked Nov 3 2026 |
Fit rule: the Q4_K_M file (or the highest precision that fits) plus an 8K-token FP16 KV cache under 90% of memory, same as the calculator. Sizes and sources: Gemma 4 31B VRAM page. Rental prices: GPU prices (checked 2026-11-09). Hourly and per-token prices measure different things, so which is cheaper depends on how much you use it.
Links marked Referral link, Affiliate partner or Amazon affiliate link may earn us credit or a commission at no cost to you. Disclosure.
Precision ladder: what each quant costs
| Precision | Needs at 8K | Smallest consumer GPU | Smallest Mac | Cheapest single-GPU rental | Cheapest rental overall (any GPU count) |
|---|---|---|---|---|---|
| 16-bit (FP16/BF16) | 63.3 GB | none (max 32 GB) | Mac, 96 GB at 16-bit (FP16/BF16) | A100 PCIe (80 GB) on Vast.ai, $0.40/hr | A100 PCIe (80 GB) on Vast.ai, $0.40/hr |
| 8-bit (Q8_0) | 34.6 GB | none (max 32 GB) | Mac, 48 GB at 8-bit (Q8_0) | RTX A6000 (48 GB) on Runpod, $0.33/hr | RTX A6000 (48 GB) on Runpod, $0.33/hr |
| 4-bit (Q4_K_M) | 21.5 GB | RTX 3090 (24 GB) | Mac, 24 GB at 4-bit (Q4_K_M) | RTX A5000 (24 GB) on Runpod, $0.16/hr | RTX A5000 (24 GB) on Runpod, $0.16/hr |
Every row names its precision, Mac rows included (Macs share unified memory with the system, so the same 90% rule applies). Single-GPU and multi-GPU rentals are listed separately: a multi-GPU rental needs software that splits the model across cards (llama.cpp, vLLM tensor parallel). Prices checked 2026-11-09.
How context length changes the bill (4-bit)
| Context | Needs | Cheapest rental overall |
|---|---|---|
| 8K tokens | 21.5 GB | RTX A5000 (24 GB) on Runpod, $0.16/hr |
| 32K tokens | 23.5 GB | RTX A6000 (48 GB) on Runpod, $0.33/hr |
| 128K tokens | 31.6 GB | RTX A6000 (48 GB) on Runpod, $0.33/hr |
What $100 of rental buys you
We list no GPU purchase prices (they change daily and we cannot check them), so this page does not compare renting with buying. At the cheapest rental that fits (RTX A5000 (24 GB) on Runpod, $0.16/hr), $100 of rental buys about 625 hours of running Gemma 4 31B at 4-bit with 8K context. Electricity, storage and setup time are not counted.
Fit quirks for this model
- Part of the KV cache is a fixed 1.26 GB that does not grow with context (sliding-window or state layers), included above.
- From config.json: 60 layers, of which 10 are global (full) attention layers with 4 KV heads of size 512, and 50 are sliding-window layers with 16 KV heads of size 256 and a 1,024-token window. At FP16 the global layers take 10 × 4 × 512 × 2 (K and V) × 2 bytes = 81,920 bytes per token, and the sliding layers stop growing at their window: 50 × 1,536 × 16 × 256 × 2 × 2 bytes = about 1.26 GB, fixed (llama.cpp caches window + 512-token batch). llama.cpp stores K and V separately even with attention_k_eq_v.
- Maximum context is 262,144 tokens; longer settings are capped.
Model data checked 2026-11-03; rental prices checked 2026-11-09. For file sizes, sources and the full fit table see the Gemma 4 31B VRAM requirements page; this page is only about cost.
Get fit alerts for your GPU
Join the fit-alerts waitlist
Alerts are not running yet. Join the waitlist and pick your card once; when alerts launch we will email you once to confirm, and only after you confirm will you get the first alerts. Unsubscribe any time.
More: all cheapest-way pages · cheapest GPU rental by VRAM · Ask VRAMNeeded · Scan my GPU