What can I run on an RTX 5060 8GB?
We earn commissions when you shop through the links below.
18 models fit in 8 GB — cheapest rental $0.06/hr.
The desktop GeForce RTX 5060 has 8 GB of GDDR7 on a 128-bit bus with 448 GB/s of memory bandwidth, 3,840 CUDA cores, 614 AI TOPS (NVIDIA's figure), 145 W total graphics power and a 550 W recommended system power.[1][2] The table below checks every language model in our calculator at three quantizations with an 8K-token context, using the same file sizes and fit rule as the rest of the site.
Best model for the RTX 5060 8GB: Qwen3.5 9B. It is the largest model in our calculator (9.7B parameters) that fits comfortably at 4-bit: Q4_K_M plus an 8K-token FP16 KV cache needs 6.4 GB, under 90% of 8 GB. The highest precision it fits comfortably at is 4-bit (Q4_K_M). In total, 18 of the 50 calculator models fit comfortably at 4-bit on this card.
| Model | 4-bit (Q4_K_M) | 8-bit (Q8_0) | 16-bit (FP16/BF16) |
|---|---|---|---|
| Qwen3.5 9BAlibaba Qwen | 6.4 GB · Fits | 10.1 GB · Too big | 18.7 GB · Too big |
| Ornith 1.5 9BOrnith AI | 6.0 GB · Fits | 10.1 GB · Too big | 18.7 GB · Too big |
| DeepSeek-R1-0528-Qwen3-8BDeepSeek | 6.2 GB · Fits | 9.9 GB · Too big | 17.6 GB · Too big |
| Qwen3 8BAlibaba Qwen | 6.2 GB · Fits | 9.9 GB · Too big | 17.6 GB · Too big |
| Llama 3.1 8B InstructMeta | 6.0 GB · Fits | 9.6 GB · Too big | 17.1 GB · Too big |
| Gemma 4 E4BGoogle | 5.6 GB · Fits | 8.2 GB · Too big | 15.2 GB · Too big |
| Nemotron 3.5 Bolt E4BNVIDIA | 6.4 GB · Fits | 8.4 GB · Too big | 15.2 GB · Too big |
| Qwen2.5 7B InstructAlibaba Qwen | 5.2 GB · Fits | 8.6 GB · Too big | 15.7 GB · Too big |
| Qwen2.5-Coder 7BAlibaba Qwen | 5.2 GB · Fits | 8.6 GB · Too big | 15.7 GB · Too big |
| Mistral 7B Instruct v0.3Mistral AI | 5.4 GB · Fits | 8.8 GB · Too big | 15.6 GB · Too big |
| Spark-X2.6 4BXHToken | 3.8 GB · Fits | 4.7 GB · Fits | 8.5 GB · Too big |
| Qwen3 4BAlibaba Qwen | 3.7 GB · Fits | 5.5 GB · Fits | 9.3 GB · Too big |
| Llama 3.2 3B InstructMeta | 3.0 GB · Fits | 4.4 GB · Fits | 7.4 GB · Tight |
| Qwen3 1.7BAlibaba Qwen | 2.2 GB · Fits | 3.1 GB · Fits | 5.0 GB · Fits |
| LFM3 1.7BLiquid AI | 1.2 GB · Fits | 1.9 GB · Fits | 3.6 GB · Fits |
| Qwen2.5 1.5B InstructAlibaba Qwen | 1.2 GB · Fits | 1.9 GB · Fits | 3.3 GB · Fits |
| Qwen3 0.6BAlibaba Qwen | 1.4 GB · Fits | 1.7 GB · Fits | 2.4 GB · Fits |
| Qwen2.5 0.5B InstructAlibaba Qwen | 0.5 GB · Fits | 0.6 GB · Fits | 1.1 GB · Fits |
| Gemma 4 12BGoogle | 7.8 GB · Tight | 13.3 GB · Too big | 24.5 GB · Too big |
| Kimi K3Moonshot AI | 1,508.9 GB · Too big | 1,561.4 GB · Too big | n/a |
| Mistral Large 4 (Le Chonk)Mistral AI | 635.5 GB · Too big | 1,115.8 GB · Too big | 2,099.5 GB · Too big |
| DeepSeek-V4.1-Flash-1017DeepSeek | 500.7 GB · Too big | 508.4 GB · Too big | n/a |
| DeepSeek-V3.2DeepSeek | 406.0 GB · Too big | 713.9 GB · Too big | 1,342.9 GB · Too big |
| GLM-5.3-FlashZ.ai (Zhipu AI) | 181.5 GB · Too big | 341.1 GB · Too big | 641.8 GB · Too big |
| Qwen3-235B-A22BAlibaba Qwen | 143.7 GB · Too big | 251.5 GB · Too big | 471.9 GB · Too big |
| Qwen3.8 Flash NextAlibaba Qwen | 119.8 GB · Too big | 188.5 GB · Too big | 354.2 GB · Too big |
| Ornith-1.5-122BOrnith | 79.9 GB · Too big | 132.8 GB · Too big | 249.6 GB · Too big |
| Qwen3.5-122B-A10BAlibaba Qwen | 76.7 GB · Too big | 130.1 GB · Too big | 244.5 GB · Too big |
| gpt-oss-120bOpenAI | 63.1 GB · Too big | 63.7 GB · Too big | 65.7 GB · Too big (MXFP4 native) |
| Nemotron 3.5 Bolt 74B-A9BNVIDIA | 50.5 GB · Too big | 79.1 GB · Too big | 148.5 GB · Too big |
| Llama 3.1 70B InstructMeta | 45.2 GB · Too big | 77.7 GB · Too big | 143.8 GB · Too big |
| Llama 3.3 70B InstructMeta | 45.2 GB · Too big | 77.7 GB · Too big | 143.8 GB · Too big |
| Mixtral 8x7B Instruct v0.1Mistral AI | 27.5 GB · Too big | 50.7 GB · Too big | 94.5 GB · Too big |
| Recursive FlintRecursive | 25.3 GB · Too big | 42.5 GB · Too big | 79.8 GB · Too big |
| Qwen3.6 35B-A3BAlibaba Qwen | 22.5 GB · Too big | 38.0 GB · Too big | 71.2 GB · Too big |
| Qwen3.8-35B-A3B (Empero Distill)Empero | 21.9 GB · Too big | 38.0 GB · Too big | 71.2 GB · Too big |
| Nanbeige4.3 33BNanbeige | 22.5 GB · Too big | 36.9 GB · Too big | 68.0 GB · Too big |
| Qwen2.5 32B InstructAlibaba Qwen | 22.0 GB · Too big | 37.0 GB · Too big | 67.7 GB · Too big |
| Qwen2.5-Coder 32BAlibaba Qwen | 22.0 GB · Too big | 37.0 GB · Too big | 67.7 GB · Too big |
| Qwen3 32BAlibaba Qwen | 21.9 GB · Too big | 37.0 GB · Too big | 67.7 GB · Too big |
| Gemma 4 31BGoogle | 21.5 GB · Too big | 34.6 GB · Too big | 63.3 GB · Too big |
| Qwen3.8 27BAlibaba Qwen | 18.0 GB · Too big | 29.7 GB · Too big | 55.2 GB · Too big |
| Gemma 4 26B A4BGoogle | 17.5 GB · Too big | 27.3 GB · Too big | 51.0 GB · Too big |
| Devstral Small 2 24BMistral AI | 15.7 GB · Too big | 26.4 GB · Too big | 48.5 GB · Too big |
| LFM2.5-24B-A2BLiquid AI | 14.7 GB · Too big | 25.7 GB · Too big | 48.1 GB · Too big |
| gpt-oss-20bOpenAI | 11.9 GB · Too big | 12.3 GB · Too big | 14.0 GB · Too big (MXFP4 native) |
| Qwen2.5 14B InstructAlibaba Qwen | 10.6 GB · Too big | 17.3 GB · Too big | 31.2 GB · Too big |
| Qwen2.5-Coder 14BAlibaba Qwen | 10.6 GB · Too big | 17.3 GB · Too big | 31.2 GB · Too big |
| Qwen3 14BAlibaba Qwen | 10.3 GB · Too big | 17.0 GB · Too big | 30.9 GB · Too big |
| Phi-4 14BMicrosoft | 10.7 GB · Too big | 17.3 GB · Too big | 31.0 GB · Too big |
Each cell is the Hugging Face GGUF file size plus FP16 KV cache for an 8K-token context (or the model's maximum, if shorter). Fits = under 90% of 8 GB; Tight = 90 to 100%; Too big = over 8 GB. * = KV cache not included (no config data). Same data as the VRAM calculator and what can I run with 8 GB. Data checked 2026-11-03.
Check one model at your own context length in the VRAM calculator with the RTX 5060 selected.
Same 8 GB ceiling as an RTX 4060 or 4060 Ti 8GB
Memory decides what fits. The RTX 5060, the RTX 4060 and the RTX 4060 Ti 8GB all have 8 GB, so they fit the same calculator models at the same quants and context: 18 of 50 at 4-bit (Q4_K_M) with an 8K FP16 KV cache under our 90% headroom rule (live table above). What the 5060 changes is speed, because it has much faster memory. Specs from NVIDIA:[1][2]
| RTX 5060 | RTX 4060 Ti 8GB | RTX 4060 | |
|---|---|---|---|
| Memory | 8 GB GDDR7, 128-bit | 8 GB GDDR6, 128-bit | 8 GB GDDR6, 128-bit |
| Memory bandwidth | 448 GB/s | 288 GB/s | 272 GB/s |
| AI TOPS (NVIDIA's figure) | 614 | 353 | 242 |
| Architecture | Blackwell | Ada Lovelace | Ada Lovelace |
How fast is it? A measured llama.cpp result
From the public llama.cpp CUDA scoreboard, same benchmark on both cards (Llama 2 7B, Q4_0, llama-bench, flash attention off, fully on the GPU):[3]
| Card | Generation (tg128) | Prompt (pp512) | llama.cpp build |
|---|---|---|---|
| RTX 5060 8GB | 96.70 tok/s | 3,269.08 tok/s | 3a3edc9, posted June 19, 2026 |
| RTX 4060 Ti 8GB | 63.86 tok/s | 3,394.63 tok/s | 89d1029 |
With flash attention on, the same RTX 5060 post shows 100.61 tok/s generation and 3,799.47 tok/s prompt.[3] Generation is about 1.5× faster on the 5060, close to the ratio of memory bandwidth (448 vs 288 GB/s); prompt speed is about the same. The two rows were posted by different people on different llama.cpp builds, so treat the gap as a ballpark. We have no sourced RTX 4060 (non-Ti) row. More cards: measured speeds.
When 8 GB is not enough
- gpt-oss-20b needs about 11.9 GB at 4-bit with 8K KV in our calculator, so it does not fit an 8 GB card. The RTX 3060 12GB or RTX 5070 12GB are the next step up.
- 14B-class models (Qwen3 14B, Phi-4) need about 10–11 GB at 4-bit: a 12 GB card.
- Long context eats the headroom. The fit table assumes 8K tokens. At 32K the KV cache can add several GB on its own; check your model in the calculator.
- MoE bigger than VRAM: you can keep experts in system RAM with expert offload (slower).
- Laptop warning: NVIDIA lists the RTX 5060 Laptop GPU with the same 8 GB of GDDR7 but 384 GB/s and 3,328 CUDA cores (vs 448 GB/s and 3,840 on the desktop card),[5] so it fits the same models but should generate more slowly than the desktop row above (our inference from bandwidth; not measured).
More for 8 GB cards
- Everything that fits in 8 GB, including image generation and gaming
- RTX 5060 Ti 16GB fit table: same bus, twice the memory
- Best Ollama models by GPU
- VRAM calculator
Rent a GPU
If 8 GB is too small for one job, renting a bigger card by the hour is cheaper than upgrading for a test. Every number below comes from our dated price file, not an estimate:
| Provider | Cheapest GPU | $ / GPU / hr | Plan / note |
|---|---|---|---|
| Vast.ai Referral link | RTX 3060 (12 GB) | $0.06 | Marketplace, verified on-demand · lowest of 28 verified on-demand single-GPU asks Verified: Nov 10, 2026 10:35 AM PTVerified: Nov 10, 2026 10:35 AM PT |
| Runpod | 12-16 GB | No single-GPU on-demand offer in this tier in our 2026-11-09 read | |
| Vast.ai Referral link | RTX 3090 (24 GB) | $0.16 | Marketplace, verified on-demand · lowest of 30 verified on-demand single-GPU asks Verified: Nov 10, 2026 10:35 AM PTVerified: Nov 10, 2026 10:35 AM PT |
| Runpod Referral link | RTX A5000 (24 GB) | $0.16 | Community Cloud Verified: Nov 10, 2026 10:35 AM PTVerified: Nov 10, 2026 10:35 AM PT |
| Novita AI Referral link | No GPU rental (API instead) | $0.02 / $0.05 per 1M tokens | Llama 3.1 8B Instruct API, in / out (not a GPU-hour rate; no hardware to rent) Verified: Nov 3, 2026Verified: Nov 3, 2026 Price may be stale |
Cheapest single-GPU on-demand rates we read from each provider (copied, never estimated). Each row shows its own Verified time (ⓘ on phones); amber + “Price may be stale” means older than 24 hours, so re-check on the provider before you rent. How we read and date prices: methodology. Full table: GPU prices.
Honest note: Vast.ai is a host marketplace (quality varies; the verified on-demand filter helps). Runpod Community Cloud is shared capacity; Secure Cloud costs more. Novita AI does not rent GPUs; it runs open models for you by API. Dated side-by-side: Vast.ai vs Runpod. More: GPU cost per hour · AI APIs.
Some links on this page are affiliate or referral links: we may get a commission or account credit if you sign up. See our disclosure.
Get fit alerts for your GPU
Join the fit-alerts waitlist
Alerts are not running yet. Join the waitlist and pick your card once; when alerts launch we will email you once to confirm, and only after you confirm will you get the first alerts. Unsubscribe any time.
Every number on this page links to its source.
How we calculate + sources ⓘ
Sources
All sources checked on .
- NVIDIA, GeForce RTX 5060 family specs (RTX 5060: 3,840 CUDA cores, 614 AI TOPS, 8 GB GDDR7, 128-bit, 145 W total graphics power, 550 W required system power; Blackwell). Checked 2026-11-10. https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5060-family/
- NVIDIA, RTX 50 Series dynamic compare table v1.1.3 (memory bandwidth RTX 5060 448 GB/s, RTX 4060 Ti 288 GB/s, RTX 4060 272 GB/s; AI TOPS 614 / 353 / 242). Checked 2026-11-10. https://images.nvidia.com/aem-dam/en-zz/Solutions/raw-html/dynamic-specs-table-50-series/v1.1.3.js
- llama.cpp GitHub discussion #15013, "Performance of llama.cpp on Nvidia CUDA" (RTX 5060 8GB post by odbguru, June 19, 2026, build 3a3edc9; RTX 4060 Ti 8 GB scoreboard row, commit 89d1029). Checked 2026-11-10. https://github.com/ggml-org/llama.cpp/discussions/15013
- Runpod pricing page and Vast.ai verified on-demand marketplace API — rates in
data/gpu_prices.json(last verified November 9, 2026). Novita AI public model list (API prices),data/novita.json. - NVIDIA, GeForce RTX 50 Series laptops (RTX 5060 Laptop GPU: 3,328 CUDA cores, 572 AI TOPS, 8 GB GDDR7, 384 GB/s). Checked 2026-11-10. https://www.nvidia.com/en-us/geforce/laptops/50-series/
Too big for your card? Rent the VRAM by the hour, or upgrade.
Run it on a cloud GPU
No upgrade needed
Rent a 24–80 GB GPU for the hour you need it.
Vast.ai 
Runpod Buy this GPU
Own the VRAM
Compare current GPU prices first on our GPU price table and measured speeds.
We may earn from links marked Affiliate partner or Referral link. Disclosure.