Kimi K3 VRAM requirements: can you run it locally?

Kimi K3 is a mixture-of-experts (MoE) model from Moonshot AI. The weights went up on Hugging Face on July 27, 2026 under the Kimi K3 License, and the repo shows about 813,000 downloads in the last month.^[1] The model card lists 2.8 trillion total parameters with 104B active per token, a 1,048,576-token context window and native image input.^[2] Its config.json shows 93 layers, 896 routed experts with 16 picked per token plus 2 shared experts, and MXFP4 4-bit weights (group size 32).^[3]

Only 104B parameters work on each token, so it runs far faster than a dense 2.8T model would. But all 2.8T weights still have to sit in memory (or be offloaded to system RAM or disk).

Memory needed at each quantization

Below are Moonshot's official checkpoint, NVIDIA's NVFP4 build and Unsloth's GGUF builds, using the exact file sizes on Hugging Face (1 GB = 1,000,000,000 bytes, multi-file builds summed), largest first.^[4][5][7] The vision projector (mmproj, 1.8 GB at F32) is only needed for image input and is not included.^[5]

Why the 8-bit and 4-bit files are almost the same size: Moonshot used quantization-aware training, so the experts were trained in MXFP4 (4-bit) from the fine-tuning stage on.^[2] The official release is the 4-bit model. Unsloth says its UD-Q8_K_XL keeps the MXFP4 experts and the BF16 rest exactly, so it is lossless against the official weights, and UD-Q4_K_XL only shrinks the non-expert tensors to 8-bit.^[6] There is no 16-bit (BF16) checkpoint; 2.8T parameters at 2 bytes each would be about 5.6 TB (our arithmetic). Real savings start below 4 bits.

Quality at low bits (Unsloth's own test): top-1 agreement with the full model is about 90% for UD-Q2_K_XL, 84% for UD-IQ2_XXS and 79% for the 1-bit UD-IQ1_S.^[6] Unsloth recommends UD-IQ1_S as the best size-to-quality balance. The UD-TQ and UD-Q1_0 builds are in the repo but Unsloth has not published quality numbers for them.^[5][6] NVIDIA's NVFP4 build (1,609.9 GB) is for Blackwell GPUs and is slightly larger than the official one.^[7]

Context length adds memory (the KV cache)

Kimi K3 mixes two kinds of attention. Its config.json lists 69 linear-attention (Kimi Delta Attention) layers, which keep a fixed-size state instead of a per-token cache, and 24 gated latent-attention (MLA) layers that grow with the conversation.^[3] Each MLA layer stores a 512-value compressed latent plus a 64-value position key per token. At BF16 that is about 27,648 bytes per token. Moonshot has not published a per-token figure, so treat this as our estimate; your engine may store the cache differently. The fixed state of the 69 linear layers adds a few hundred MB per conversation, whatever the length.

So even the full 1M-token window adds about 29 GB, small next to 1.56 TB of weights. Moonshot's vLLM recipe runs the cache in FP8 (--kv-cache-dtype fp8), which halves it.^[8]

Which GPU setups and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the total memory of the setup.^{[9][10][11][12]}

Column headers show weights + an 8,192-token KV cache (our estimate from config.json). Multi-GPU rows assume the model is split evenly across all GPUs. NVFP4 needs Blackwell GPUs (B200, B300), so ignore that column on H100/H200 and Mac. The 16-GPU row means two servers linked by a fast network, which most single cloud pods do not offer.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own buffers or a longer context can push it over. "No" means it won't fit. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat that row as optimistic.

Our recommendations

These follow from the tables above.

How it compares with other big open MoE models

4-bit file sizes come from each model's Hugging Face repo, the same data as our comparison table.^[13]

†Unsloth UD-Q4_K_XL (no Q4_K_M build exists). ‡ggml-org Q4_K. Mistral Large 4 is Q4_K_M.

How to run it

The model card recommends vLLM, SGLang and TokenSpeed.^[2] vLLM's Kimi K3 recipe defaults to 8× B300 with tensor parallelism on the MXFP4 weights, says "at least 8x GB300" on NVIDIA and "at least 8x MI355X/MI350X" on AMD, and also has 16-GPU profiles (TEP16, TP8×PP2).^[8]

For llama.cpp, Unsloth's guide uses its own llama.cpp fork (a pull request branch with vision support) rather than mainline llama.cpp.^[6] Unsloth's rule of thumb: RAM + VRAM should be about the size of the quant; with less, it still runs from disk but much more slowly.^[6] Its hardware table lists total memory of 610 GB for UD-IQ1_S, 665 GB for UD-IQ1_M, 726 GB for UD-IQ2_XXS, 880 GB for UD-Q2_K_XL and 1.6 TB for the lossless Q8.^[6] See our MoE expert offload guide for keeping experts in system RAM. We have not measured its speed.

The card says Kimi K3 always thinks before answering; effort is set with reasoning_effort (low, high or max, default max).^[2]

Cheapest GPUs to rent for Kimi K3

Eight H200s hold the 2-bit UD-Q2_K_XL; eight B300s hold the official MXFP4 weights. On-demand prices from Runpod's pricing page (checked October 31, 2026; Secure Cloud prices re-checked November 2, 2026; Community Cloud is cheaper but runs on shared hosts):^[12]

Prices checked on 2026-11-02. They change often, so confirm on the provider's page. Runpod lists per-GPU prices; the 8-GPU rows are that price times 8. Whether an 8-GPU pod is available at any moment varies. Rent on Runpod Referral link

Want to try your own setup? Open the calculator with Kimi K3 on 2,304 GB (eight B300s) and change the memory. See all GPU rental prices for more providers.

Frequently asked questions

Can I run Kimi K3 locally?

Only with a very large machine. The smallest build, Unsloth's UD-Q1_0, is 466.4 GB, and Unsloth's recommended 1-bit UD-IQ1_S is 594.0 GB.^[5] You need about that much RAM + VRAM combined; with less, llama.cpp falls back to reading from disk and gets much slower.^[6] No consumer GPU, and no single Mac, holds it in memory.

How much VRAM does Kimi K3 need?

About 1,561 GB (1.56 TB) for the official MXFP4 weights, plus about 0.23 GB for an 8K-token cache by our estimate.^[4][3] That is eight 288 GB B300s or 16 H200s. At 2 bits (UD-Q2_K_XL, 861.3 GB) eight H200s or eight B200s are enough.^[5]

How much VRAM does Kimi K3 need at Q4 and Q8?

Q4 (UD-Q4_K_XL) is 1,508.7 GB and Q8 (UD-Q8_K_XL) is 1,561.2 GB.^[5] They are almost the same because the official model is already 4-bit; Q8 here just copies it losslessly.^[6]

Will it run on a 512 GB Mac Studio?

Not fully in memory. The 466.4 GB UD-Q1_0 build is tight (about 91% of 512 GB by our rule), and the 594.0 GB UD-IQ1_S does not fit.^[5][11] Unsloth mentions running it on a Mac Studio connected to a second machine with 128 GB of RAM.^[6]

Can I use Kimi K3 commercially?

The Kimi K3 License allows commercial use. Companies running it as a paid model API with over $20 million in revenue over 12 months need a separate agreement with Moonshot, and products with over 100 million monthly users or $20 million monthly revenue must show "Kimi K3" in the interface.^[14]

How we calculated

• Weights are the exact file sizes on Hugging Face, checked November 2, 2026, with multi-file builds added up.^[4][5][7] The vision projector is not included.
• KV cache is our estimate from config.json (24 latent-attention layers × (512 latent + 64 position values) at 2 bytes; the 69 linear-attention layers keep a fixed state instead).^[3] Moonshot has not published a figure.
• Device memory is from NVIDIA, Apple and Runpod spec and pricing pages. Multi-GPU rows add up memory and assume an even split.
• Rental prices are the listed on-demand Runpod per-GPU rates from our GPU price data, checked October 31, 2026, times the GPU count.^[12]
• Not included: engine working buffers, the OS and display. These vary by engine and settings, so leave headroom.

Sources

All sources checked on 2026-11-02.

1. Hugging Face, moonshotai/Kimi-K3 (license, first public commit July 27, 2026, downloads in the last month; checked November 2, 2026). https://huggingface.co/moonshotai/Kimi-K3
2. Kimi K3 model card (2.8T total / 104B active, 1M context, MXFP4 quantization-aware training, recommended engines, reasoning settings). https://huggingface.co/moonshotai/Kimi-K3/blob/main/README.md
3. Hugging Face, config.json (93 layers, 69 KDA and 24 MLA layers, kv_lora_rank 512, rope dim 64, 896 experts, 16 per token, MXFP4 quantization config). https://huggingface.co/moonshotai/Kimi-K3/blob/main/config.json
4. Hugging Face, moonshotai/Kimi-K3 file list (official MXFP4 safetensors). https://huggingface.co/moonshotai/Kimi-K3/tree/main
5. Hugging Face, unsloth/Kimi-K3-GGUF file list. https://huggingface.co/unsloth/Kimi-K3-GGUF/tree/main
6. Unsloth, "Kimi K3 - How to Run Locally" (hardware table, quality tests, llama.cpp fork, RAM + VRAM rule of thumb). https://unsloth.ai/docs/models/kimi-k3
7. Hugging Face, nvidia/Kimi-K3-NVFP4 file list. https://huggingface.co/nvidia/Kimi-K3-NVFP4/tree/main
8. vLLM Recipes, moonshotai/Kimi-K3 (default 8× B300 tensor parallel, hardware minimums, FP8 KV cache). https://recipes.vllm.ai/moonshotai/Kimi-K3
9. NVIDIA, H100 (80 GB). https://www.nvidia.com/en-us/data-center/h100/
10. NVIDIA, H200 (141 GB). https://www.nvidia.com/en-us/data-center/h200/
11. Apple, Mac Studio tech specs (up to 512 GB unified memory with M5 Ultra). https://www.apple.com/mac-studio/specs/
12. Runpod, GPU pricing (B300 288 GB, B200 180 GB, H200 141 GB; per-GPU prices; page "Updated October 28, 2026"). https://www.runpod.io/pricing
13. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
14. Kimi K3 License. https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE