Kimi K3.1 Pro VRAM requirements: 1.56 TB official, 593 GB smallest GGUF

Kimi K3.1 Pro is Moonshot AI's follow-up to Kimi K3, post-trained from K3 with large-scale reinforcement learning. The Hugging Face repo was created on November 2, 2026 under the Kimi K3.1 Pro License and showed 27,926 downloads and 1,807 likes when we checked on November 10.^[1] The model card lists 2.8 trillion total parameters with 104B active per token, a 1,048,576-token context window and native image input.^[2]

What changed from K3 (memory-wise): the layout is the same (93 layers, 896 routed experts with 16 per token plus 2 shared, MXFP4 experts), but the 24 full-attention layers switch from Gated MLA to Gated DeepSeek Sparse Attention (DSA) with a top-2048 indexer.^[2][3] The weights are about the same size; the per-token cache is slightly bigger because DSA also stores an indexer key (see below).

Memory needed at each build

Below are Moonshot's official checkpoint and the community builds on Hugging Face, using the exact file sizes (1 GB = 1,000,000,000 bytes, multi-file builds summed), largest first.^[4][5][6][7] The vision projector (mmproj, 0.9 GB at BF16 in the DevQuasar repo) is only needed for image input and is not included.^[5]

Why there is no "Q4_K_M" or "Q8_0" here: like K3, the official release is a 4-bit model (MXFP4 experts from quantization-aware training, everything else BF16), so an 8-bit file would not be smaller and a standard 4-bit GGUF saves only about 3%.^[2][3] DevQuasar's MXFP4_MOE GGUF (1,508.9 GB) is that 3% saving; its card calls it "VERY EXPERIMENTAL" and it was made with a llama.cpp branch.^[5] Real savings start below 4 bits.

Which small build? KarLexAI's KLX builds (IQ1_S 592.7 GB, IQ1_M 647.9 GB, IQ2_XXS 709.9 GB) say they run on stock llama.cpp. KarLexAI reports mean KL divergence 0.106 / 0.084 / 0.072 against the full model, and that IQ1_S runs fully on one 8× H100 node with 131K tokens of context at 19 tokens/s.^[6] Those are the uploader's own numbers; we have not reproduced them, and the repo is small (603 downloads at our check).

The pruned build is a different trade-off. The vLLM project's Kimi-K3.1-Pro-pruned75 keeps 224 of 896 routed experts per layer (75% removed, no calibration data). Its card says it is for "serving and evaluation smoke tests on fewer GPUs, not for quality-sensitive use" and reports GSM8K 43.7%, run at TP=4 on B300.^[7] Treat it as a test build, not as Kimi K3.1 Pro.

Context length adds memory (the KV cache)

Of the 93 layers, config.json lists 69 Kimi Delta Attention (linear) layers, which keep a fixed-size state, and 24 DSA layers that grow with the conversation.^[3] Each DSA layer stores a 512-value compressed latent plus a 64-value position key per token (as in K3), and the DSA indexer adds a 128-value key per token (index_head_dim 128).^[3] vLLM's recipe confirms both the latent and the indexer-key caches are kept.^[8] At BF16 that is about 33,792 bytes per token, roughly 22% more than K3's 27,648. Moonshot has not published a per-token figure, so treat this as our estimate; engines may store the indexer key in FP8 and use less.

So even the full 1M-token window adds about 35 GB, small next to 1.56 TB of weights. vLLM's recipe reports that on 8× B300 with FP8 KV, plain TP8 holds about 2.6M cached tokens and decode context parallelism (DCP8) about 18.5M.^[8]

Which GPU setups and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the total memory of the setup.^{[9][10][11][12]}

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own buffers or a longer context can push it over. "No" means it won't fit. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat that row as optimistic.

Our recommendations

These follow from the tables above.

Kimi K3.1 Pro vs Kimi K3: memory at a glance

Sources: our Kimi K3 page and the sources below.^[3][13][14] Unsloth had not published a K3.1 Pro GGUF when we checked (its repo holds the safetensors only), so the low-bit range is thinner than for K3.^[15]

How to run it

The model card recommends vLLM, SGLang and TokenSpeed.^[2] vLLM's recipe (vLLM 0.34.0) runs the MXFP4 checkpoint with TP8 on two GB300 trays or one B300 node, or one MI355X node on AMD; its default preset is 4× GB300 NVL4 multi-node.^[8]

For llama.cpp, text support for Kimi K3.1 Pro was merged on November 6, 2026 (PR #31779). Vision support was still an open pull request (#31980) on November 10.^[13][14] Rule of thumb from Unsloth's Kimi K3 guide (see our Kimi K3 page): RAM + VRAM should be about the size of the file, or llama.cpp reads from disk and slows down a lot. See our MoE expert offload guide. We have not measured its speed.

The card sets thinking effort with reasoning_effort (low, high or max, default max) and says multi-turn chats must pass back the full assistant message, including reasoning_content.^[2]

Cheapest GPUs to rent for Kimi K3.1 Pro

Four B300s hold the 2-bit Q2_K GGUF; eight B300s hold the official MXFP4 weights. On-demand per-GPU prices from Runpod (our price file, checked November 9, 2026; Community Cloud is cheaper but runs on shared hosts), times the GPU count:^[12]