Mistral Large 4 (Le Chonk) VRAM requirements: what it takes to run the 1T model

Mistral Large 4, nicknamed "Le Chonk", is Mistral AI's largest model. Mistral released the open weights on October 29, 2026, after a public preview earlier in October.^[4] The model card describes it as a 1 trillion-parameter natively multimodal mixture-of-experts (MoE) model with 49 billion active parameters per token (52 billion counting the embedding and output layers), with a 1M-token context window and text-plus-image input.^[2] Its config.json shows 72 layers, 128 routed experts with 4 picked per token plus 1 shared expert, and multi-head latent attention.^[3] It is released under the Mistral Ecosystem Open-Weight License, a modified MIT license; the card says running a model-as-a-service business needs a separate commercial license from Mistral.^[2]

Because it is an MoE, only a few experts work on each token, so it runs faster than a dense model of the same size. But all of the weights still have to sit in memory. At about a trillion parameters, even the smallest builds need hundreds of gigabytes.

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files need less memory, at some cost in quality, and the cost grows below about 4 bits. Below are Mistral's two official checkpoints (FP8 and NVFP4) and the GGUF builds published by ggml-org (the llama.cpp team) and Unsloth, using the exact file sizes on Hugging Face (1 GB = 1,000,000,000 bytes, multi-file builds summed), largest first.^[5][6][7][8] The separate vision projector (mmproj, 3.4 GB) is only needed for image input and is not included.^[5]

Mistral's own checkpoint is stored in FP8 (the quantization_config in config.json), so the BF16 GGUF is twice its size.^[3][8] The NVFP4 checkpoint (610.0 GB) targets Blackwell GPUs and runs on vLLM and SGLang; Mistral says it keeps the same performance with a smaller footprint (a developer claim we have not tested).^[2][7] Builds whose names start with "UD-" are Unsloth's dynamic quants.

Which build should you pick? On eight 96 GB GPUs, Q4_K_M or NVFP4 is the usual balance. On eight 80 GB GPUs, UD-Q3_K_XL fits with room to spare. On a 512 GB Mac Studio, UD-IQ3_XXS or Q2_K. Treat 1-bit and 2-bit builds as a last resort.

Context length adds memory (the KV cache)

On top of the weights, the engine keeps a key/value cache for the conversation. Mistral Large 4 uses multi-head latent attention, which stores a small compressed cache: from config.json, 576 values per token for each of the 63 layers that keep their own cache (9 layers reuse another layer's), plus a 128-value indexer key on 44 layers.^[3] At BF16 that is about 83,840 bytes per token. Mistral has not published a per-token figure, so treat this as our estimate; your engine may store the cache differently.

Even the full 1,048,576-token window adds about 88 GB by this estimate, which is small next to the weights.

Which GPU setups and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the total memory of the setup.^{[9][10][11][12][13]}

Column headers show weights + an 8,192-token KV cache (our estimate from config.json). Multi-GPU rows assume the model is split evenly across all GPUs (tensor parallel). NVFP4 needs Blackwell GPUs (B200, B300, RTX PRO 6000), so ignore that column on H100/H200 and Mac.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own buffers or a longer context can push it over. "No" means it won't fit. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat that row as optimistic.

Our recommendations

These follow from the tables above.

The model card's vLLM example serves the official checkpoint with --tensor-parallel-size 8, so plan on eight GPUs for the FP8 weights.^[2]

How it compares with other giant open models

4-bit file sizes come from each model's Hugging Face GGUF repo, the same data as our comparison table.^[14]

†Unsloth UD-Q4_K_XL (no Q4_K_M build exists); the others are Q4_K_M. Mistral Large 4 activates far more parameters per token than the other two, so expect it to be slower per token on the same hardware.

How to run it

For a server, the model card recommends vLLM with Mistral's Docker image vllm/vllm-openai:mistral-large-4 and --tensor-parallel-size 8, with the mistral tool-call and reasoning parsers; SGLang and Transformers are also supported.^[2] The card recommends temperature 1.0 and top-p 0.95 at every reasoning effort, and reasoning effort can be set to low, medium or high (the default).^[2] For llama.cpp, download a GGUF build from ggml-org's or Unsloth's repo.^[5][6]

Published speeds (Hugging Face's own tests, not ours): Q2_K on a single M3 Ultra Mac Studio ran at about 15 tokens per second for generation and about 90 tokens per second for prompt processing. On four RTX PRO 6000 cards, Hugging Face reports about 10 tokens per second with NVFP4 at a 32K context, and about 30 tokens per second with IQ2_XXS at short contexts, falling to about 3 tokens per second at very long contexts. The post does not give every setting (such as any offloading to system RAM), so treat these as rough guides.^[15]

If the model is bigger than your VRAM, llama.cpp can keep some experts in system RAM; see our MoE expert offload guide. With 49B active parameters per token it will be slow this way.

If you only want to try it, Mistral also serves it on its own API as mistral-large-4.^[2]

Cheapest GPUs to rent for Mistral Large 4

Eight 96 GB RTX PRO 6000s hold the Q4_K_M or NVFP4 build; eight B200s hold the official FP8 weights. On-demand prices from Runpod's pricing page (checked October 31, 2026; Community Cloud is cheaper but runs on shared hosts):^[12]

Prices checked on 2026-10-31. They change often, so confirm on the provider's page. Runpod lists per-GPU prices; the 8-GPU rows are that price times eight. Whether an 8-GPU pod is available at any moment varies. Rent on Runpod Referral link

See all GPU rental prices for more providers and multi-GPU setups.

Frequently asked questions

Can I run Mistral Large 4 on one GPU?

No. Even the smallest build, Unsloth's 1-bit UD-IQ1_S, is 233.0 GB, more than any single GPU in our tables.^[6] You need a multi-GPU server, a 512 GB Mac Studio for the 3-bit builds, or a lot of system RAM with expert offloading.

How much VRAM does Mistral Large 4 need at 4-bit?

The Q4_K_M GGUF is 634.8 GB, and an 8K-token context adds about 0.7 GB by our estimate, so about 635.5 GB before engine overhead.^[5][3] Eight 96 GB GPUs (768 GB) hold it with room to spare; eight 80 GB GPUs (640 GB) are too tight.

Will it run on a 512 GB Mac Studio?

Yes, at 3-bit or lower. UD-IQ3_XXS (423.5 GB) fits with room for context, and UD-Q3_K_XL (492.7 GB) is tight.^[6][13] The 4-bit builds (604.8 GB and up) do not fit. Hugging Face reports about 15 tokens per second generating with Q2_K on a single M3 Ultra.^[15]

What is "Le Chonk"?

It is Mistral's nickname for Mistral Large 4, used on its blog and model card.^[4][2]

Is it really open source?

The weights are open under the Mistral Ecosystem Open-Weight License, a modified MIT license for commercial and non-commercial use. Running a model-as-a-service business requires a separate commercial license from Mistral.^[2]

How we calculated

• Weights are the exact file sizes on Hugging Face, checked October 31, 2026, with multi-file builds added up.^[5][6][7][8] The vision projector is not included.
• KV cache is our estimate from config.json (63 cached layers × 576 latent values plus a 128-value indexer key on 44 layers, at 2 bytes).^[3] Mistral has not published a figure.
• Device memory is from NVIDIA, Apple and Runpod spec and pricing pages. Multi-GPU rows add up memory and assume an even split.
• Rental prices are the listed on-demand Runpod per-GPU rates from our GPU price data, checked October 31, 2026, times eight.^[12]
• Not included: engine working buffers, the OS and display. These vary by engine and settings, so leave headroom.

Sources

All sources checked on 2026-10-31.

1. Hugging Face, mistralai/Mistral-Large-4-1T-A52B (license, repo and release commit dates). https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B
2. Mistral Large 4 model card (1T total / 49B and 52B active, experts, 1M context, license terms, vLLM command, sampling settings, NVFP4 note). https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/blob/main/README.md
3. Hugging Face, config.json (layers, experts, latent attention, shared KV layers, indexer, FP8 quantization config). https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/blob/main/config.json
4. Mistral AI, "Mistral Large 4 (Le Chonk) is now open weight" (October 29, 2026). https://mistral.ai/news/mistral-large-4-oss/
5. Hugging Face, ggml-org/Mistral-Large-4-1T-A52B-GGUF file list. https://huggingface.co/ggml-org/Mistral-Large-4-1T-A52B-GGUF/tree/main
6. Hugging Face, unsloth/Mistral-Large-4-1T-A52B-GGUF file list. https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF/tree/main
7. Hugging Face, mistralai/Mistral-Large-4-1T-A52B-NVFP4 file list. https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B-NVFP4/tree/main
8. Hugging Face, mistralai/Mistral-Large-4-1T-A52B file list (official FP8 checkpoint, 54 files). https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B/tree/main
9. NVIDIA, H100 (80 GB). https://www.nvidia.com/en-us/data-center/h100/
10. NVIDIA, H200 (141 GB). https://www.nvidia.com/en-us/data-center/h200/
11. NVIDIA, RTX PRO 6000 Blackwell (96 GB). https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000/
12. Runpod, GPU pricing (B200 180 GB, B300 288 GB, per-GPU prices; checked October 31, 2026). https://www.runpod.io/pricing
13. Apple, Mac Studio tech specs (up to 512 GB unified memory). https://www.apple.com/mac-studio/specs/
14. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
15. Hugging Face blog, "Welcome Le Chonk: Mistral Large 4 on Hugging Face" (October 29, 2026; llama.cpp quants and measured speeds). https://huggingface.co/blog/mistral-large4