Qwen3.5-122B-A10B VRAM requirements: every GGUF size and which GPUs fit

Qwen3.5-122B-A10B is the middle-to-large model in Alibaba's Qwen3.5 family, a mixture-of-experts (MoE) model with image and video input, released on Hugging Face on February 24, 2026 under the Apache 2.0 license. The model card says 122B parameters in total and 10B activated per token; Hugging Face counts 125,086,497,008 parameters in the checkpoint, which also holds the vision encoder and a multi-token-prediction (MTP) layer.^[1][2] It is the most downloaded large Qwen model we don't already cover: Hugging Face counts about 994,398 downloads in the last month for the main repo, plus 554,366 for Qwen's GPTQ-Int4 version and 154,277 for the FP8 version (checked November 2, 2026).^[1][17]

Its config.json shows 48 layers in a repeating pattern of three Gated DeltaNet (linear-attention) layers and one Gated Attention layer, 256 experts with 8 routed plus 1 shared per token, and a context of 262,144 tokens; the model card says it can be extended to about 1,010,000 tokens.^[3][2]

Because it is an MoE, only about 10B parameters work on each token, so it generates much faster than a dense 122B model. But all of the weights still have to sit in memory (or be offloaded to system RAM).

Memory needed at each quantization

Qwen does not publish its own GGUF files for this model. Below are the builds in Unsloth's GGUF repo, using the exact file sizes on Hugging Face (1 GB = 1,000,000,000 bytes, multi-part files added together), largest first.^[4] Builds whose names start with "UD-" are Unsloth's dynamic quants. The vision projector (mmproj, 0.9 GB at F16) is a separate file you only need for image input.^[4]

Which build should you pick? Q4_K_M or UD-Q4_K_XL is the usual balance of size and quality. On an 80 GB GPU, Q4_K_S. On a 64 GB Mac, UD-Q3_K_XL or Q3_K_M. With 128 GB or more, Q6_K or UD-Q6_K_XL. Unsloth's own guide lists about 60 GB of total memory for 3-bit, 70 GB for 4-bit, 106 GB for 6-bit, 132 GB for 8-bit and 245 GB for BF16 (RAM + VRAM, or unified memory).^[6]

Context length adds memory (the KV cache)

Only the 12 Gated Attention layers keep a cache that grows with context. From config.json (2 KV heads, head size 256),^[3] an FP16 cache takes 2 × 12 × 2 × 256 × 2 bytes = 24,576 bytes per token, about 0.2 GB at 8K, 0.81 GB at 32K, 3.22 GB at 128K and 6.44 GB at the full 262,144 tokens. The 36 Gated DeltaNet layers keep a fixed-size state that doesn't grow with context; it is not included here.

That is why this model is cheap to run at long context compared with older dense or full-attention models. Qwen advises keeping at least 128K of context to preserve its thinking ability, and suggests reducing context if you hit out-of-memory errors.^[2]

Which GPUs and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the total memory of the setup.^{[7][8][9][10][11][12][13][14]}

Column headers show weights + 8,192-token KV cache (12 attention layers, FP16). Multi-GPU rows add up memory and assume the model is split evenly across the GPUs. Fits = whole model in that memory; MoE expert offload to system RAM is covered in 'How to run it'.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own buffers or a longer context can push it over. "No" means it won't fit. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat those rows as optimistic.

Our recommendations

These follow from the file sizes and the 90% rule.^[4]

How it compares with similar models

4-bit sizes come from each model's Hugging Face GGUF repo, the same data as our comparison table.^[16]

Ornith-1.5-122B uses the same Qwen3.5 MoE architecture and the same parameter count, so its memory needs are almost the same. Newer Qwen models on this site include Qwen3.8 Flash Next, Qwen3.8 35B-A3B and Qwen3.8 27B.

How to run it

With llama.cpp, LM Studio or Ollama, download a GGUF build from Unsloth's repo.^[4] The large builds are split into several files; llama.cpp loads the first part and finds the rest. Ollama's qwen3.5:122b tag (Q4_K_M) is listed at 81 GB, including image input.^[5] If the model is bigger than your VRAM, llama.cpp can keep some experts in system RAM; see our MoE expert offload guide. With about 10B active parameters, that works better than with dense models, but expect lower speed than an all-GPU setup. We have not measured its speed.

For serving with vLLM or SGLang, Qwen's card shows 8-GPU tensor-parallel commands at the full 262,144-token context, an MTP speculative-decoding option, and a text-only flag that skips the vision encoder to free memory for KV cache.^[2] Qwen also publishes FP8 and GPTQ-Int4 versions for those engines.^[17]

Cheapest GPUs to rent for Qwen3.5-122B-A10B

On-demand prices from Runpod's pricing page for single GPUs that hold Q4_K_M with an 8K context (checked November 2, 2026; Community Cloud is cheaper but runs on shared hosts):^[15]

Prices checked on 2026-11-02. They change often, so confirm on the provider's page. Single GPUs that hold Q4_K_M + 8K (76.7 GB) under our 90% line. An 80 GB A100 holds Q4_K_S (71.9 GB with 8K, 89.9%) but Q4_K_M is tight there (95.9%). Whether a GPU is available at any moment varies. Rent on Runpod Referral link

See all GPU rental prices for more providers and multi-GPU setups, or try the VRAM calculator with your own quant and context.

Frequently asked questions

How much VRAM does Qwen3.5-122B-A10B need at 4-bit?

About 76.7 GB: the Q4_K_M weights (76.5 GB) plus an 8K-token FP16 KV cache (0.2 GB), before engine overhead.^[4][3] Q4_K_S is 71.7 GB and the MXFP4_MOE build is 74.7 GB.^[4]

Can I run Qwen3.5-122B-A10B on one GPU?

On one 96 GB RTX PRO 6000, yes at 4-bit. On an 80 GB A100 or H100, Q4_K_S fits by our rule and Q4_K_M is tight. On a 24 GB or 32 GB consumer card, no: the smallest build, UD-IQ1_M, is 34.2 GB.^[4] You can still split it between a consumer GPU and system RAM with expert offloading, at lower speed.

Will it run on a 64 GB or 128 GB Mac?

On 64 GB, the 3-bit UD-Q3_K_XL (57.0 GB) fits by our rule at 89%. On 128 GB, everything up to UD-Q6_K_XL (112.4 GB) fits; Q8_0 (129.9 GB) does not.^[4][13] macOS keeps part of unified memory for itself by default, so you may need to raise the GPU memory limit; see our Mac guide.

How much does the full 256K context add?

At FP16, the KV cache for 262,144 tokens is about 6.44 GB, because only 12 of the 48 layers keep a growing cache.^[3]

Is it open source?

The weights are released under the Apache 2.0 license, which allows commercial use.^[1]

How we calculated

• Weights are the exact GGUF file sizes on Hugging Face, checked November 2, 2026, with multi-part files added up.^[4]
• KV cache is calculated from config.json for the 12 Gated Attention layers. We assume an FP16 cache. The fixed Gated DeltaNet state is not included.^[3]
• Device memory is from NVIDIA and Apple spec pages. Multi-GPU rows add up memory and assume an even split.
• Rental prices are the listed on-demand Runpod per-GPU rates from our GPU price data, checked November 2, 2026.^[15]
• Not included: engine working buffers, the vision projector, the OS and display. These vary by engine and settings, so leave headroom.

Sources

All sources checked on 2026-11-02.

1. Hugging Face, Qwen/Qwen3.5-122B-A10B (parameter count, license, repo date, 30-day downloads; checked November 2, 2026). https://huggingface.co/Qwen/Qwen3.5-122B-A10B
2. Qwen3.5-122B-A10B model card (122B total / 10B activated, layer layout, context length, serving commands). https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/README.md
3. Hugging Face, Qwen/Qwen3.5-122B-A10B config.json (48 layers, 12 full-attention layers, 2 KV heads, head size 256, 256 experts, 8 per token, 262,144 context). https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/config.json
4. Hugging Face, unsloth/Qwen3.5-122B-A10B-GGUF file list (checked November 2, 2026). https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF/tree/main
5. Ollama, qwen3.5 tags (qwen3.5:122b, 81 GB). https://ollama.com/library/qwen3.5/tags
6. Unsloth, Qwen3.5: how to run locally (hardware requirements table). https://unsloth.ai/docs/models/qwen3.5
7. NVIDIA, RTX 6000 Ada Generation (48 GB). https://www.nvidia.com/en-us/products/workstations/rtx-6000/
8. NVIDIA, A100 (80 GB). https://www.nvidia.com/en-us/data-center/a100/
9. NVIDIA, H100 (80 GB). https://www.nvidia.com/en-us/data-center/h100/
10. NVIDIA, RTX PRO 6000 Blackwell (96 GB). https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000/
11. NVIDIA, DGX Spark (128 GB unified memory). https://www.nvidia.com/en-us/products/workstations/dgx-spark/
12. NVIDIA, H200 (141 GB). https://www.nvidia.com/en-us/data-center/h200/
13. Apple, MacBook Pro tech specs (up to 128 GB unified memory). https://www.apple.com/macbook-pro/specs/
14. Apple, Mac Studio tech specs (256 GB and 512 GB unified memory). https://www.apple.com/mac-studio/specs/
15. Runpod, GPU pricing (per-GPU prices; checked November 2, 2026). https://www.runpod.io/pricing
16. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
17. Hugging Face, Qwen/Qwen3.5-122B-A10B-FP8 and Qwen/Qwen3.5-122B-A10B-GPTQ-Int4 (30-day downloads; checked November 2, 2026). https://huggingface.co/Qwen/Qwen3.5-122B-A10B-GPTQ-Int4