Qwen3.8-Flash-Next VRAM requirements: can you run it locally?

Qwen3.8-Flash-Next is Alibaba Qwen's experimental preview of the architecture that will underpin Qwen4. Its Hugging Face repo was created on August 24, 2026, it uses the Qwen Community License 1.0, and Hugging Face counts 179,999,981,459 parameters.^[1] The model card explains the gap with the "125B" you see on X: the language model is 125B with 6B activated, plus a 51B n-gram embedding and a 4B multi-token-prediction (MTP) layer.^[2] It has 48 layers, 512 experts with 10 routed + 1 shared per token, a vision encoder, and a native context of 262,144 tokens, extensible to about 1M with YaRN.^[2][3] Qwen reports, for example, 62.5 on SWE-bench Pro against 61.7 for Qwen3.8-27B; these are developer-reported scores.^[2]

It is the model people are asking about this week: X posts mentioning it jumped from about 280 a day to more than 1,500 on October 26 (X counts API, checked October 27, 2026), driven by reports of it running on a single 16–24 GB gaming GPU. Those reports are real, but they depend on offloading, so this page separates what fits fully in memory from what runs with offloading.

Because it is an MoE, only about 6B parameters work on each token, which is why it is fast. But every expert and the n-gram table still have to live somewhere (VRAM, unified memory, system RAM or SSD), so memory decides how it runs for you.

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files fit in less memory, at some cost in quality, and the cost grows below about 4 bits. File sizes below are the actual GGUF downloads on Hugging Face (1 GB = 1,000,000,000 bytes, multi-part files summed), largest first. Standard quants are from bartowski's repo; the "GSQ-RCO" rows are ISTA-DASLab's builds, which are the most-downloaded GGUFs of this model (about 7.3 million downloads).^[4][6] The vision projector (mmproj, 0.9 GB) and the optional MTP draft file (2.2–7.8 GB) are separate and not included.^[4]

Unsloth's repo has its own dynamic builds: UD-Q4_K_XL is 111.3 GB, UD-Q3_K_XL 90.0 GB, UD-Q2_K_XL 78.9 GB and UD-IQ1_S 72.5 GB.^[5] Ollama lists qwen3.8-flash-next:125b-a6b-q4_K_M at 120 GB, 125b-a6b-q8_0 at 189 GB, 125b-a6b-bf16 at 355 GB and an MLX 125b-a6b-nvfp4 build at 105 GB.^[7] NVIDIA's own NVFP4 checkpoint (for Blackwell GPUs, vLLM/SGLang) is 132.7 GB of safetensors.^[17]

Notice how little the sizes drop below 4-bit: the 1-bit IQ1_M is still 72.0 GB.^[4] That is the 51B n-gram embedding and the many small experts at work, and it is why people offload instead of quantizing harder.

Context length adds memory (the KV cache)

Qwen3.8-Flash-Next is cheap here. From config.json, only 12 of its 48 layers are Qwen Sparse Attention layers with a key/value cache that grows; the other 36 are Gated DeltaNet (linear attention) layers with a fixed-size state.^[3] With 2 KV heads and a head size of 256, an FP16 cache takes 2 × 12 × 2 × 256 × 2 bytes = 24,576 bytes per token:

So even the full 262,144-token context needs only about 6.4 GB of KV cache (our calculation from the config, not a published figure). Two small items are not in the table: the fixed Gated DeltaNet state, and the QSA indexer's own cache (1 key head of size 128, compressed 4×), which Qwen has not published a size for.^[2][3]

Which GPUs and Macs it fits on (whole model in memory)

Each cell compares the weights plus an 8K-token KV cache with the device's memory.^{[9][10][11][12][13][14][15]} Two new machines are included. NVIDIA's RTX Spark laptops (on sale since October 16, 2026) have up to 128 GB of unified memory.^[12] AMD's next Ryzen AI Halo developer platform (announced October 12, 2026) has up to 192 GB, of which up to 160 GB can be allocated to the GPU, so we list it at 160 GB.^[14]

Column headers show weights + 8,192-token KV cache (12 attention layers, FP16). Fits = whole model in that memory; offloading experts or n-gram tables to system RAM or SSD is covered in 'How to run it'.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own working memory or a longer context can push it over. "No" means the whole model won't fit in that memory alone. The 90% line is our own rule of thumb, not a vendor figure. On unified-memory machines the operating system also needs some memory, so treat those rows as optimistic.

Our recommendations

These follow from the tables above.

Running it on a 16–32 GB gaming GPU (offloading)

This is what the viral posts are about. Because only about 6B parameters are active per token, engines can keep the attention layers and hot experts on the GPU and pull the rest from system RAM, or even straight from an NVMe SSD. Qwen's own card notes the n-gram embedding is "more amenable to offloading" than the experts.^[2] Speed then depends on your RAM and SSD bandwidth, not just your GPU. Third-party reports (not tested by us):

• RTX 4090: the Lean-QwenFlash project reports 262K context with about 15.6 GB of VRAM and 8 GB of system RAM, streaming experts from NVMe, at about 20 tok/s decode and 1,627 tok/s prefill.^[19]
• RTX 3090 + 64 GB RAM: a user reports about 100 tok/s with Strata and the GSQ-RCO IQ3_S build (83.6 GB file).^[20]
• RTX 5090 + 96 GB DDR5: a user reports about 100 tok/s at 256K context.^[21]
• Mac Studio M4 Max 64 GB: a user ran an 83.3 GB Q3_K_S file streamed from an external NVMe drive at a median of about 29 tok/s.^[22]

There is also a pruned and re-quantized route. AgentionAI's Gyro builds say Gyro-S needs 27.64 GiB of GPU memory (32 GB cards), Gyro-M 34.98 GiB and Gyro-L 50.21 GiB, with a coding-only variant at 19.92 GiB for 24 GB cards.^[8] The download files are larger (Gyro-S is 58.5 GB) and upstream llama.cpp cannot load them yet; they need Strata or the authors' forks.^[8] The quality figures on that card are the authors' own.

How it compares with similar models

4-bit file sizes and KV cache per token come from each model's Hugging Face GGUF repo and config.json, the same data as our comparison table.^[18]

*Counts only the 12 full-attention layers. †Ornith's 4-bit file is Unsloth UD-Q4_K_XL; the others are Q4_K_M. If Qwen3.8 27B already fits your GPU, it is the simpler choice; Flash-Next pays off when you have 96 GB+ of fast memory or a good offloading setup.

How to run it

With Ollama: ollama run qwen3.8-flash-next:125b-a6b-q4_K_M pulls the 120 GB build.^[7] With llama.cpp or LM Studio, download a GGUF from bartowski, Unsloth or ISTA-DASLab (for example the 83.6 GB GSQ-RCO IQ3_S on a 96 GB card).^[4][5][6] For servers, the model card lists Transformers, vLLM, SGLang and TokenSpeed, and recommends temperature 1.0, top-p 0.95, top-k 20 in thinking mode (0.7 / 0.8 / 20 in non-thinking mode).^[2] NVIDIA's NVFP4 checkpoint targets Blackwell GPUs.^[17]

Cheapest GPU to rent for Qwen3.8-Flash-Next

A 96 GB RTX Pro 6000 runs the 3-bit builds; an H200 (141 GB) runs Q4_K_M with the full 256K context; Q8_0 needs a B300 or two H200s. On-demand prices listed on Runpod's pricing page (checked October 31, 2026; Community Cloud is cheaper but runs on shared hosts):^[16]

Prices checked on 2026-10-27. They change often, so confirm on the provider's page. Multi-GPU row is Runpod's listed per-GPU price times two. Rent on Runpod Referral link

See all GPU rental prices for more providers and multi-GPU setups.

Frequently asked questions

Can I run Qwen3.8-Flash-Next on an RTX 4090 or 3090?

Not fully in VRAM: even the smallest standard build is 66.4 GB.^[6] It does run with offloading. Third-party reports show about 20 tok/s on a 4090 streaming experts from NVMe, and about 100 tok/s on a 3090 with 64 GB of system RAM.^[19][20] Pruned builds such as Gyro-Coder (19.92 GiB on the GPU) also target 24 GB cards, but need a non-upstream engine.^[8] See Qwen3.8-Flash-Next Coder VRAM for every Coder build by GPU tier.

Is it 125B or 180B parameters?

Both. The card says 125B in the language model (6B active), plus a 51B n-gram embedding and a 4B MTP layer; Hugging Face counts all of it, 180.0B.^[1][2] Memory needs follow the actual file sizes, which include the n-gram table.

Will it run on a 128 GB Mac, DGX Spark or RTX Spark laptop?

Yes, at IQ4_XS (97.7 GB), with room for the full 256K context (about 104.1 GB).^[4][11][12] Q4_K_M (119.6 GB) is tight at 94% of 128 GB, so expect trouble once the OS and engine take their share.

Does the new Ryzen AI Halo run it?

AMD says the Ryzen AI Halo platform can allocate up to 160 GB of its 192 GB to the GPU.^[14] That holds Q4_K_M (about 126.0 GB with the full 256K context) or Q5_K_M (134.7 GB) at short contexts, by our calculation; we have not seen a published benchmark.

How much VRAM does the full 256K context need?

The FP16 KV cache for 262,144 tokens is about 6.4 GB, because only 12 layers keep a full cache.^[3] With Q4_K_M weights that is about 126.0 GB before engine overhead.

How we calculated

• Weights are the exact GGUF file sizes on Hugging Face, checked October 27, 2026, with multi-part builds added up.^[4][5][6] Vision projector and MTP draft files are not included.
• KV cache is calculated from config.json for the 12 attention layers at FP16.^[3] The fixed linear-attention state and the indexer cache are not in the tables.
• Device memory is from NVIDIA, AMD and Apple pages. We compare in decimal GB, so the results lean conservative. Ryzen AI Halo uses AMD's stated 160 GB GPU allocation, not the full 192 GB.
• Rental prices are the listed on-demand Runpod rates from our GPU price data, checked October 31, 2026.^[16]
• Offloading speeds are third-party reports from X, labelled as such; we have not reproduced them.
• Not included: engine working buffers, the OS and display. Leave headroom.

Sources

All sources checked on 2026-10-28.

1. Hugging Face, Qwen/Qwen3.8-Flash-Next (parameter count, license, repo date, downloads; checked October 27, 2026). https://huggingface.co/Qwen/Qwen3.8-Flash-Next
2. Hugging Face, Qwen3.8-Flash-Next model card (125B/6B active, 51B n-gram, 4B MTP, layout, context, benchmarks, sampling). https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/README.md
3. Hugging Face, config.json (layer types, KV heads, head size, indexer, context). https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/config.json
4. Hugging Face, bartowski/Qwen3.8-Flash-Next-GGUF file list. https://huggingface.co/bartowski/Qwen3.8-Flash-Next-GGUF/tree/main
5. Hugging Face, unsloth/Qwen3.8-Flash-Next-GGUF file list. https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF/tree/main
6. Hugging Face, ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF file list and downloads. https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF/tree/main
7. Ollama, qwen3.8-flash-next tags. https://ollama.com/library/qwen3.8-flash-next/tags
8. Hugging Face, agentionai/Qwen3.8-Flash-Next-Gyro-GGUF model card (GPU memory per build, runtime support; developer claims). https://huggingface.co/agentionai/Qwen3.8-Flash-Next-Gyro-GGUF
9. NVIDIA, GeForce RTX 5090 specs (32 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
10. NVIDIA, RTX PRO 6000 Blackwell (96 GB). https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000/
11. NVIDIA, DGX Spark (128 GB unified memory). https://www.nvidia.com/en-us/products/workstations/dgx-spark/
12. NVIDIA Newsroom, "NVIDIA RTX Spark PCs Available Today" (up to 128 GB unified memory; October 16, 2026). https://nvidianews.nvidia.com/news/nvidia-rtx-spark-pcs-available-today-a-new-beginning-for-windows-pcs
13. NVIDIA, H200 (141 GB). https://www.nvidia.com/en-us/data-center/h200/
14. AMD blog, "Meet the Next AMD Ryzen AI Halo" (up to 192 GB unified memory, up to 160 GB allocatable to the GPU; October 12, 2026). https://www.amd.com/en/blogs/2026/meet-the-next-amd-ryzen-ai-halo.html
15. Apple, Mac Studio tech specs (256 GB and 512 GB unified memory). https://www.apple.com/mac-studio/specs/
16. Runpod, GPU pricing (checked October 23, 2026). https://www.runpod.io/pricing
17. Hugging Face, nvidia/Qwen3.8-Flash-Next-NVFP4 (file list). https://huggingface.co/nvidia/Qwen3.8-Flash-Next-NVFP4/tree/main
18. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
19. @dextune on X, Lean-QwenFlash on RTX 4090 (third-party report, October 26, 2026). https://x.com/dextune/status/2114559519640744029
20. @snajper47 on X, RTX 3090 + 64 GB RAM with Strata IQ3_S (third-party report, October 25, 2026). https://x.com/snajper47/status/2114405721840529815
21. @rusanasol on X, RTX 5090 + 96 GB DDR5 at 256K context (third-party report, October 26, 2026). https://x.com/rusanasol/status/2114763507681890693
22. @User_TiRO on X, Mac Studio M4 Max 64 GB with NVMe streaming (third-party report, October 25, 2026). https://x.com/User_TiRO/status/2114211099730170097