Qwen3.8-35B-A3B VRAM: can an RTX 3060 12GB run it?

This is not an official Alibaba Qwen3.8-35B release. It is Empero's community distillation of Qwen3.8 teacher traces into the Qwen3.6-35B-A3B Mixture-of-Experts student (empero-ai/Qwen3.8-35B-A3B-Distill).^[1][2] Hugging Face lists about 35.1 billion parameters (safetensors total), Apache 2.0, with a repo created on September 16, 2026.^[1] The card describes ≈3B active parameters per token, 40 layers (30 Gated DeltaNet + 10 full attention), 256 experts with 8 routed per token, and a native context of 262,144 tokens.^[2][3]

Because it is an MoE, only about 3B parameters work on each token, which is why it can feel fast. But every expert still has to live somewhere (VRAM, system RAM, or SSD), so memory decides how it runs for you — especially on a 12 GB gaming GPU.

Can an RTX 3060 12GB run Qwen3.8-35B-A3B?

Short answer: not fully in VRAM. Q4_K_M alone is 21.7 GB.^[4] NVIDIA rates the GeForce RTX 3060 desktop card at 12 GB.^[5] Even IQ2_M (12.6 GB) is bigger than the card before you add KV cache or engine buffers.^[4]

What the viral posts are doing: they offload MoE experts to system RAM. A widely shared third-party report from @DogukanUrker (September 16, 2026) describes Empero's Qwen3.8-35B-A3B on a single RTX 3060 with 12 GB VRAM + 16 GB RAM, full 262K context, about ~50 tok/s decode and ~550 tok/s prefill.^[13] A follow-up summary of that setup names llama.cpp with roughly -ngl 99 --n-cpu-moe 26 -c 262144, Q4_K_M (~21.7 GB file), and Q8_0 KV cache types — again a community report, not something we measured.^[14]

Practical checklist for a 12 GB + 16 GB RAM box:

1. Use a recent llama.cpp build with Qwen3.6 / Gated DeltaNet MoE support (older builds fail to load the architecture).^[4]
2. Download Q4_K_M (21.7 GB) if you have enough free disk and ~16 GB+ RAM for expert offload, or a smaller quant if RAM is tight.^[4]
3. Keep attention / hot layers on the GPU (-ngl high) and push experts to the CPU with --n-cpu-moe (the shared report used 26).^[14]
4. Expect speed to depend on RAM bandwidth, not just the 3060. Treat ~50 tok/s as a third-party claim on one machine.^[13]
5. If you want the whole model in VRAM with headroom, rent or buy a 24 GB card instead (see rental section below).

Memory needed at each quantization

Quantization stores each weight in fewer bits. File sizes below are the actual GGUF downloads from Empero's repo (1 GB = 1,000,000,000 bytes), checked October 28, 2026.^[4] The vision projector (mmproj, 0.90 GB) is separate and only needed for image input; the distillation was text-only and vision was not evaluated.^[2][4]

Empero's own guidance (modest context, weight-size only): IQ2_M / Q2_K → about 16 GB VRAM or system RAM; IQ3 → 20–24 GB; IQ4_XS / Q4_K_M → 24 GB for a full GPU load; Q5/Q6 → 32 GB; Q8_0 → 48 GB; BF16 → 80 GB+.^[4] Those lines match our fit table once a small KV cache is added.

Context length adds memory (the KV cache)

From config.json, only 10 of 40 layers are full attention; the other 30 are Gated DeltaNet (linear attention) with a fixed-size state.^[3] With 2 KV heads and a head size of 256, an FP16 cache takes 2 × 10 × 2 × 256 × 2 bytes = 20,480 bytes per token:

So even the full 262,144-token context needs only about 5.4 GB of KV cache on top of the weights (our calculation from the config, not a published figure). The fixed linear-attention state is not included above.

Which GPUs and Macs it fits on (whole model in memory)

Each cell compares the weights plus an 8K-token KV cache with the device's memory.^{[5][6][7][8][9][10][11]} Offloading is covered above; this table is for full GPU / unified-memory residency.

Column headers show weights + 8,192-token KV cache (10 attention layers, FP16). Fits = whole model in that memory; MoE expert offload to system RAM (--n-cpu-moe) is covered in the RTX 3060 section.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%. "No" means it won't fit on that device alone. The 90% line is our rule of thumb. On a Mac, leave headroom for macOS.

Our recommendations

How it compares with similar models

4-bit file sizes and KV cache per token come from each model's Hugging Face GGUF repo and config.json, the same data as our comparison table.

*Counts only full-attention layers. †Flint's 4-bit file is Unsloth UD-Q4_K_XL. This Distill is smaller than Flint at 4-bit and activates a similar ~3B vs Flint's ~2.5B, but it is a community student, not the frontier Qwen3.8 teachers.^[2]

How to run it

With llama.cpp or LM Studio, download a GGUF from empero-ai/Qwen3.8-35B-A3B-Distill-GGUF — for example Qwen3.8-35B-A3B-Q4_K_M.gguf (21.7 GB) on a 24 GB card.^[4] Empero recommends temperature=0.6, top_p=0.95, top_k=20, a generous max tokens (every answer opens with a thinking block), and a recent engine with Qwen3.6 / Gated DeltaNet MoE support.^[2][4]

For Transformers / vLLM / SGLang, use the safetensors repo; you need a recent transformers release plus Gated DeltaNet kernels (flash-linear-attention and a matching causal_conv1d) or linear-attention layers fall back to slow PyTorch ops.^[2]

Cheapest GPU to rent for Qwen3.8-35B-A3B

A 24 GB card runs Q4_K_M fully in VRAM; 32 GB is more comfortable for Q5/Q6 or long context. On-demand Secure Cloud prices listed on Runpod's pricing page (checked October 31, 2026):^[12]

Prices checked on 2026-10-28. They change often, so confirm on the provider's page. Rent on Runpod Referral link

See all GPU rental prices for more providers. A marketplace option (often cheaper for short runs) is Vast.ai (affiliate link).

Frequently asked questions

Can I run Qwen3.8-35B-A3B on an RTX 3060 12GB?

Not as a full VRAM load: Q4_K_M is 21.7 GB and even IQ2_M is 12.6 GB.^[4][5] With ~16 GB system RAM and llama.cpp MoE offload (--n-cpu-moe), third-party reports claim ~50 tok/s decode — treat those as community measurements.^[13][14]

Is this the official Qwen3.8 35B?

No. It is Empero's distillation of Qwen3.8 teachers into the Qwen3.6-35B-A3B MoE student.^[1][2] Base weights and license come from Qwen3.6-35B-A3B (Apache 2.0).^[2]

Will it fit a 24 GB RTX 3090 or 4090?

Yes at Q4_K_M for short-to-moderate contexts (about 21.9 GB with 8K cache).^[4][7][8] The full 256K context at Q4_K_M is about 27.1 GB, which needs a 32 GB card or a smaller quant / quantized KV cache.

How much VRAM does the full 256K context need?

The FP16 KV cache for 262,144 tokens is about 5.4 GB (10 attention layers only).^[3] With Q4_K_M weights that is about 27.1 GB before engine overhead.

Can a Mac run it?

A 32 GB Mac can hold Q4_K_M with a moderate context; a 64 GB Mac can hold Q6_K (29.2 GB) or Q8_0 (37.8 GB) more comfortably.^[4][11] Leave headroom for macOS.

How we calculated

• Weights are the exact GGUF file sizes on Hugging Face, checked October 28, 2026.^[4] The mmproj vision projector is not included.
• KV cache is calculated from config.json for the 10 full-attention layers at FP16.^[3] The fixed Gated DeltaNet state is not in the tables.
• Device memory is from NVIDIA and Apple spec pages. We compare in decimal GB.
• Rental prices are listed on-demand Runpod Secure Cloud rates from our GPU price data, checked October 31, 2026.^[12]
• 3060 speeds are third-party reports from X, labelled as such; we have not reproduced them.^[13][14]
• Not included: engine working buffers, OS and display. Leave headroom.

Sources

All sources checked on 2026-10-28.

1. Hugging Face, empero-ai/Qwen3.8-35B-A3B-Distill (parameter count, license, repo date; checked October 28, 2026). https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill
2. Hugging Face, Qwen3.8-35B-A3B-Distill model card (distill provenance, ≈3B active, MoE layout, sampling, limitations). https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill
3. Hugging Face, config.json (layer_types, KV heads, head_dim, experts, context). https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill/blob/main/config.json
4. Hugging Face, empero-ai/Qwen3.8-35B-A3B-Distill-GGUF file list and card (exact GGUF sizes, fit guidance, llama.cpp notes; checked October 28, 2026). https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF
5. NVIDIA, GeForce RTX 3060 family (12 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3060-3060ti/
6. NVIDIA, GeForce RTX 5080 (16 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080/
7. NVIDIA, GeForce RTX 3090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
8. NVIDIA, GeForce RTX 4090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
9. NVIDIA, GeForce RTX 5090 specs (32 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
10. NVIDIA, RTX 6000 Ada Generation (48 GB). https://www.nvidia.com/en-us/products/workstations/rtx-6000/
11. Apple, MacBook Pro tech specs (unified memory options). https://www.apple.com/macbook-pro/specs/
12. Runpod, GPU pricing (checked October 23, 2026). https://www.runpod.io/pricing
13. @DogukanUrker on X, RTX 3060 12GB + 16GB RAM report (~50 tok/s decode / ~550 prefill, 262K context; third-party, September 16, 2026). https://x.com/DogukanUrker/status/2100346050473074805
14. Community summary of that 3060 setup (Q4_K_M, --n-cpu-moe 26, llama-server flags; third-party). https://x.com/grok/status/2109138089000006039