Nemotron 3.5 Bolt E4B VRAM requirements: can you run it locally?

Nemotron 3.5 Bolt E4B is NVIDIA's small on-device model, post-trained from Gemma 4 E4B. The card lists 8B total (4B effective) parameters; Hugging Face counts 7,996,156,490 for nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, created on 2026-10-05, under NVIDIA's own model license.^[1] Context is 64K tokens in and up to 64K out (65,536 in config.json).^[2] Hugging Face counts about 12,658 downloads in the last 30 days for the BF16 repo, about 97,982 for NVIDIA's 4 GB GGUF repo and about 350,142 for Unsloth's GGUFs (checked November 3, 2026).^[1]

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files fit on smaller GPUs, at some cost in quality, and the cost grows below about 4 bits. Below is every GGUF build we could source, using the exact file sizes on Hugging Face (1 GB = 1,000,000,000 bytes; split files added together), largest first.^[3] NVIDIA publishes one GGUF, a Q4_K_M in nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB (6.18 GB, despite the "4GB" name).^[4] Every other size is from Unsloth's unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF, which has no plain Q4_K_M or Q8_0 (its small "Q8_0" file is a draft model for speculative decoding, not counted).

Which quant should you pick? Q4_K_M is the usual balance of size and quality. Use Q5_K_M or Q6_K if you have the memory, and Q8_0 if you want results close to the original. Treat 2-bit and 3-bit builds as a last resort.

Context length adds memory (the KV cache)

On top of the weights, the engine keeps a key/value cache for every token. Its config.json has the same attention layout as Gemma 4 E4B: 42 layers, the last 18 reuse earlier layers' cache, leaving 4 global layers (2 KV heads of size 512) and 20 sliding-window layers (2 KV heads of size 256, 512-token window) with their own cache.^[2] At FP16 that is 16,384 bytes per token plus about 42 MB fixed for the sliding layers (20 × 1,024 × 2 × 256 × 2 × 2 bytes). llama.cpp (checked at commit 20a51f24, November 3, 2026) stores a separate V cache for Gemma 4 even when attention_k_eq_v is set: V starts from the same projection as K but is normalized differently and not rotated, so both are cached.^[15][16] Its sliding-window cache holds the window plus one batch of 512 tokens (the default n_ubatch), rounded up to a multiple of 256, so we count that size, not just the window.^[16]

Some engines can store the cache in 8-bit to roughly halve it.

Which GPUs and Macs it fits on

Pick your GPU or Mac and a context length to see which quantizations fit. The weights plus the KV cache are compared with the device's memory.^{[5][6][7][8][9][10][11][12]}

How to read it: "Fits" means weights + cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own working memory or a longer context can push it over. "No" means it won't fit on the GPU alone. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat Mac rows as optimistic.

Our recommendations

These follow from the file sizes and the 90% rule, at an 8K context.^[3]

How to run it

With llama.cpp, LM Studio or Ollama, download a GGUF file from the quant list above (for example NVIDIA's own Q4_K_M from nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB, or an Unsloth UD build).^[4][3] If the model is bigger than your VRAM, these tools can keep some layers in system RAM, but every token then waits on the slower memory, so expect much lower speed.

Cheapest GPU to rent for Nemotron 3.5 Bolt E4B

On-demand prices listed on Runpod's pricing page for cards that hold Q4_K_M with an 8K context (checked November 2, 2026; Community Cloud is cheaper but runs on shared hosts):^[13]

See all GPU rental prices for Lambda, Vast.ai and multi-GPU setups, or compare it with other models in the VRAM table.^[14]

Frequently asked questions

How much VRAM does Nemotron 3.5 Bolt E4B need at 4-bit?

About 6.4 GB for the Q4_K_M weights (6.2 GB) plus an 8K-token FP16 KV cache (0.18 GB), before engine overhead.^[3][2]

What is the smallest GPU that runs Nemotron 3.5 Bolt E4B?

In our list, an 8 GB card (RTX 4060) holds Q4_K_M with an 8K context under our 90% rule. Smaller cards need a lower-bit quant from the table above.^[3]

How much does the full 64K context add?

At FP16 the KV cache for 65,536 tokens is about 1.12 GB (32K tokens: 0.58 GB).^[2]

How we calculated

• Weights are the exact GGUF file sizes on Hugging Face, checked November 3, 2026.^[3]
• KV cache is calculated from config.json. We assume an FP16 cache.^[2]
• Device memory is from NVIDIA and Apple spec pages. We compare in decimal GB, so the results lean conservative.
• Rental prices are the listed on-demand Runpod rates, checked November 2, 2026.^[13]
• Not included: engine working buffers, the vision/audio projector, the OS and display. These vary by engine and settings, so leave headroom.

Sources

All sources checked on 2026-11-03.

1. Hugging Face, nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 (parameter count, license, repo date, 30-day downloads; checked November 3, 2026). https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
2. Hugging Face, nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 config.json (layers, KV heads, head size, context). https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16/blob/main/config.json
3. Hugging Face, unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF file list (exact GGUF sizes; checked November 3, 2026). https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF/tree/main
4. Hugging Face, nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB file list (exact GGUF sizes; checked November 3, 2026). https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB/tree/main
5. NVIDIA, GeForce RTX 4060 family (8 GB / 16 GB) https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4060-4060ti/
6. NVIDIA, GeForce RTX 3060 family (12 GB) https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3060-3060ti/
7. NVIDIA, GeForce RTX 3090 specs (24 GB) https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
8. NVIDIA, GeForce RTX 4090 specs (24 GB) https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
9. NVIDIA, GeForce RTX 5090 specs (32 GB) https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
10. NVIDIA, RTX 6000 Ada Generation (48 GB) https://www.nvidia.com/en-us/products/workstations/rtx-6000/
11. NVIDIA, A100 (80 GB) https://www.nvidia.com/en-us/data-center/a100/
12. Apple, MacBook Pro tech specs (unified memory options) https://www.apple.com/macbook-pro/specs/
13. Runpod, GPU pricing (checked November 2, 2026). https://www.runpod.io/pricing
14. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
15. GitHub, ggml-org/llama.cpp src/models/gemma4.cpp (Gemma 4 K/V handling; commit 20a51f24, checked November 3, 2026). https://github.com/ggml-org/llama.cpp/blob/20a51f24a8523b290dcc1fc4fe9ce8fbdef9de13/src/models/gemma4.cpp
16. GitHub, ggml-org/llama.cpp src/llama-kv-cache-iswa.cpp (sliding-window cache size; same commit). https://github.com/ggml-org/llama.cpp/blob/20a51f24a8523b290dcc1fc4fe9ce8fbdef9de13/src/llama-kv-cache-iswa.cpp