Recursive Flint VRAM requirements: can you run it locally?

Flint is the first model from Recursive, a mixture-of-experts (MoE) model built for coding agents. Its Hugging Face repo was created on October 21, 2026, it is Apache 2.0 licensed, and Hugging Face lists 39,785,287,456 parameters.^[1] The model card says about 2.5 billion parameters are active per token, it has 49 layers (2 dense, then 47 MoE layers with 256 routed experts, 6 picked per token, plus a shared expert), and a documented context of 262,144 tokens.^[2] Recursive reports 54.9% on Terminal-Bench 2.1 and 74.2% on SWE-bench Verified (developer-reported scores).^[2]

Because it is an MoE, only a small slice of the weights does work for each token, which is why it runs fast. But all 39.8B parameters still have to sit in memory (VRAM, or VRAM plus system RAM), so memory, not speed, is what decides whether it runs for you.

Memory needed at each quantization

Quantization stores each weight in fewer bits. Smaller files fit on smaller GPUs, at some cost in quality, and the cost grows below about 4 bits. Below is every GGUF build in Unsloth's Flint repo, from 1-bit to the original BF16, using the exact file sizes on Hugging Face (1 GB = 1,000,000,000 bytes), largest first.^[4] Unsloth uses its own "UD" dynamic quants here, so there is no plain Q4_K_M file; UD-Q4_K_XL is its recommended 4-bit build.^[5]

Unsloth's guide gives these RAM + VRAM (or unified memory) requirements: 13–15 GB at 2-bit, 16–20 GB at 3-bit, 26 GB at 4-bit, 38 GB at 6-bit and 44 GB at 8-bit.^[5] Ollama's flint tags are 23 GB for the default (flint:latest, flint:40b), 26 GB for q4_1, and 80 GB for the mlx-bf16 build.^[6]

Which quant should you pick? UD-Q4_K_XL is the usual balance of size and quality if you have 32 GB. On a 24 GB card, UD-Q3_K_XL is the practical choice. Use Q8_0 if you want results close to the original. Treat 1-bit and 2-bit builds as a last resort.

Context length adds memory (the KV cache)

On top of the weights, the engine keeps a key/value cache for the conversation. Flint is unusually cheap here. From config.json, only 14 of its 49 layers use full attention; the other 35 use a sliding window of 513 tokens.^[3][2] The full-attention layers (4 KV heads, head size 128) take 2 × 14 × 4 × 128 × 2 bytes = 28,672 bytes per token at FP16:

The sliding-window layers add a fixed amount that does not grow with context: 35 × 513 tokens × 2,048 bytes, about 37 MB in total. That is our calculation from the config, not a published figure.^[3] Even the full 262,144-token context needs only about 7.5 GB of KV cache.

Which GPUs and Macs it fits on

Each cell compares the weights plus an 8K-token KV cache with the device's memory.^{[7][8][9][10][11][12][13]}

Column headers show weights + 8,192-token KV cache.

How to read it: "Fits" means weights + 8K cache use under 90% of memory. "Tight" means 90–100%: it may load, but the engine's own working memory or a longer context can push it over. "No" means it won't fit on the GPU alone. The 90% line is our own rule of thumb, not a vendor figure. On a Mac, macOS also needs some of the unified memory, so treat Mac rows as optimistic.

Our recommendations

These follow from the tables above.

How it compares with similar models

4-bit file sizes and KV cache per token come from each model's Hugging Face GGUF repo and config.json, the same data as our comparison table.^[15]

*Counts only the full-attention layers (16 for Qwen3.8 27B, 14 for Flint). †Flint's 4-bit file is Unsloth UD-Q4_K_XL; the others are Q4_K_M. Flint's 4-bit file is about 7.6 GB bigger than Qwen3.8 27B's, but it activates only about 2.5B parameters per token against Qwen3.8 27B's 27.8B (dense), so it generates tokens much faster on the same hardware.^[2]

How to run it

The easiest way is Ollama: ollama run flint pulls the 23 GB default build.^[6] llama.cpp added Flint support in pull request #30928, merged October 21, 2026, so use a build from after that date.^[16] With llama.cpp or LM Studio, download a GGUF such as flint-UD-Q4_K_XL.gguf (or flint-UD-Q3_K_XL.gguf on 24 GB) from the list above.^[4] Unsloth's guide uses an 8K context with --n-gpu-layers 99 and recommends temperature 1.0, top-p 0.95 and top-k 20, the same settings Recursive gives.^[5][2] Recursive's own SGLang kit targets one H200 or B300 data-center GPU with the BF16 weights (about 80 GB).^[2]

If the model is bigger than your VRAM, llama.cpp can keep some layers in system RAM. Because only about 2.5B parameters are active per token, Flint handles this better than a dense model of the same size. One user on X reported running it on a 12 GB RTX 3060 with 16 GB of system RAM at about 46 tokens per second (a third-party report we have not tested).^[17]

Rent a GPU for this model

A 32 GB RTX 5090 runs UD-Q4_K_XL, and a 48 GB card runs Q8_0, so renting is cheap. On-demand prices listed on Runpod's pricing page (checked October 23, 2026; Community Cloud is cheaper but runs on shared hosts):^[14]

Prices checked on 2026-10-24. They change often, so confirm on the provider's page. A 48 GB card runs Q4_K_M tightly; 80 GB gives room for longer context or Q5/Q6.

See all GPU rental prices for more providers and multi-GPU setups.

Where to rent or buy

Frequently asked questions

Can I run Recursive Flint on a 24 GB GPU like the RTX 4090?

Yes, at 3 bits. The UD-Q3_K_XL file (18.5 GB) plus an 8K cache is about 18.7 GB, or 78% of 24 GB, and a 32K context brings it to about 19.4 GB.^[4][3] The 4-bit UD-Q4_K_XL (25.1 GB) is bigger than 24 GB on its own, so it needs some offloading to system RAM.

Is "24 GB" enough, as people say?

For the 3-bit build, yes. Unsloth says UD-Q3_K_XL "fits 24GB RAM setups at short contexts" and lists 16–20 GB for 3-bit and 26 GB for 4-bit.^[5] Ollama's default download is 23 GB, which leaves very little room on a 24 GB card once the cache and engine buffers are added.^[6]

How much VRAM does Flint need for the full 256K context?

The FP16 KV cache for 262,144 tokens is about 7.5 GB, because only 14 layers keep a full cache.^[3] With UD-Q4_K_XL weights that's about 32.6 GB, just over a 32 GB card, and with UD-Q3_K_XL about 26.0 GB, before engine overhead.

Can I run Flint on a 12 GB or 16 GB GPU?

Not comfortably on the GPU alone. On 16 GB, UD-Q2_K_XL (13.6 GB, about 13.8 GB with an 8K cache) fits. On 12 GB even the 1-bit UD-IQ1_M (11.4 GB) is tight.^[4] Splitting the model between the GPU and system RAM is the realistic route on these cards.

Can a Mac run Flint?

Yes. Unsloth reports 125+ tokens per second on an M4 Pro MacBook Pro.^[5] A 24 GB Mac can hold UD-Q3_K_XL at a short context, a 32 GB Mac UD-Q4_K_XL, and a 64 GB Mac Q8_0 (42.3 GB).^[4][13] Ollama also has an MLX BF16 build (80 GB) for large-memory Macs.^[6]

Flint or Qwen3.8 27B?

On 24 GB, Qwen3.8 27B fits at 4-bit Q4_K_M (17.4 GB), while Flint needs its 3-bit build.^[15] Flint is far faster per token because it activates about 2.5B parameters, and it is built for coding agents. In Recursive's own chart, Qwen 3.8 scores higher on Terminal-Bench 2.1 (73.0 against 54.9).^[5]

How we calculated

• Weights are the exact GGUF file sizes on Hugging Face, checked October 24, 2026.^[4] BF16 is the sum of the three BF16 GGUF shards.
• KV cache is calculated from config.json for the 14 full-attention layers at FP16.^[3] The fixed sliding-window cache (about 37 MB) is not in the tables.
• Device memory is from NVIDIA and Apple spec pages. We compare in decimal GB, so the results lean conservative.
• Rental prices are the listed on-demand Runpod rates from our GPU price data, checked October 23, 2026.^[14]
• Not included: engine working buffers, the OS and display. These vary by engine and settings, so leave headroom.

Sources

All sources checked on 2026-10-24.

1. Hugging Face, RecursiveSI/flint (parameter count, license, repo date). https://huggingface.co/RecursiveSI/flint
2. Recursive Flint model card (architecture, active parameters, context, benchmarks, settings, SGLang kit). https://huggingface.co/RecursiveSI/flint/blob/main/README.md
3. Hugging Face, config.json (layer types, KV heads, head size, sliding window). https://huggingface.co/RecursiveSI/flint/blob/main/config.json
4. Hugging Face, unsloth/flint-GGUF file list. https://huggingface.co/unsloth/flint-GGUF/tree/main
5. Unsloth, Flint: Run Locally Guide (memory table, recommended quants, speeds). https://unsloth.ai/docs/models/flint
6. Ollama, flint tags. https://ollama.com/library/flint/tags
7. NVIDIA, GeForce RTX 3060 family (12 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3060-3060ti/
8. NVIDIA, GeForce RTX 5080 (16 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080/
9. NVIDIA, GeForce RTX 3090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
10. NVIDIA, GeForce RTX 4090 specs (24 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
11. NVIDIA, GeForce RTX 5090 specs (32 GB). https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
12. NVIDIA, RTX 6000 Ada Generation (48 GB). https://www.nvidia.com/en-us/products/workstations/rtx-6000/
13. Apple, MacBook Pro tech specs (unified memory options). https://www.apple.com/macbook-pro/specs/
14. Runpod, GPU pricing (checked October 23, 2026). https://www.runpod.io/pricing
15. VRAMNeeded comparison table (sources listed per model). https://vramneeded.com/compare/
16. GitHub, llama.cpp pull request #30928 "models: Add Flint model (FlintForCausalLM)". https://github.com/ggml-org/llama.cpp/pull/30928
17. X post by @DogukanUrker (RTX 3060 12 GB + 16 GB RAM report, October 22, 2026). https://x.com/DogukanUrker/status/2113060329038970892