Edge0-120B-NVFP4

---

license: other license_name: edge0-community-model-1.0 license_link: LICENSE library_name: edge0 tags:

• moe
• edge-inference
• latent-moe
• 4-bit
• agent
• coding
• cuda base_model:
• Qwen/Qwen3.8-Flash-Next pipeline_tag: text-generation

---

Capacity controls · VRAM and host RAM

All of these live in the pack's model.json, not on a command line: flags go in args, environment variables in env, with paths relative to the pack directory. That is the same file the Edge0 Chat app and edge0-infer read, so a change there applies to every launch.

"args": ["--expert-cache", "auto", "--kv-resident", "32768", "--vram-reserve-mib", "1024"],
"env":  {"STRATA_LATENT_DENSE": "./latent_dense_q4.bin", "STRATA_LATENT_DENSE_FMT": "4",
         "STRATA_LAT_FUSED_Q3K": "0"}

VRAM

Flag	Meaning	Default
--vram-reserve-mib N	VRAM left free for the desktop, the driver and other processes; the engine sizes its caches from what remains. It means "leave this much free", not "give the engine this much", so the same number means different things on cards with different totals. Accepted but not listed by --help.	700
--expert-cache N|auto	Expert slots kept in VRAM. auto derives the count from the free VRAM after the fixed footprint (skeleton projections ≈ 2.9 GiB, MTP draft ≈ 0.8 GiB, verify and draft-head buffers ≈ 0.45 GiB, plus the D/U device buffers).	0 (none)
--expert-cache-device1..3 N|auto	Use the second and third GPU as further expert-cache tiers.	off
--kv-resident N	Keep N KV cells per QSA layer in VRAM and stream the K/V from pinned host RAM, which frees VRAM for experts.	0 (all in VRAM)
--mtp-q4 all|proj|head	4-bit copies of the draft layer's projections and/or head, about 0.3–0.5 GiB less VRAM.	off
--prefill CHUNK	Prompt chunk size; bigger chunks need bigger device buffers. auto uses the largest chunk whose buffers the expert cache can lend.	auto
--spec N	The MTP drafter's verify window: how many tokens it proposes per check. A bigger window needs bigger verify buffers (about 180 MiB at 4) and pays off when every window has a fixed per-layer cost. The shipped packs use 4; the macOS setup picks 2, or 3 when the experts stream from SSD. Without an MTP layer there is no speculation.	4

Host RAM

Flag	Meaning	Default
--memory-limit-mib N	The RAM this engine's container or cgroup may use, for when the container cannot see its own limit.	0 (detect)
--resident-budget-gib N	Cap on the host-RAM expert cache. It holds only the experts the VRAM cache does not hold, so the two sets are disjoint, and the value is clamped to the RAM actually available minus headroom. Requires a static --expert-profile.	0 (off)
--resident-experts	The same with the whole complement as the budget.	off
--mmap-experts	Read experts through the pack file on demand instead of pinning them, so nothing is resident in RAM.	off
--conversation-cache-mib N	Conversation cache. 0 turns it off.	adaptive
--ple-io direct|mmap|ram	The PLE table on its SSD: direct reads it unbuffered and it never enters memory; ram locks the whole table in RAM.	direct
--kv-resident N	The streamed K/V lives in pinned host RAM (1.55 GiB at 32768 cells).	—

Community builds

Third-party conversions of this model. They are not made or endorsed by Edge0; this model's licence and the upstream terms still apply to them.

• infosave/Edge0-120B-cmf — a single-file CMF build for cortiq (Rust, CUDA and Vulkan, OpenAI-compatible). The routed experts are this NVFP4 build's weights copied bit for bit; the D/U projections and the shared expert are re-encoded to 8 bits and the router to 16. The author reports it running from 12 GB cards.