---

license: apache-2.0

base_model: Qwen/Qwen3-Next-80B-A3B-Instruct

base_model_relation: quantized

quantized_by: alfodaniello

pipeline_tag: text-generation

tags:

- gguf

- qwen3_next

- moe

- apple-silicon

- dwarfstar-red-lite

---

Qwen3-Next-80B-A3B-Instruct, Red Lite mixes (19.3 GB and 33.5 GB)

2-bit-class GGUFs of Qwen/Qwen3-Next-80B-A3B-Instruct

with the same size as Bartowski's IQ2_XXS file and better quality: the bits are placed where the model needs

them, instead of by a fixed rule.

Made for DwarfStar Red Lite, a native Metal runtime for this one

model on Apple Silicon. They are standard GGUFs and load in llama.cpp too.

| file | size | perplexity | SHA-256 |

|---|---:|---:|---|

| Qwen3-Next-80B-A3B-Instruct-RedLite-F2.gguf (recommended) | 19.32 GB | 15.379 | d22dbbc4e96ede01a028789d281992b758b817742d2842b82f2e3ce5a9a948c5 |

| Qwen3-Next-80B-A3B-Instruct-RedLite-E3.gguf | 19.30 GB | 16.216 | b62067d6f28c52c07f8fca55196e8a63916264e40c6641f6a8d2e710768d291c |

| Qwen3-Next-80B-A3B-Instruct-RedLite-G2.gguf (48 GB Macs) | 33.54 GB | 14.199 | f0b05ae3cd98946fb35d0d5412253920cf3da14664159c026fa9f4deed321c1f |

Bartowski's scheme, rebuilt the same way: 16.370. G2 is compared with Bartowski's IQ3_XXS (14.292) below.

What is different

Bartowski's IQ2_XXS.

• Routed experts: IQ2_XS (2.31 bits per weight) on layers 0–5 and 43–47, IQ1_M (1.75 bits) on the other 37 layers.
• Dense weights: the DeltaNet and attention input projections and part of the shared experts IQ2_XXS (2.06 bits),

the rest Q4_K / Q6_K / Q8_0, token embedding Q2_K, output head Q5_K.

E3. Every tensor keeps its type. The 11 IQ2_XS expert layers move to 37–47, the layers whose experts receive

the most input energy in the importance matrix (down-projection input energy grows from 1.3 at layer 0 to 565 at

layer 47).

F2.

• The dense tensors that are IQ2_XXS become Q4_K. They are only about 290 MiB, but every token reads them, while

a token reads 10 of 512 experts per layer.

• To keep the size, the IQ2_XS expert layers are 40–47.

Quality, measured

Every file was quantized from Bartowski's Q8_0 with his imatrix.gguf and the same llama.cpp. Perplexity was measured

on a fixed 111-chunk corpus at context 512, the answers on 235 prompts against Qwen's own API.

| | Bartowski's scheme, reproduced | E3 | F2 |

|---|---:|---:|---:|

| size | 19.30 GB | 19.30 GB | 19.32 GB |

| perplexity | 16.370 | 16.216 | 15.379 (−6.1 %) |

| paired per-chunk test | — | vs Bartowski's: t = −3.91, better on 72 / 111 | vs E3: t = −14.9, better on 103 / 111 |

| greedy answers identical to Qwen's API | 2 | 4 | 5 |

| mean share of words matching the API from the start | 12.6 % | 13.6 % | 15.2 % |

• Also tried:

- dense weights at IQ3_XXS with IQ2_XS on 38–47: 15.419, not distinguishable from F2;

- IQ2_XS on layers 0–2 and 40–47: 16.326;

- IQ2_XS on every down projection: 3 % larger, 16.423.

• Speed (Red Lite, every expert resident):

- plain decode: F2 is 3 % slower than E3 (M4 Max 82.8 vs 85.2 tok/s, M4 Pro 24 GiB 44.5 vs 45.8);

- with MTP speculative decoding on the M4 Pro: as fast or faster (54.8 / 58.2 / 48.9 vs 50.8 / 58.0 / 47.0 tok/s

on three prompts).

• Red Lite checks on the M4 Max 48 GiB:

- F2 passes the regression suite (41 checks, no failure; the stage tools for the reference layout are skipped);

- it passes the parity checks against the pinned llama.cpp (logits, greedy tokens);

- it gives token-identical greedy output on an M4 Pro.

Records and method: dev54,

dev58.

G2, for 48 GB Macs

Bartowski's IQ3_XXS (31.73 GB) leaves room on a 48 GB Mac. G2 spends 1.8 GB of it on the routed experts: IQ3_S

(3.44 bits) instead of IQ3_XXS for ffn_down on every layer and for ffn_gate / ffn_up on layers 24–47. Every

other tensor keeps IQ3_XXS's type.

| | Bartowski IQ3_XXS | G2 |

|---|---:|---:|

| size | 31.73 GB | 33.54 GB |

| perplexity, text (111 chunks) | 14.292 | 14.199 (−0.65 %, t = −3.5) |

| perplexity, code (90 chunks) | 1.975 | 1.948 (−1.4 %, t = −5.5) |

| Red Lite decode with MTP, M4 Max (median of six prompts) | 89.3 tok/s | 90.7 tok/s |

• Also tried:

- the dense weights at Q8_0 too (G3): −1.0 % / −1.7 %, but too large to keep MTP on a 48 GB Mac;

- the dense weights at Q8_0 alone (G1): −0.4 %.

• Checks: G2 passes Red Lite's parity checks against llama.cpp on the M4 Max. Its two long-context argmax

differences are near-ties.

• Limit: with 25K-token prompts a 48 GB Mac with other applications open swapped, and G2's prompt ingestion

varied from 401 to 676 tok/s (IQ3_XXS: 596–638). For very long prompts IQ3_XXS is steadier.

• Record: dev64.

Speed with Red Lite 0.9.2–0.9.5

F2 shares its Q4_K dense weights with the Coder file, the largest cost of a decoded token, which Red Lite 0.9.2

rewrote. F2 on an M4 Max 48 GiB, every expert resident (decode-bench, three alternated pairs, measured the same day):

94.3 → 108.4 tok/s (Red Lite 0.9.1 → 0.9.3). Bartowski's IQ2_XXS gains less (83.5 → 90.9 tok/s, M4 Max; 46.3 →

53.3 on an M4 Pro 24 GiB with every expert resident, 33.9 → 37.6 with the 4 GiB expert cache; Red Lite 0.9.3),

because it has fewer Q4_K weights. E3 and G2 were not measured

separately. Record: dev74.

Use

• Red Lite (24 GiB and larger Apple Silicon Macs): redlite download 24gb (F2), then redlite chat. On a 48 GB

Mac: redlite download 48gb-g2 (G2).

- On a 24 GiB Mac, run redlite gpu-limit --boot once (Red Lite 0.9.5, asks for your password).

- It raises macOS's GPU memory limit so every expert stays resident: Bartowski's IQ2_XXS goes from 37.6 to

53.3 tok/s on an M4 Pro 24 GiB (Red Lite 0.9.3).

- The limit is a ceiling: the memory stays free for other apps when Red Lite is not running.

- redlite gpu-limit --off undoes it.

• llama.cpp: llama-cli -m Qwen3-Next-80B-A3B-Instruct-RedLite-F2.gguf -cnv. It needs a build with Qwen3-Next

support.

Reproduce

python3 scripts/dev/quant_mix.py --like Qwen_Qwen3-Next-80B-A3B-Instruct-IQ2_XXS.gguf \
    --q8 Qwen_Qwen3-Next-80B-A3B-Instruct-Q8_0-00001-of-00003.gguf \
    --imatrix Qwen_Qwen3-Next-80B-A3B-Instruct-imatrix.gguf --iq2xs-layers 40-47 --dense-type q4_K --out F2.gguf
# E3: --iq2xs-layers 37-47, no --dense-type
# G2: --like Qwen_Qwen3-Next-80B-A3B-Instruct-IQ3_XXS.gguf --keep-experts --base-type IQ3_XXS \
#     --set ffn_down_exps=iq3_s --set ffn_gate_exps=iq3_s@24-47 --set ffn_up_exps=iq3_s@24-47

The script is in the Red Lite repository; it calls llama-quantize with one type per tensor.

Credits and limits

• Model: Qwen team, Apache-2.0.
• Q8_0 source, importance matrix and the reference IQ2_XXS layout:

bartowski.

• Quantizer: llama.cpp.
• Limits: quality was measured by perplexity (one text corpus; a code corpus for G2) and, for F2 and E3, by

agreement with Qwen's API, not by task benchmarks.