---

base_model:

• orcarouter/Qwen3.8-Flash-Next-Uncensored

license: other

library_name: gguf

pipeline_tag: text-generation

tags:

• gguf
• qwen
• mixture-of-experts
• long-context
• vision
• uncensored
• abliterated

quantized_by: groxaxo

---

Qwen3.8 Flash Next — Uncensored (Heretic) GGUF

GGUF quantizations of Qwen3.8-Flash-Next-Uncensored (heretic variant), an uncensored/abliterated release of the Qwen3.8 Flash Next sparse MoE architecture.

What's in this repo

| Folder | Quant | Files | Notes |

|--------|-------|-------|-------|

| UD-IQ4_XS/ | Unsloth Dynamic IQ4_XS | 41 shards + F16 mmproj | Smallest footprint, full 262K native context |

| AD-5.00bpw-Q5_K_M-M64/ | ADQuant 5.00 bpw Q5_K_M-M64 | 32 shards + F16 mmproj | Higher fidelity, M64 expert layout |

Both include the F16 mmproj for vision (image input).

Architecture (from GGUF metadata, qwen4exp)

• 48 decoder layers, hybrid attention with indexer heads
• MoE: 512 experts, 10 active per token (+ shared expert) — huge weights, cheap decode
• Native context: 262,144 tokens
• Vocab: 248,320

Because only ~10/512 experts fire per token, the model is far larger on disk than its per-token compute — expect fast decode even on consumer hardware, with expert weights partially served from system RAM via mmap when VRAM is insufficient.

Recommended llama.cpp serving recipe

Tuned on 3x RTX 3090 (24 GB) in production:

llama-server -m <model>.gguf --mmproj <mmproj>.gguf \
  -c 225280 -ngl 999 -sm layer -ts 1,1,1 \
  -fa on -ctk q8_0 -ctv q8_0 \
  --jinja --reasoning-preserve \
  -md <mtp-draft.gguf> --spec-type draft-mtp --spec-draft-n-max 4 -ngld 999

• Flash attention on + Q8_0 KV cache keeps long contexts memory-feasible.
• MTP speculative decoding (native multi-token-prediction head, draft up to 4 tokens) roughly doubles decode speed; keep the draft's expert tensors on a single GPU for best acceptance.
• Full offload needs ~68 GB aggregate VRAM; below that, let MoE experts overflow to host RAM (--load-mode mmap) — sparse activation makes this cheap.

Benchmarks (3x RTX 3090, measured with MTP draft on)

Measured on the sibling quant AD-4.27bpw-Q4_K_M-M64 agentic-v2 of the same base model, 225K ctx, Q8 KV, single slot:

| Workload | Decode tok/s | Draft acceptance |

|----------|-------------|------------------|

| Structured math | 95–98 | 0.80–0.86 |

| Code generation | ~79 | 0.69–0.70 |

| Reasoning / explanation | 56–64 | 0.43–0.53 |

| Creative prose | 52–64 | 0.37–0.53 |

| Vision (mmproj) | ~57 | 0.45–0.48 |

• TTFT: ~0.3 s (231-token prompt), prefill ~630 tok/s
• Speed tracks speculative-draft acceptance almost linearly: predictable text flies, open-ended prose sits near ~50 tok/s.

Notes

• Uncensored/heretic finetune: no built-in refusal behavior; you are responsible for output filtering and usage policies.
• Q5_K_M-M64 shards are a single-file unsplit merge away from the master; use --keep-split workflows if you re-quantize.