---

license: apache-2.0

base_model:

• Qwen/Qwen3.5-122B-A10B

tags:

• gguf
• llama.cpp
• mixture-of-experts
• quantized
• iq3_xxs
• instinctrazor

pipeline_tag: text-generation

---

InstinctRazor — Qwen3.5-122B-A10B · IQ3_XXS GGUF

A 122B hybrid Gated-DeltaNet MoE (256 experts, 8 active) — packed to 48 GiB so it runs on **one 80 GB

GPU (or a small card + CPU offload). Quantized from the original BF16** with an importance matrix

(math + code + general calibration), via llama.cpp.

Framework, recipe, and full reproduction: https://github.com/General-Instinct/InstinctRazor

Speed (llama.cpp, this artifact)

• 1× H100-80GB, all layers on GPU: 115.9 tok/s decode (prefill ≈2541 tok/s).
• Small card + CPU expert-offload (--n-cpu-moe 48, peak ≈7.6 GiB VRAM): 45.7 tok/s decode — runs on an 8 GB GPU + ≈48 GiB system RAM.

Run

# full GPU
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf -ngl 999 -fa on -p "Your prompt"
# small card + CPU offload (routed experts on CPU)
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf -ngl 999 --n-cpu-moe 48 -t 52 -p "Your prompt"
# multimodal (image input)
llama-cli -m InstinctRazor-Qwen3.5-122B-A10B-IQ3_XXS.gguf --mmproj InstinctRazor-Qwen3.5-122B-A10B-mmproj-f16.gguf --image pic.png -p "Describe the image"

Requires a llama.cpp build with qwen3_5_moe support (upstream, 2026-02+).

Scope & roadmap

This GGUF matches or beats the footprint-matched A4B on knowledge, reasoning, and multimodal-MMMU. Where it

still trails — code (LiveCodeBench v6) and math / multimodal-math — the loss is largely

token-inefficiency introduced by quantization, and is the target of OPD (on-policy distillation), a

separate framework we'll open-source later. Eval absolutes are subject to a same-harness validation gate;

see the GitHub results/RESULTS.md

for full per-number provenance.

Attribution

• Base model: Qwen3.5-122B-A10B © Qwen — subject to its own model license.
• Quantization recipe + framework: General Instinct, released under Apache-2.0.