---

license: apache-2.0

base_model: Qwen/Qwen3.5-122B-A10B

tags:

• gguf
• quantized
• moe
• iq2_xxs
• mtp
• qwen3.5

library_name: gguf

---

Qwen3.5-122B-A10B PS-IQ2_XXS-MTP (Peasant Smith)

The first IQ2_XXS-class GGUF of Qwen3.5-122B-A10B — smaller than any published quant (unsloth's ladder floors at Q3_K_S 52.5 GB) — built entirely from the original Qwen/Qwen3.5-122B-A10B weights with the native MTP (nextn) head embedded at Q8_0.

File: Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf — 41.77 GB (38.9 GiB), ≈2.7 bits/param over 122B total params.

SHA-256: 01bd59623e7d4f2f45bd1ce8665042bf449b8000a76480281bef7866cfce5557

Provenance chain (100% original weights)

1. Qwen/Qwen3.5-122B-A10B official BF16 safetensors (39 shards, 244 GB — no third-party quant or modified repo involved)
2. → GGUF master via llama.cpp convert_hf_to_gguf.py (arch qwen35moe, nextn_predict_layers = 1, MTP remapped to blk.48.* and embedded at full precision)
3. → llama-quantize importance-matrix quant with the tensor split below

Imatrix: imatrix_unsloth.gguf from unsloth/Qwen3.5-122B-A10B-GGUF (359 MB, large diverse calibration corpus). Verified before use: 144/144 main expert tensors covered, zero naming mismatches against our master; the only uncovered tensors are the 3 MTP expert tensors, which were pinned out of IQ quantization.

Tensor split (verified from output file — 0 violations)

| Class | Tensors | Type |

|---|---|---|

| Routed expert gate/up | 96 | IQ2_XXS |

| Routed expert down | 48 | Q2_K |

| Token embeddings | 1 | Q6_K |

| Attention, shared experts, router, output, norms | 384 | Q8_0 |

| Layer norms/biases | 370 | F32 (kept) |

| MTP / nextn block (blk.48) | 20 | Q8_0 |

Down-projection gets one full step above gate/up because its errors land directly and additively on the residual stream, while gate/up errors pass through activations first. The MTP head is kept at maximum fidelity so the draft head stays sharp.

Measured quality checks

| Test | Result |

|---|---|

| Greedy smoke ("The capital of France is") | ✅ "The capital of France is Paris." (finish=stop) |

| Needle recall ("9137 steps", 8k ctx) | ✅ exact answer 9137 |

| LRU cache code task (8k ctx) | ✅ complete correct doubly-linked-list LRU with sentinel nodes, O(1) get/put, eviction (5000-token continuous generation, zero degradation) |

| Long-generation stability | ✅ 5000 tokens continuous at steady t/s |

| Perplexity (wikitext-2) | not measured — see honesty note below |

Usage

llama-server -m Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP.gguf -ngl 99 -ncmoe 20 -c 8192 --load-mode none

Requires a recent llama.cpp master (qwen35moe + linear-attention + MRoPE support). The model ships its chat template; use --jinja.

Limitations

• 2-bit experts: real quality delta vs BF16 exists by construction; task battery above passed but long-tail knowledge degradation is expected at this footprint
• Text-only (vision tower of the original repo is not part of this GGUF)
• Not verified beyond 8k context
• Ollama can load the file, but -ncmoe-style expert pinning is llama.cpp-only

Running with ollama

ollama derives its tag from the GGUF filename (last dash-separated segment):

ollama run hf.co/peasantsmith/Qwen3.5-122B-A10B-PS-IQ2_XXS-MTP-GGUF:IQ2_XXS

Notes:

• The embedded MTP/nextn head is currently inert (llama.cpp has no draft wiring for the qwen35moe architecture yet); the model runs as a standard A10B MoE.
• Expert-offload flags like -ncmoe are llama.cpp-only; ollama loads with its default placement. Measured ~17 t/s with llama.cpp on 3x RTX 3060 (experts partially on CPU) - see benchmarks below.

Credit

Upstream model by Qwen (Apache-2.0). imatrix by Unsloth. Quant by Peasant Smith.