---

license: apache-2.0

base_model: Jackrong/Qwopus3.6-27B-Coder

tags:

- gguf

- quantization

- moq

- mixture-of-quantizations

- qwen3.5

- coder

- code

- verified

---

Qwopus3.6-27B-Coder-GGUF-Predator-Q (MoQ-4.0)

Mixture-of-Quantizations (MoQ) 4.0 BPW GGUF for Qwopus3.6-27B-Coder.

Canonical Predator-Q winner as of 2026-06-24 — verified on 6 benchmarks (HumanEval+ 164, MBPP+ 100, BigCodeBench 50, LCB-30 30). See "Multi-benchmark validation" section below for the full comparison vs. ASI-Evolved v2.

Summary

| Metric | Value |

|---|---|

| File size | 12.6 GB |

| Effective BPW | 4.04 |

| WikiText-2 PPL (Q) | 6.18 (vs F16 base 6.02) |

| Mean PPL(Q)/PPL(base) | 1.026 |

| Mean KL divergence (vs F16) | 0.034 |

| Same top p (vs F16) | 91.6% |

| RMS Δp | 5.43% |

| HumanEval+ base pass@1 | 92.7% (152/164) |

| HumanEval+ plus pass@1 | 89.6% (147/164) |

| MBPP+ base pass@1 | 92.0% (92/100) |

| MBPP+ plus pass@1 | 79.0% (79/100) |

| BigCodeBench pass@1 | 42.0% (21/50) |

| LCB-30 pass@1 | 50.0% (15/30) |

Multi-benchmark validation (2026-06-24, Phase 5)

This is the canonical winner. A more complex alternative (ASI-Evolved v2) was built and tested on the same 4 multi-task benchmarks. Result: no statistically significant difference on any benchmark (all McNemar p > 0.31, Bonferroni-corrected α = 0.0167). MoQ-4.0 is preferred because it is 2 GB smaller and requires zero LLM iteration cost to build.

!Multi-benchmark comparison

| Benchmark | MoQ-4.0 (12.4 GiB) | ASI-Evolved v2 (14.4 GiB) | Δ | McNemar p |

|---|---|---|---|---|

| HumanEval+ base (164) | 152 (92.7%) | 153 (93.3%) | +0.6pp | 0.65 |

| HumanEval+ plus (164) | 147 (89.6%) | 144 (87.8%) | -1.8pp | 0.65 |

| MBPP+ base (100) | 92 (92.0%) | 93 (93.0%) | +1.0pp | 0.56 |

| MBPP+ plus (100) | 79 (79.0%) | 78 (78.0%) | -1.0pp | 0.56 |

| BigCodeBench (50) | 21 (42.0%) | 22 (44.0%) | +2.0pp | 0.32 |

| LCB-30 (30) | 15 (50.0%) | 16 (53.3%) | +3.3pp | 0.72 |

!Summary table

The v2 model has a 16% lower KL-divergence against F16 (0.0287 vs 0.034) but does not produce measurably better outputs on any of the 6 code tasks tested. The 2 GB file size savings of MoQ-4.0 are pure upside.

Pareto frontier

!Pareto frontier

!Pareto size vs capability

MoQ-4.0 sits at the Pareto-optimal corner: smaller than all baseline quantization recipes, with capability equivalent to (or statistically tied with) the larger alternatives.

Comparison to Baselines

| Build | File Size | BPW | KL-div | Same top p | PPL ratio | LCB-30 |

|-------|-----------|-----|--------|------------|-----------|--------|

| Q4_K_M | 15.4 GB | 4.69 | 0.019 | 93.6% | 1.009 | 50.0% |

| 4A BEST | 16.98 GB | 5.05 | 0.018 | 93.7% | 1.012 | 50.0% |

| ASI-Evolved v2 | 14.4 GiB | 4.51 | 0.0287 | 92.55% | 1.020 | 53.3% |

| This (MoQ-4.0) | 12.6 GB | 4.04 | 0.034 | 91.6% | 1.026 | 50.0% |

Net result: Same HumanEval+/MBPP+/LCB-30 capability as Q4_K_M and 4A BEST, with 18% smaller file size than Q4_K_M and 26% smaller than 4A BEST. ASI-Evolved v2 has a marginal KL-div win that did not translate to task performance.

What is MoQ?

MoQ (Mixture of Quantizations) is a search-based mixed-precision quantization method by Waleed Ahmad. It estimates per-tensor "elasticity" — sensitivity to low precision — then assigns each tensor a different Q-type to minimize quality loss per bit spent. This produces non-uniform recipes like:

• ffn_down → IQ3_S (aggressive — most redundant tensor)
• ffn_gate → IQ3_S (aggressive)
• ffn_up → IQ4_XS (slightly more conservative)
• attn_qkv (most layers) → IQ4_XS
• attn_qkv (special every-4th full-attention layers) → split: attn_q=IQ4_NL, attn_k=bf16, attn_v=bf16, attn_output=Q5_K
• token_embd → Q4_K
• Norms → bf16

The recipe was applied with importance matrix (imatrix) calibration, which makes IQ-quant decisions properly importance-aware. Without imatrix, the same recipe produces worse KL-div (0.059 vs 0.034).

Recipe Source

The per-tensor recipe used here is MoQ_qwen3.6-27b_tensors_4.0.txt from kaitchup/Qwen3.6-27B-GGUF-MoQ. Since Qwopus3.6-27B-Coder uses the same Qwen3.5 architecture (qwen35, 64 layers, 851 tensors), the recipe transfers directly without modification.

Build Details

# 1. Generate imatrix (WikiText-2 test set, 100 chunks)
llama-imatrix -m qwopus_f16.gguf -f wikitext-2-raw/wiki.test.raw \
  -o qwopus_imatrix.gguf -ngl 50 --chunks 100

# 2. Quantize with MoQ recipe + imatrix
llama-quantize --imatrix qwopus_imatrix.gguf \
  --tensor-type-file MoQ_qwen3.6-27b_tensors_4.0.txt \
  qwopus_f16.gguf qwopus_moq_4.0.gguf Q4_K

Evaluation Details

• WikiText-2 KL-div: llama-perplexity -c 512 -b 512 --kl-divergence --kl-divergence-base base_logits_0.dat --chunks 50 --seed 1337
• HumanEval+ / MBPP+: evalplus library, untrusted_check with base+plus variants, max_tokens=1024, temperature=0
• BigCodeBench: custom harness using unittest test cases, 30s timeout per problem
• LCB-30: LiveCodeBench easy subset, 30 problems × 3 attempts each (pass@1)
• Statistical test: McNemar's test (paired, binary outcomes), Bonferroni correction across 6 benchmarks
• Base logits: F16 GGUF reference logits saved once and reused for all variant evaluations

Known Limitations

• Code-specific tasks: Tested on 6 coding benchmarks. Performance on math/reasoning/long-form generation not measured.
• Long context: Not stress-tested beyond LCB-30's 4096-token context. Qwopus's full 262K context window requires the 4D-RoPE / Mamba-SSM hybrid architecture to work correctly — preserved here since all norms/embeddings are at full precision.
• KL-div is higher than Q4_K_M (0.034 vs 0.019). This is the cost of smaller file size with mixed-precision quantization. Same top token agreement remains high (91.6%).

Provenance

• Base model: Jackrong/Qwopus3.6-27B-Coder (BF16)
• Recipe source: kaitchup/Qwen3.6-27B-GGUF-MoQ MoQ_qwen3.6-27b_tensors_4.0.txt
• Methodology reference: Waleed Ahmad's MoQ (w-ahmad); kaitchup substack article
• Calibration data: WikiText-2 test set (~80k lines), chunks=100
• Multi-benchmark validation date: 2026-06-24 (Phase 5)
• Build date: 2026-06-23
• SHA256: 587840e75895199e5ad771bfa7dfd9682f6d85ae295ad00001b78adb485c52c1

Reproduce

# Clone recipes from kaitchup
wget https://huggingface.co/kaitchup/Qwen3.6-27B-GGUF-MoQ/resolve/main/MoQ_qwen3.6-27b_tensors_4.0.txt

# Download Qwopus BF16 shards and convert to F16 GGUF
# (see Jackrong/Qwopus3.6-27B-Coder for conversion instructions)

# Generate imatrix
llama-imatrix -m qwopus_f16.gguf -f wikitext-2-raw/wiki.test.raw \
  -o qwopus_imatrix.gguf -ngl 50 --chunks 100

# Quantize
llama-quantize --imatrix qwopus_imatrix.gguf \
  --tensor-type-file MoQ_qwen3.6-27b_tensors_4.0.txt \
  qwopus_f16.gguf qwopus_moq_4.0.gguf Q4_K

# Validate
./llama-perplexity -m qwopus_moq_4.0.gguf -f wikitext-2-raw/wiki.test.raw \
  -c 2048 --chunks 30

License

Apache 2.0 (inherited from base model)