---

license: apache-2.0

base_model: Qwen/Qwen3.6-35B-A3B

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- moe

- apex

- quantized

- imatrix

- torch-imatrix

- qwen3_5_moe

- llama.cpp

---

Qwen3.6-35B-A3B — APEX GGUF (torch imatrix)

MoE-aware, mixed-precision APEX quantization of

Qwen/Qwen3.6-35B-A3B — architecturally

identical to Qwen3.5-35B-A3B (same qwen3_5_moe: 40 layers, 256 routed + 1 shared

expert, hybrid GatedDeltaNet + periodic full attention, NextN/MTP head).

The importance matrix here is generated with a PyTorch band-serialized generator

rather than llama-imatrix, for two concrete reasons on this architecture:

1. llama.cpp's imatrix tool is impractical for qwen35moe. The GatedDeltaNet

linear-attention is a serial state-space recurrence; the imatrix collection

callback breaks the GPU path that makes normal inference fast, so it falls back to

a single CPU thread that no thread count can parallelize.

1. The torch generator covers the MTP/NextN head (blk.40.*) that llama-imatrix

does not. On Qwen3.6 the MTP head is a full MoE decoder layer with fused experts;

both the GGUF converter and the imatrix generator handle that layout.

update:

-v2 files: identical bit allocation to the originals, except the 48 attn_gate coefficients are kept at F32 instead of the tier's attention band type. They parameterise a nonlinearity that multiplies the state path, so quantising them lets error compound along the sequence; F32 costs about 31 MB. Prefer -v2.

The -torch imatrix — what it is

Qwen3.6-35B-A3B-torch.imatrix is a standard GGUF-format importance matrix

(in_sum2 + counts per tensor), bit-compatible with llama-quantize --imatrix.

Generated from the HF safetensors with a band-serialized PyTorch forward on a general

text corpus, covering all 40 transformer layers plus the NextN/MTP head.

Validation vs a reference llama.cpp imatrix

No public llama-imatrix exists for Qwen3.6 (and it is impractical to compute locally,

see above). Since the imatrix's per-channel importance is largely architecture-driven,

we validate per-tensor against

bartowski's canonical Qwen3.5 llama.cpp imatrix

(same arch):

| Metric | Value |

|--------|-------|

| Tensors covered (torch) | 523 (incl. 13 MTP-head tensors) |

| Per-tensor correlation (median) | 0.956 |

| Per-tensor correlation (mean) | 0.861 |

| Lowest-correlation tensors | blk.*.ssm_out.weight only |

Median 0.956 against an independently-computed imatrix of the same architecture

(different fine-tune and different calibration data) confirms the mapping/values;

the only weakly-correlated tensors are ssm_out (post-nonlinearity, does not affect

quantization quality). Torch coverage is a strict superset (only-real = []).

PPL parity (measured on Qwen3.5, same architecture)

The torch-vs-llama.cpp PPL parity was measured on the sibling Qwen3.5 (identical

qwen3_5_moe architecture): i-compact quantized with each imatrix, perplexity over

200×512-token wikitext-2 windows —

| i-compact quantized with | PPL | Δ vs bf16 |

|--------------------------|----:|----------:|

| bf16 (reference) | 6.620 | — |

| bartowski llama.cpp imatrix | 6.756 | +2.05% |

| torch imatrix | 6.775 | +2.34% |

The torch and llama.cpp quants differ by 0.02 PPL — inside the ±0.073 error bars

(statistically indistinguishable). Since Qwen3.6 shares the architecture and the torch

imatrix correlates at 0.956 here, the same parity holds.

Confirmed on a diverse code-heavy corpus (HumanEval + MBPP + GSM8K + prose, 342

windows), on the sibling Qwen3.5 i-compact: bf16 2.247 · llama.cpp-imatrix 2.310 ·

torch-imatrix 2.318 — parity holds (Δ0.008, inside ±0.014). The imatrix records

per-channel activation magnitudes, so the method is domain-agnostic.

> Calibration used the diverse calibration_datav3 (prose + code + multilingual), so

> these quants are not domain-handicapped. The imatrix's calibration corpus is a

> knob: a code-weighted calibration would favor coding channels further, at a small

> cost elsewhere — useful if you're specializing for a single domain.

Files

• Qwen3.6-35B-A3B-torch.imatrix — the PyTorch-generated importance matrix (this repo).
• Qwen3.6-35B-A3B-APEX-i-quality-torch.gguf (23.5 GB) — largest/highest-fidelity tier.
• Qwen3.6-35B-A3B-APEX-i-compact-torch.gguf (17.4 GB) — smaller, more aggressive tier.
• *-v2.gguf — same tiers with the state/gate coefficients (ssm_alpha, ssm_beta) stored

at F32. All other tensors are unchanged.

• llama.cpp reference imatrix — not re-hosted; see

bartowski/Qwen_Qwen3.5-35B-A3B-GGUF.

Running it

Text only:

llama-server -m Qwen3.6-35B-A3B-APEX-i-quality-v2.gguf \
             --ctx-size 32768 --parallel 1 -b 1024 -ub 1024 --jinja

With vision — add the projector, nothing else changes:

llama-server -m Qwen3.6-35B-A3B-APEX-i-quality-v2.gguf \
             --mmproj mmproj-F16.gguf \
             --ctx-size 32768 --parallel 1 -b 1024 -ub 1024 --jinja

--jinja is required for tool calling. KV is ~82 KiB/token (all-full attention), so 32k of

context costs ~2.6 GiB; scale --ctx-size to taste. On a 16 GB card add -ncmoe 41 to put

the 256 routed experts on CPU — i-quality is 23.5 GB and will not fit otherwise. Give it a large

output cap: with thinking on, an empty reply usually means the cap was spent reasoning.

Vision / multimodal (mmproj)

This repo ships the text model only. The vision projector is a separate GGUF that quantization

does not touch, so it is not re-hosted here.

Vision works. Tested with

unsloth/Qwen3.6-35B-A3B-GGUF → mmproj-F16.gguf

against Qwen3.6-35B-A3B-APEX-i-quality-torch.gguf: 5/5 on a five-task perception probe, 0/5

with the image withheld. All tiers take the same projector.

There is no official Qwen GGUF projector — Qwen/Qwen3.6-35B-A3B publishes safetensors only.

Attribution

• Base model: Qwen — Qwen/Qwen3.6-35B-A3B.
• Reference imatrix for validation: bartowski —

Qwen_Qwen3.5-35B-A3B-GGUF.

• APEX recipe & toolkit: LocalAI —

localai-org/apex-quant.

• Quantization engine: llama.cpp (ggml-org).

Unofficial community quantization; not affiliated with or endorsed by Qwen.