---

license: apache-2.0

base_model: CCSSNE/trohrbaugh-Qwen3.5-122B-A10B-heretic

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- qwen3.5

- qwen3.5-moe

- heretic

- rocmfp4

- imatrix

- amd

- vulkan

- rocm

- mtp

- speculative-decoding

- experimental

---

<div align="center">

Qwen3.5 122B-A10B Heretic · ROCmFP4 iMatrix

A compact, importance-calibrated ROCmFP4 build for high-memory AMD systems

122B total · 10B active · 60.70 GiB · 28.45 tok/s native · 33.68 tok/s MTP · BF16 KLD 0.100716 · 28.11 Decode t/s

Decode Speed 34.16% Faster

Size 13.47gb Smaller.

</div>

> [!IMPORTANT]

> This GGUF uses custom ROCmFP4 tensor types. It requires

> ROCmFPX or a runtime with equivalent

> support; stock llama.cpp, Ollama, LM Studio, and similar stock runtimes

> cannot load it.

I also suggest usign the template from https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates Froggeric is tha dude.

Downloads

| Artifact | Direct download |

|---|---|

| Main iMatrix model — 60.70 GiB | Download Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix.gguf |

| Optional MTP companion — 2.14 GiB | Download Qwen3.5-122B-A10B-Heretic-ROCmFP4-MTP.gguf |

| Calibration matrix — 342.28 MiB | Download Qwen3.5-122B-A10B-Heretic-code-technical.imatrix |

Browse every repository file →

What it is

This is an iMatrix-calibrated quantization of the full-weight Qwen3.5 122B-A10B

Heretic checkpoint. It was built with ROCmFPX's compact

Q4_0_ROCMFP4_STRIX_LEAN recipe, but the hardware-oriented preset name has

been left out of the public filename.

| | Result |

|---|---:|

| BF16 mean KLD | 0.100716 ± 0.003407 |

| Two-sigma KLD upper bound | 0.107530 |

| Improvement from iMatrix | 14.01% lower KLD |

| Greedy decode | 28.448 tok/s |

| Greedy decode with MTP | 33.677 tok/s |

| 4,277-token prefill | 350.248 tok/s |

Files

| File | Purpose | Size | SHA-256 |

|---|---|---:|---|

| Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix.gguf | Main text model | 65,184,265,248 bytes | f563fba693c65a05431d4beecd11736edf7e54208902cae70d7b725057a7556e |

| Qwen3.5-122B-A10B-Heretic-ROCmFP4-MTP.gguf | Optional external MTP companion | 2,294,290,880 bytes | 5554538448efb668099be898ce7c8da5920e86b321f45e8ac1f7dca3518cc0a8 |

| Qwen3.5-122B-A10B-Heretic-code-technical.imatrix | Calibration matrix | 358,906,432 bytes | 3a2a0e0bea5fe380a27cc938ad02e727dcbd840744ab3ddb3a8e8402a67deb05 |

| chat_template.jinja | Pinned Qwen3.5 chat template | 7,756 bytes | a4aee8afcf2e0711942cf848899be66016f8d14a889ff9ede07bca099c28f715 |

The main model is complete and runs independently. The 23-tensor, one-layer

MTP file is an optional external companion; no MTP layer is embedded in the

main GGUF. Runtimes call this role a draft model, but Draft is omitted from

the public filename because it does not mean unfinished.

Quality

The calibrated model and its uncalibrated control were replayed against the

same saved distributions from the exact BF16 parent on ROCm0.

| Build | Mean KLD | PPL ratio | Same top | RMS Δp |

|---|---:|---:|---:|---:|

| iMatrix | 0.100716 ± 0.003407 | 1.080217 ± 0.010198 | 87.629% ± 0.622% | 10.881% |

| Uncalibrated | 0.117119 ± 0.004183 | 1.109251 ± 0.011299 | 86.774% ± 0.640% | 12.077% |

iMatrix calibration reduced mean KLD by 14.01%, reduced the PPL ratio by

0.029034, and improved same-top agreement by 0.855 percentage points. The

two-sigma KLD upper bound is 0.107530, clearing the selected 0.126

performance-oriented target.

The paired replay covers 11 complete 512-token chunks and 2,805 evaluated

next-token distributions. These were recovered at an exact probability-block

boundary from an interrupted 32-chunk BF16 run. This is useful preliminary

evidence, not a completed 32-chunk acceptance gate.

Performance

Measured on a 128 GB AMD Strix Halo system with Vulkan/RADV, 131,072 context,

parallel 1, batch/ubatch 2048/1024, Q8_0 KV cache, flash attention, and MTP off.

Values are medians from repeated runs.

| Workload | Repeats | Decode | Prompt processing |

|---|---:|---:|---:|

| Greedy, 256 generated tokens | 5 | 28.448 tok/s | 71.929 tok/s |

| Sampled, 256 generated tokens | 3 | 28.418 tok/s | 71.294 tok/s |

| 4,277-token prefill + 128 generated | 3 | 28.041 tok/s | 350.248 tok/s |

Relative to the otherwise matched uncalibrated build, decode improved by

2.43%, 5.23%, and 2.84% respectively. Long-prompt prefill changed by -1.03%.

These are single-system measurements, not general performance guarantees.

Optional MTP acceleration

The external MTP companion was tested against this exact iMatrix trunk in a

matched MTP-off/on A/B. The model, runtime, context, cache, batch geometry,

prompts, and companion identity were held constant; only MTP execution changed.

| Workload | Native decode | MTP decode | Decode gain | Acceptance | End-to-end gain |

|---|---:|---:|---:|---:|---:|

| Greedy, 256 tokens | 28.328 tok/s | 33.677 tok/s | +18.89% | 85.03% | +16.44% |

| Sampled, 256 tokens | 28.035 tok/s | 30.953 tok/s | +10.41% | 80.00% | +8.76% |

| 4,277-token prefill + 128 | 27.585 tok/s | 33.752 tok/s | +22.36% | 90.73% | -0.45% |

MTP reduced prompt-processing throughput by about 7% because the companion must

catch up on the prompt. It therefore helps most when generation is long enough

to repay that fixed cost. The exact identity probe was unchanged, and both the

MTP-off and MTP-on lanes passed technical prose, executable code semantics,

strict JSON, tool-calling, balanced-analysis, and defensive-safety smokes.

Quantization

The model was quantized once from the validated BF16 GGUF using a 339-chunk

code/technical importance matrix and the Q4_0_ROCMFP4_STRIX_LEAN preset.

| Tensor type | Count |

|---|---:|

| Q4_0_ROCMFP4_FAST | 457 |

| Q4_0_ROCMFP4 | 60 |

| F32 | 361 |

| Q5_K | 1 |

In this preset, attention K/V tensors retain dual-scale ROCmFP4 protection,

most transformer weights use the compact FAST layout, and token

embeddings/output use Q5_K. The artifact contains 879 tensors across 48 blocks

and no embedded MTP layer.

Run

llama-server \
  --model Qwen3.5-122B-A10B-Heretic-ROCmFP4-iMatrix.gguf \
  --host 127.0.0.1 --port 8080 \
  -dev Vulkan0 --n-gpu-layers 999 \
  --ctx-size 131072 --parallel 1 \
  --flash-attn on --batch-size 2048 --ubatch-size 1024 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --jinja --reasoning-format deepseek

Adjust context and cache settings for your memory budget. The tested runtime

was charlie12345/ROCmFPX commit

a6a93765f7ce9779c13f9881164a65f7a9f31198, built in Release mode for

gfx1151 with Vulkan and HIP enabled, plus a local Qwen3.5 MoE graph

correction. Results with newer ROCmFPX revisions have not yet been established.

To enable the optional companion, append:

  --spec-type draft-mtp \
  --spec-draft-model Qwen3.5-122B-A10B-Heretic-ROCmFP4-MTP.gguf \
  --spec-draft-device Vulkan0 --spec-draft-ngl 999 \
  --spec-draft-type-k f16 --spec-draft-type-v f16 \
  --spec-draft-n-max 2 --spec-draft-p-min 0.6 \
  --spec-draft-p-split 0.10 --spec-draft-backend-sampling

Tested system

| | |

|---|---|

| Platform | AMD Strix Halo, 128 GB unified memory |

| GPU target | Radeon 8060S / gfx1151 |

| Backend | Vulkan / RADV |

| Kernel | Linux 6.17.0-1028-oem |

| Mesa | 25.2.8 |

No vision projector is included; treat this release as text-only.

Lineage and credits

This repository exists because several people published substantial work

openly. Their roles are kept distinct so downstream users can find the original

sources.

• Official base model: Qwen/Qwen3.5-122B-A10B, Apache 2.0.
• Full-weight Heretic source: trohrbaugh/Qwen3.5-122B-A10B-heretic, accessed through the pinned CCSSNE mirror at fd6d6bc89e9be47577736c952dd4d1d7c830f09f.
• MTP tensor source: OptimizeLLM/Qwen3.5-122B-A10B-heretic-MTP-NVFP4 at 07b7c2101c33f23a84fd3c618d00f17ea12269c3; its model card reports that the MTP tensors were restored from the official Qwen checkpoint.
• Heretic software and method: p-e-w/heretic, created by Philipp Emanuel Weidmann. The source checkpoint reports Heretic v1.2.0.
• Related full-weight Heretic reference: scrunter/Qwen3-VL-235B-A22B-Thinking-heretic. This related 235B model is not the tensor source for this release.
• ROCmFP4 implementation, quantizer, and runtime: charlie12345/ROCmFPX.
• Strix Halo deployment stack: hec-ovi/llama-vulkan-strix.
• Release inspiration: plunderstruck/Qwen3.6-27B-MTP-ROCmFP4-GGUF.
• Conversion, validation, benchmarking, and packaging: vmlinux.

Please preserve this lineage, the Apache 2.0 license, and a description of your

changes when redistributing derivatives.

License

The distributed model derivative is provided under the upstream Apache License

2.0. The included LICENSE is copied from the pinned full-weight source.

Runtime and tooling repositories retain their own licenses: Heretic software is

AGPL-3.0, while ROCmFPX and the deployment tooling are separately licensed. No

runtime source code is bundled in this model repository.