---

license: apache-2.0

language:

- en

- zh

- multilingual

tags:

- gguf

- llama.cpp

- qwen

- qwen3.6

- moe

- mxfp4

- mtp

- speculative-decoding

- text-generation

pipeline_tag: text-generation

base_model:

- HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive

- khoi11/Qwen3.6-35B-A3B-Uncensored-HauhauCS-FP16

---

hauhau_qwen36_35b_a3b_mxfp4_gguf_mtp

This repo contains a local MXFP4_MOE GGUF export of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive together with a separate MTP draft GGUF for speculative decoding in a patched local llama.cpp toolchain.

Files

• hauhau_qwen36_35b_a3b_mxfp4_robinstyle.gguf: main text model in robinshao-style MXFP4 layout
• hauhau_qwen36_35b_a3b_mxfp4_mtp.gguf: MTP draft GGUF used with --spec-type draft-mtp
• llama.cpp.src-patched.zip: patched local llama.cpp source snapshot used for conversion and serving
• build_result.json: build and validation summary for the main model and MTP draft
• mtp-benchmark.json: short local speed comparison between baseline and MTP serving

Quantization Notes

• Source FP16 weights: khoi11/Qwen3.6-35B-A3B-Uncensored-HauhauCS-FP16
• Upstream model lineage: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
• Main model converted for text inference and quantized into a robinshao-style MXFP4_MOE GGUF
• MTP draft exported as a separate GGUF for speculative decoding with patched llama.cpp
• Vision tensors are not included in the final serving package

Local Deployment Notes

This pair was validated locally on Windows with patched llama.cpp using speculative decoding:

llama-server -m hauhau_qwen36_35b_a3b_mxfp4_robinstyle.gguf \
  --spec-type draft-mtp \
  --spec-draft-model hauhau_qwen36_35b_a3b_mxfp4_mtp.gguf \
  --spec-draft-device CUDA0 \
  --spec-draft-ngl all \
  --jinja \
  --device CUDA0 \
  --gpu-layers all \
  --ctx-size 8192 \
  --batch-size 768 \
  --ubatch-size 128 \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --cache-ram 0

In a short local API benchmark on an RTX 4000 Ada, the MTP setup improved decode speed from about 103.45 tok/s to about 131.65 tok/s, roughly 1.27x.

Caveats

• This is a text-only export, not the original multimodal package.
• The draft file is intended for speculative decoding and is not a standalone replacement for the main model.
• Native training context metadata remains 262144, but the actual no-offload local limit depends on GPU VRAM, KV cache settings, and whether the draft model is loaded.