---

base_model: HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF

library_name: gguf

pipeline_tag: text-generation

license: apache-2.0

tags:

- gguf

- nvfp4

- fp4

- imatrix

- blackwell

- llama.cpp

- qwen3.8

- uncensored

- fastmtp

language:

- en

- zh

---

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive — NVFP4 imatrix GGUF

NVFP4 quantisations of

HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF,

built with michaelw9999/advanced-gguf-quantizer

(a llama.cpp fork focused on NVFP4/MXFP6 quantization).

Three calibration variants, each with MTP and noMTP builds:

| File | Calibration | MTP | Size | Tensors |

|---|---|---|---|---|

| ...NVFP4-v2.gguf | none (data-free) | yes | 14.68 GB | 1866 |

| ...NVFP4-v2-noMTP.gguf | none | no | 14.46 GB | 1835 |

| ...NVFP4-v2-imx-v5.gguf | v5 — plain text | yes | 14.68 GB | 1866 |

| ...NVFP4-v2-imx-v5-noMTP.gguf | v5 — plain text | no | 14.46 GB | 1835 |

| ...NVFP4-v2-imx-v6-froggeric.gguf | v6 — chat-template rendered | yes | 14.68 GB | 1866 |

| ...NVFP4-v2-imx-v6-froggeric-noMTP.gguf | v6 — chat-template rendered | no | 14.46 GB | 1835 |

All six share the same source, protection policy, and architecture profile

(qwen35dense). They differ only in the calibration data used to compute NVFP4

input scales (or, for v2, the absence of it).

Source

| | |

|---|---|

| Source GGUF | Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf (29990 MiB, 9.21 BPW) |

| Architecture | qwen35 (dense Qwen3.8), 27.32 B params, 64 layers + 1 MTP/NextN head |

| Context | 262144 |

| general.file_type | 39 (LLAMA_FTYPE_MOSTLY_NVFP4) |

Requantised from Q8_K_P, not BF16. Tensors that were q8 in the source go through

one extra rounding step; bf16 source tensors requantise cleanly.

Imatrix variants

NVFP4 uses per-block input scales. Without an imatrix those scales fall back to

identity (all 1.0). An imatrix pass measures how much each weight influences

activations on real text, and the quantiser spends the input scale budget where

it matters.

v5 — plain text. Bartowski's v5 calibration set, ~1.7 MB of plain prose.

Broad language coverage, no chat-template structure.

v6 — chat-template rendered. Bartowski prose plus 173 conversations (137

base + 36 tool-calling) rendered through the target model's chat template using

froggeric/Qwen-Fixed-Chat-Templates

v22.5. Rendering preserves the special/control tokens and role structure the

model sees at inference time, so the activation statistics better match

chat/tool-use workloads. llama-imatrix was run with --parse-special so the

rendered special-token markup is treated as real tokens, not literal text.

For chat, reasoning, or tool calling: use v6. For plain completion or broad

text work: v5 is fine. v2 (no imatrix) is the baseline.

Tensor protection policy

The protection policy follows Luffy's per-tensor quant policy, mapped from the

MoE Qwen 3.6 35B (ffn_down_exps) to this dense model (ffn_down).

Implemented with anchored --tensor-type regex overrides (last-match-wins, so

patterns are anchored with ^...$ to avoid partial matches like ssm_a

matching ssm_alpha).

F16 singular-collapse protection:

| tensor | type |

|---|---|

| blk.0.attn_gate.weight | F16 |

| blk.0.attn_qkv.weight | F16 |

| blk.0.ffn_down.weight | F16 |

| blk.13.ffn_down.weight | F16 |

Note: blk.0.attn_gate / attn_qkv were q8_0 in the source, so F16 here stores

dequantised q8 values — keeps q8-level fidelity instead of dropping to 4 bits,

but is not true F16 precision.

F32 architecture-specific protection (not in the original Luffy policy;

derived from runtime failures during testing of the sibling Genesis V1 build):

| tensor | reason |

|---|---|

| blk.*.attn_norm.weight | 1D norm — avoids emitting .scale/.input_scale the FastMTP loader does not declare |

| blk.*.post_attention_norm.weight | " |

| blk.*.attn_q_norm.weight | " |

| blk.*.attn_k_norm.weight | " |

| blk.*.ssm_norm.weight | " |

| blk..nextn.norm.weight | MTP/NextN 1D norms — same reason |

| output_norm.weight | " |

| blk.*.ssm_conv1d.weight | patched CUDA SSM conv kernel requires F32 input |

| blk.*.ssm_dt.bias | SSM scalar, kernel compatibility |

| blk.*.ssm_a | SSM scalar, kernel compatibility |

Without the 1D-norm F32 protection, the quantiser emits 198 extra .scale /

.input_scale tensors that the FastMTP loader at the investigated revision does

not declare, causing a tensor-count mismatch (expected 1866, got 2064).

Forced NVFP4 (do not push lower, collapses):

| tensor | type |

|---|---|

| blk.0.ssm_out.weight | NVFP4 |

| blk.1.attn_gate.weight | NVFP4 |

| blk.1.attn_qkv.weight | NVFP4 |

Everything else eligible takes NVFP4.

Tensor mix

| type | count |

|---|---|

| F32 | 1360 |

| NVFP4 | 502 |

| F16 | 4 |

| total | 1866 |

• general.file_type = 39 (NVFP4)
• qwen35.block_count = 65 (64 + 1 MTP)
• qwen35.nextn_predict_layers = 1
• No unexpected 1D norm .scale tensors

noMTP derivatives strip the 31 blk.64.* tensors and the

qwen35.nextn_predict_layers metadata key, decrement block_count 65 → 64,

and end at 1835 tensors / 14.46 GB.

FastMTP

The MTP variants are designed for

HauhauCS FastMTP

using the prebuilt draft sidecar from the same repo. The sibling Genesis V1

build was tested with the same architecture and FastMTP loader; this build

shares the identical tensor layout and protection policy.

Usage

llama-cli -m Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-v2-imx-v6-froggeric.gguf \
  -ngl 99 -c 4096 --temp 0.6 --top-k 20 --top-p 1.0 --min-p 0.0

Sampling follows Genesis guidance (temp 0.6, top_k 20, top_p and min_p

disabled). For code, also set --repeat-penalty 1.0 — code repeats identifiers

by nature, and any penalty pushes the model off the identifier it just chose.

Give it room to think

This model reasons at length before answering. It can spend 900, even 2600

tokens inside the thinking block without reaching its final answer. Budget

generously (-n 4096 or more) for anything non-trivial.

FastMTP

llama-server \
  -m Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-v2-imx-v6-froggeric.gguf \
  --spec-draft-model Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf \
  --spec-draft-ngl all \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0 \
  --ctx-size 32768 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --n-gpu-layers all \
  --split-mode none \
  --flash-attn on \
  --jinja

Use the MTP file for FastMTP, not the noMTP derivative. The noMTP file is

for ordinary inference where the MTP/NextN head is not wanted.

Hardware

• Blackwell (RTX 50xx): native FP4 path, fastest. Build llama.cpp with

BLACKWELL_NATIVE_FP4 = 1.

• Ampere (RTX 30xx): NVFP4 inference works via fallback kernels. The F32

SSM protections prevent a CUDA assert in the patched SSM convolution kernel

on this architecture.

• Quantisation was done CPU-only because the Ampere CUDA NVFP4 encoder is

inefficient (hangs/spins at 99% GPU usage with ~300MB VRAM).

Reproducibility

v5 imatrix

llama-imatrix \
  -m Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf \
  -f calibration_datav5.txt \
  -o imatrix_v5.dat \
  --output-format dat \
  -ngl all -ts 3,1 -b 2048 -ub 512 -t 6

v6 imatrix

Conversations rendered through froggeric v22.5 with

`transformers.AutoTokenizer.apply_chat_template(..., tools=tools,

add_generation_prompt=False, tokenize=False)`, concatenated with prose, fed to

llama-imatrix with --parse-special:

llama-imatrix \
  -m Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf \
  -f calibration_v6_froggeric_rendered.txt \
  -o imatrix_v6_froggeric.dat \
  --output-format dat \
  -ngl all -ts 3,1 -b 2048 -ub 512 -t 6 \
  --parse-special

Quantisation

All variants use the same protection policy and CPU-only quantise command,

differing only in the imatrix file (or its absence for v2):

llama-quantize \
  --allow-requantize --mode fast \
  --tensor-type .*=nvfp4 \
  --tensor-type '^blk\..*\.attn_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.post_attention_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_q_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_k_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.nextn\..*norm\.weight$=f32' \
  --tensor-type '^output_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_conv1d\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_dt\.bias$=f32' \
  --tensor-type '^blk\..*\.ssm_a$=f32' \
  --tensor-type '^blk.0.ssm_out.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_gate.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_qkv.weight$=nvfp4' \
  --tensor-type '^blk.0.attn_gate.weight$=f16' \
  --tensor-type '^blk.0.attn_qkv.weight$=f16' \
  --tensor-type '^blk.0.ffn_down.weight$=f16' \
  --tensor-type '^blk.13.ffn_down.weight$=f16' \
  --imatrix imatrix_v6_froggeric.dat \
  Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf \
  Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-v2-imx-v6-froggeric.gguf \
  Q8_0 6

general.file_type is patched to 39 after quantisation.

noMTP derivatives

strip_mtp.py removes the 31 blk.64.* tensors and the

qwen35.nextn_predict_layers metadata key, and decrements

qwen35.block_count 65 → 64.

Credits

• Base model and finetune: HauhauCS
• Per-tensor protection policy: LuffyTheFox
• Chat template for v6 calibration:

froggeric/Qwen-Fixed-Chat-Templates

v22.5

• Quantiser: michaelw9999/advanced-gguf-quantizer
• Calibration corpus: Bartowski v5 plain text; v6 prose + conversations rendered

through the target chat template