---

base_model: LuffyTheFox/Qwen3.5-9B-Uncensored-Genesis-BF16-GGUF

library_name: gguf

pipeline_tag: image-text-to-text

license: apache-2.0

tags:

- gguf

- nvfp4

- fp4

- imatrix

- blackwell

- llama.cpp

- qwen3.5

- uncensored

language:

- en

- zh

---

Qwen3.5-9B-Uncensored-Genesis — NVFP4 imatrix GGUF

NVFP4 quantisations of

LuffyTheFox/Qwen3.5-9B-Uncensored-Genesis-BF16-GGUF,

built with michaelw9999/advanced-gguf-quantizer

(a llama.cpp fork focused on NVFP4/MXFP6 quantization).

Files

v3 — Recommended (correct pipeline)

| File | Calibration | Size | Tensors |

|---|---|---|---|

| ...NVFP4-v3.gguf | none (data-free) | 6.53 GiB | 819 |

| ...NVFP4-v3-imx-v5.gguf | v5 — plain text | 6.53 GiB | 819 |

| ...NVFP4-v3-imx-v6.gguf | v6 — chat-template rendered | 6.53 GiB | 819 |

v2 / original — Deprecated (flawed pipeline)

| File | Calibration | Size | Tensors |

|---|---|---|---|

| ...NVFP4.gguf | none (data-free) | 5.3 GB | 915 |

| ...NVFP4-v2.gguf | none (data-free) | 5.3 GB | 915 |

| ...NVFP4-v2-imx-v5.gguf | v5 — plain text | 5.3 GB | 915 |

| ...NVFP4-v2-imx-v6.gguf | v6 — chat-template rendered | 5.3 GB | 915 |

The v2/original files used a --tensor-type .*=nvfp4 catch-all that

wrongly forced output.weight to NVFP4 (should be Q6_K) and all FFN

tensors to NVFP4 (should stay F16). The v3 files use the correct

pipeline — see the Pipeline fix section below.

Source

| | |

|---|---|

| Source GGUF | Qwen3.5-9B-Uncensored-Genesis-BF16.gguf (16719 MiB, 16.00 BPW) |

| Architecture | qwen35 (dense Qwen3.5), 9.22 B params, 32 layers |

| Context | 262144 |

| general.file_type | 39 (LLAMA_FTYPE_MOSTLY_NVFP4) |

| MTP/NextN | none (no nextn_predict_layers) |

Requantised from BF16 — no extra rounding step from a lower-precision source.

Pipeline fix (v2 → v3)

The v2/original files were quantized with a --tensor-type .*=nvfp4

catch-all that forced every eligible tensor to NVFP4, overriding the

default NVFP4 policy that keeps sensitive tensors at higher precision.

The v3 files drop the catch-all and use NVFP4 as the output type, letting

the default policy protect sensitive tensors:

| Tensor | v2 (flawed) | v3 (correct) |

|---|---|---|

| output.weight | NVFP4 | Q6_K |

| ffn_down.weight (all blocks) | NVFP4 | F16 (blk.0, blk.13) / NVFP4 (rest) |

| ffn_gate.weight (all blocks) | NVFP4 | NVFP4 |

| ffn_up.weight (all blocks) | NVFP4 | NVFP4 |

| token_embd.weight | NVFP4 | F16 |

| ssm_beta.weight | NVFP4 | BF16 (kept as source) |

The v3 files are slightly larger (6.53 GiB vs 5.3 GB) because Q6_K and F16

tensors take more space than NVFP4, but the output projection and embedding

quality is preserved.

Imatrix variants

NVFP4 uses per-block input scales. Without an imatrix those scales fall back to

identity (all 1.0). An imatrix pass measures how much each weight influences

activations on real text, and the quantiser spends the input scale budget where

it matters.

v5 — plain text. Bartowski's v5 calibration set, ~1.7 MB of plain prose.

Broad language coverage, no chat-template structure. 256 chunks.

v6 — chat-template rendered. Bartowski prose plus 173 conversations (137

base + 36 tool-calling) rendered through the Qwen3.5-9B native chat template

(Qwen/Qwen3.5-9B). Rendering preserves the special/control tokens and role

structure the model sees at inference time, so the activation statistics better

match chat/tool-use workloads. llama-imatrix was run with --parse-special

so the rendered special-token markup is treated as real tokens, not literal

text. 256 chunks.

For chat, reasoning, or tool calling: use v6. For plain completion or broad

text work: v5 is fine. v3 (no imatrix) is the baseline.

> No PPL/KLD measurements are provided. Quantized-model perplexity and KLD

> against the BF16 source have not been measured. The imatrix PPL values

> reported by llama-imatrix during calibration reflect the source model's

> perplexity on the calibration text, not the quantized model's quality. They

> are not comparable across variants and are omitted for that reason.

Tensor protection policy

The protection policy follows Luffy's per-tensor quant policy, mapped from the

MoE Qwen 3.6 35B (ffn_down_exps) to this dense model (ffn_down).

F16 singular-collapse protection:

| tensor | type |

|---|---|

| blk.0.attn_gate.weight | F16 |

| blk.0.attn_qkv.weight | F16 |

| blk.0.ffn_down.weight | F16 |

| blk.13.ffn_down.weight | F16 |

F32 architecture-specific protection:

| tensor | reason |

|---|---|

| blk.*.attn_norm.weight | 1D norm — avoids emitting .scale/.input_scale the loader does not declare |

| blk.*.post_attention_norm.weight | " |

| blk.*.attn_q_norm.weight | " |

| blk.*.attn_k_norm.weight | " |

| blk.*.ssm_norm.weight | " |

| output_norm.weight | " |

| blk.*.ssm_conv1d.weight | patched CUDA SSM conv kernel requires F32 input |

| blk.*.ssm_dt.bias | SSM scalar, kernel compatibility |

| blk.*.ssm_a | SSM scalar, kernel compatibility |

Forced NVFP4 (do not push lower, collapses):

| tensor | type |

|---|---|

| blk.0.ssm_out.weight | NVFP4 |

| blk.1.attn_gate.weight | NVFP4 |

| blk.1.attn_qkv.weight | NVFP4 |

Default NVFP4 policy (no catch-all):

The v3 pipeline uses NVFP4 as the output type without a .*=nvfp4

catch-all. The default NVFP4 policy keeps these at higher precision:

• output.weight → Q6_K
• token_embd.weight → F16

Everything else eligible takes NVFP4.

Tensor mix (v3)

| type | count | size |

|---|---|---|

| F32 | 593 | 16.1 MiB |

| F16 | 5 | 2228.0 MiB |

| Q6_K | 1 | 795.7 MiB |

| BF16 | 24 | 6.0 MiB |

| NVFP4 | 196 | 3627.0 MiB |

| total | 819 | 6.52 GiB |

• general.file_type = 39 (NVFP4)
• qwen35.block_count = 32
• No nextn_predict_layers (no MTP)
• No unexpected 1D norm .scale tensors

Usage

llama-cli -m Qwen3.5-9B-Uncensored-Genesis-NVFP4-v3-imx-v6.gguf \
  -ngl 99 -c 4096 --temp 0.6 --top-k 20 --top-p 1.0 --min-p 0.0

Sampling follows Genesis guidance (temp 0.6, top_k 20, top_p and min_p

disabled). For code, also set --repeat-penalty 1.0 — code repeats identifiers

by nature, and any penalty pushes the model off the identifier it just chose.

Vision

Qwen3.5-9B is a vision-language model. The text model is quantised here; the

multimodal projection (mmproj-Qwen3.5-9B-Uncensored-Genesis-BF16.gguf) is

available in the source repo and should be used unmodified alongside these

weights.

Hardware

• Blackwell (RTX 50xx): native FP4 path, fastest. Build llama.cpp with

BLACKWELL_NATIVE_FP4 = 1.

• Ampere (RTX 30xx): NVFP4 inference works via fallback kernels. The F32

SSM protections prevent a CUDA assert in the patched SSM convolution kernel

on this architecture.

• Quantisation was done CPU-only because the Ampere CUDA NVFP4 encoder is

inefficient (hangs/spins at 99% GPU usage with ~300MB VRAM).

Reproducibility

v5 imatrix

llama-imatrix \
  -m Qwen3.5-9B-Uncensored-Genesis-BF16.gguf \
  -f calibration_datav5.txt \
  -o imatrix_v5.dat \
  --output-format dat \
  -ngl all -b 2048 -ub 512 -t 6 --chunks 256

v6 imatrix

Conversations rendered through the Qwen3.5-9B native chat template with

`transformers.AutoTokenizer.apply_chat_template(..., tools=tools,

add_generation_prompt=False, tokenize=False)`, concatenated with prose, fed to

llama-imatrix with --parse-special:

llama-imatrix \
  -m Qwen3.5-9B-Uncensored-Genesis-BF16.gguf \
  -f calibration_v6_rendered.txt \
  -o imatrix_v6.dat \
  --output-format dat \
  -ngl all -b 2048 -ub 512 -t 6 --chunks 256 \
  --parse-special

Quantisation (v3 — correct pipeline)

All v3 variants use the same protection policy and CPU-only quantise command,

differing only in the imatrix file (or its absence for v3 baseline):

llama-quantize \
  --allow-requantize \
  --tensor-type-file tensor_types_protection.txt \
  --imatrix imatrix_v6.dat \
  Qwen3.5-9B-Uncensored-Genesis-BF16.gguf \
  Qwen3.5-9B-Uncensored-Genesis-NVFP4-v3-imx-v6.gguf \
  NVFP4 6

Key differences from the v2 pipeline:

• Uses NVFP4 as the output type (not Q8_0 with .*=nvfp4 catch-all)
• Uses --tensor-type-file for per-tensor overrides (not inline --tensor-type regex)
• No --mode fast (unnecessary, only affects selector)
• The default NVFP4 policy keeps output.weight at Q6_K and token_embd.weight at F16

Credits

• Base model and finetune: LuffyTheFox
• Per-tensor protection policy: LuffyTheFox
• Chat template for v6 calibration:

Qwen/Qwen3.5-9B native template

• Quantiser: michaelw9999/advanced-gguf-quantizer
• Calibration corpus: Bartowski v5 plain text; v6 prose + conversations rendered

through the Qwen3.5-9B chat template