---

license: apache-2.0

base_model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- nvfp4

- qwen3.6

- blackwell

- mtp

- speculative-decoding

- llama.cpp

language:

- en

- multilingual

---

Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-LOW-GGUF

A tensor-spliced hybrid GGUF of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP:

• Backbone (all 64 transformer blocks): native NVFP4, converted by us to GGUF from maci0's NVFP4 safetensors checkpoint.
• Extras (output head, token embedding, MTP draft head): DavidAU's "LOW" quantization choices, a Q5_0 output tensor plus IQ4_XS embedding and MTP tensors, taken from DavidAU's published GGUF.

The goal was simple: keep the NVFP4 prefill advantage, shrink the file below the 24 GB budget, and get faster decode via a higher-quality MTP draft head. On our dual 16 GB Blackwell setup it does fit and it does run. See the notes below before treating any numbers as meaningful.

Vision works too. The GGUF itself is text-only, but the model is vision-capable: pair it with the vision projector from DavidAU's NEO-MAX-MTP-GGUF repo, unmodified. DavidAU publishes mmproj-BF16.gguf, mmproj-F16.gguf, and mmproj-F32.gguf; we used the BF16 one ourselves (verified byte-identical), but any of the three works. No separate mmproj upload is needed here; just point --mmproj at one of his files.

What is this, exactly?

This is a single-file GGUF (1,858 tensors, 15.48 GB payload) with a mixed tensor layout:

| GGML type | Tensors | Size | Component |

|---|---|---|---|

| NVFP4 | 496 | 13.70 GB | all 64 transformer blocks (attention QKV/output, FFN gate/up/down, gates) |

| Q5_0 | 1 | 0.87 GB | output.weight (LM head) |

| IQ4_XS | 9 | 0.90 GB | token_embd.weight plus the full MTP draft block (blk.64.*, including nextn.eh_proj) |

| F32 | 1,352 | 0.01 GB | norms, gates, scales |

The MTP draft head is embedded in the GGUF, so no separate drafter file is needed. Enable it in llama.cpp with --spec-type draft-mtp.

Why these extra tensors? The MTP draft head's job is to predict tokens the main model will accept. Keeping it at IQ4_XS instead of NVFP4 preserves draft quality, which is what makes speculative decoding pay off. The Q5_0 LM head keeps output quality on the exact tensors that determine the final token, while the IQ4_XS embedding saves the most memory per byte (675 MB for the embedding vs 2.5 GB at BF16).

Attribution & provenance

This is a derivative work built entirely from existing Apache-2.0 artifacts. Nothing here was trained or fine-tuned. Credit belongs to:

1. Alibaba / Qwen team for the base model, Qwen/Qwen3.6-27B (Apache-2.0): dense 27B, 64 layers, Gated DeltaNet + Gated Attention hybrid layout, native 262,144-token context.
2. DavidAU for the fine-tune/merge DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP, and for the "LOW" quant design (special low-memory quants with Q5_0/Q6_K output tensor and IQ4_XS extras) from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF.
3. maci0 for the NVFP4 safetensors checkpoint maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 (Apache-2.0). Note: maci0 ships safetensors; the GGUF conversion was done by us (see "How this was made").
4. This repo's author for the tensor splice itself (combining the two sources) and this write-up.

The NVFP4 tensors are native GGML type 40, preserved from the source checkpoint through GGUF conversion with no re-quantization and no dequant-to-requant round trip. The Q5_0/IQ4_XS tensors are standard GGUF types taken from DavidAU's LOW quant.

How this was made

Two inputs, both Apache-2.0:

1. maci0's NVFP4 safetensors checkpoint was converted to GGUF by us using llama.cpp's convert_hf_to_gguf.py with --outtype auto (which preserves the native NVFP4 tensors as-is instead of dequantizing them). This produced the 19.65 GB all-NVFP4 backbone GGUF.
2. DavidAU's IQ4XS LOW GGUF from his NEO-MAX-MTP-GGUF repo was used as the source for the "LOW-style" extra tensors: output.weight (Q5_0), token_embd.weight (IQ4_XS), and the entire MTP draft block blk.64.* (IQ4_XS).

These extra tensors were then spliced into the NVFP4 backbone GGUF, replacing the corresponding backbone tensors. No weights were retrained or re-quantized in the process; each tensor is used exactly as it came from its source.

Repository contents

Besides the GGUF itself, this repo includes one file from our own work that others can reuse:

• overrides.txt: the per-tensor quantization-type map from our conversion planning (770 entries: every weight tensor and its target GGML type, e.g. blk.0.attn_qkv.weight=nvfp4, norms pinned to f32). Useful if you want to reproduce or audit the tensor layout without re-deriving it.

First observations (naive, single-run, not a benchmark)

We did not run a proper benchmark. What follows are informal first impressions from a handful of single-stream runs, included only so others know what to expect. Do not treat these as claims.

• Hardware: 2x NVIDIA Blackwell 16 GB (RTX 5070 Ti + RTX 5060 Ti), split-mode: tensor, llama.cpp via LocalAI, flash attention on, quantized KV cache.
• Prompt: one 180k-token payload (context 204,800). This payload was highly repetitive (it contained the same boilerplate unit 2,000 times), which likely inflates MTP acceptance and speedups; other prompts of different sizes behaved similarly, but this is far from comprehensive.
• Sampling: temperature 0.6, top_p 0.95, top_k 20, min_p 0.
• MTP: --spec-type draft-mtp, spec_n_max 6, spec_p_min 0.75.

The comparison rows:

• maci0 NVFP4 row is our own GGUF conversion of maci0's safetensors checkpoint (the same --outtype auto conversion described above), not a GGUF published by maci0.
• IQ4XS LOW (DavidAU) row is DavidAU's published GGUF from his NEO-MAX-MTP repo.
• NVFP4-LOW (this file) row is the hybrid.

| | maci0 NVFP4 (our GGUF conversion) | IQ4XS LOW (DavidAU's GGUF) | NVFP4-LOW (this file) |

|---|---|---|---|

| File size | 19.65 GB | 15.14 GB | 15.49 GB |

| Prefill (first impression) | 640.9 tok/s | 563.5 tok/s | 639.1 tok/s |

| Decode with MTP (first impression) | 15.98 tok/s | 19.78 tok/s | ~23.5 tok/s |

| MTP acceptance (first impression) | 0.880 | 0.878 | ~0.92 |

Everything ran in reasonable time and finished naturally (finish_reason: stop) in our tests, including one 17,727-token generation. Repetition-analysis flags on the repetitive-payload runs traced back to the model quoting source sentences verbatim, not to a loop. SHA256 of the file: 05520e78e68de2b2e89a6e72cf8e14627fb9e29998b599ba4863ac22399ba200.

Known caveat: rare non-deterministic repetition loop

During development, one out of five long-generation runs degenerated: the model repeated itself until hitting max_tokens exactly. The signatures were:

• finish_reason: length at exactly the token cap (20,000)
• decode rate far above the typical range seen in our other runs (~38 tok/s where ~20 to 24 was typical)
• MTP acceptance creeping toward 0.98 (repetition is trivially draftable, so it inflates acceptance)

Four subsequent runs, including the exact same 180k to 20k scenario on a fresh process each time, did not reproduce it, and all finished with finish_reason: stop. The cause is not yet fully understood; treat it as a rare, non-deterministic failure mode to watch for, not a guaranteed one. If you see these three signatures together in a long generation, this is likely what is happening.

Usage

llama.cpp / llama-server

llama-server \
  --model Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-LOW.gguf \
  --mmproj <optional: Qwen3.6-27B mmproj-BF16.gguf> \
  --ctx-size 204800 \
  --flash-attn on \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  --spec-draft-p-min 0.75 \
  --temp 0.6 --top-p 0.95 --top-k 20

• Requires a recent llama.cpp with NVFP4 (GGML type 40) CUDA kernels and sm_120 support (Blackwell).
• Requires the draft-mtp spec path (merged upstream as LLAMA_CONTEXT_TYPE_MTP).
• For vision input, point --mmproj at one of DavidAU's mmproj-BF16.gguf / mmproj-F16.gguf / mmproj-F32.gguf files from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF (see the vision note above).
• --spec-draft-p-min 0.75 matters: without it the drafter wastes steps on low-confidence tokens and the speedup shrinks.
• MTP performance is hardware-dependent: try --spec-draft-n-max values 1 through 6 and keep whatever is fastest on your system.

Memory footprint

• Weights: ~15.5 GB (fits comfortably under 24 GB)
• KV cache at 204,800 ctx with q4_0/q4_0: ~3 GB
• Total: ~18.5 GB on a 16+16 GB dual-GPU setup, leaving headroom

License

Apache-2.0, identical to every upstream artifact. The base model license governs; GGUF conversion and quantization are transformations, not new training. When redistributing, please retain attribution to Qwen (Alibaba), DavidAU, and maci0 as above.

"Qwen" is a trademark of Alibaba. Trademarks are used here only to identify upstream models; this repository is not affiliated with, sponsored by, or endorsed by Alibaba, DavidAU, or maci0.

Note on this card

This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.