---

license: apache-2.0

base_model:

• unsloth/Qwen3.6-35B-A3B-NVFP4-Fast
• unsloth/Qwen3.6-35B-A3B-NVFP4
• nvidia/Qwen3.6-35B-A3B-NVFP4

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• nvfp4
• fp8
• mtp
• speculative-decoding
• qwen3.6
• qwen35moe
• blackwell
• rtx-5090
• windows
• unsloth
• fast

---

<div style="font-family: Inter, ui-sans-serif, system-ui, sans-serif; border: 1px solid #3a2a1f; background: #0f0f13; border-radius: 16px; overflow: hidden; margin: 0 0 28px 0;">

<div style="padding: 14px 18px; background: linear-gradient(90deg, #1a120d 0%, #2b1708 100%); border-bottom: 1px solid rgba(255,122,26,0.35); color: #ffd7ad; font-weight: 900;">RTX 5090 Windows native GGUF comparison - updated July 19, 2026</div>

<img src="https://huggingface.co/neko-legends/Qwen3.6-35B-A3B-NVFP4-MTP-GGUF/resolve/main/rtx-5090-qwen35-moe-vs-qwopus.png" alt="RTX 5090 native llama.cpp long-context model comparison including Unsloth NVFP4 Fast" style="display:block; width:100%; border:0;" />

</div>

Qwen3.6 35B A3B NVFP4 MTP GGUFs

Native llama.cpp conversions of NVIDIA and Unsloth Qwen3.6 35B A3B NVFP4

checkpoints with their bundled MTP blocks preserved. The newest artifact comes

from unsloth/Qwen3.6-35B-A3B-NVFP4-Fast.

Files

| File | Source | Layout | Size |

| --- | --- | --- | ---: |

| qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf | Unsloth NVFP4 Fast | All expert layers native NVFP4, bundled MTP | 21.33 GiB |

| qwen3.6-35b-a3b-unsloth-nvfp4-mtp-gguf.gguf | Unsloth NVFP4 | Native NVFP4 expert FFNs, source FP8 tensors stored as Q8_0, bundled MTP | 24.34 GiB |

| qwen3.6-35b-a3b-nvfp4-mtp.gguf | NVIDIA NVFP4 | Native NVFP4 with bundled MTP | 22.21 GiB |

Fast GGUF SHA256:

ABE31A8082E44E5C03D0F08595715584AEADEA8C2A76C6F9F066A4ECBA64E78A.

The Fast source stores every expert layer as NVFP4. The earlier Unsloth source

uses a mixed compressed-tensors layout; its conversion keeps packed NVFP4

expert tensors native and stores source FP8 weights as Q8_0. Both expose their

MTP layer to recent llama.cpp builds through draft-mtp.

RTX 5090 Result

Windows 11, RTX 5090, 200k context, q4_0 target/draft KV, `draft-mtp

n=2`, no-thinking, identical BookContext fixtures, and one measured 1,024-token

completion per point. These are llama.cpp decode-only rates after the separately

reported prompt prefill.

| GGUF | Runtime | 10k decode | 10k prefill | 200k decode | 200k prefill | VRAM after |

| --- | --- | ---: | ---: | ---: | ---: | ---: |

| Unsloth Fast | b10068 | 135.23 tok/s | 2.16s | 84.08 tok/s | 52.84s | 25,138 MiB |

| Unsloth mixed | b9851 | 151.16 tok/s | 2.37s | 93.20 tok/s | 50.41s | 26.7 GiB |

| NVIDIA source | b9851 | 150.94 tok/s | 2.39s | 90.24 tok/s | 51.35s | 24.6 GiB |

The Fast artifact is about 3.0 GiB smaller than the earlier mixed Unsloth

conversion and leaves roughly 7 GiB free with the tested 200k server. However,

Unsloth's upstream Fast claim is based on vLLM concurrency throughput. That

advantage did not transfer to this native Windows llama.cpp, single-request

test: the earlier mixed conversion remains faster at both prompt lengths.

Benchmark CSV: benchmarks/qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf-benchmark-20260719.csv.

Quick Start - Fast

llama-server \
  --model qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf \
  --alias qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf \
  --host 0.0.0.0 \
  --port 39202 \
  --device CUDA0 \
  --gpu-layers all \
  --gpu-layers-draft all \
  --ctx-size 200000 \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --cache-type-k-draft q4_0 \
  --cache-type-v-draft q4_0 \
  --flash-attn on \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning off \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.0

Companion Scripts

Portable Windows source download, conversion, official llama.cpp b10068 runtime

installation, Hermes setup, launcher, and benchmark scripts are in

neko-legends/nvidia-local-llm-profiles.

Models are stored relative to the checkout parent rather than in a machine-specific

drive path.

Apache 2.0, inherited from the source releases.