GraySoft
Download guIDE →
Projects Models Compare Cloud benchmarks FAQ Download guIDE →
Model Intelligence Sheet

CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF overview

Nvidia Qwen3.6 27B NVFP4 GGUF Quantized GGUF versions of nvidia/Qwen3.6 27B NVFP4 https://huggingface.co/nvidia/Qwen3.6 27B NVFP4 . These were generated using …

ggufchain-of-thoughtnvfp4qwenqwen3.6quantizedreasoningllama.cpptext-generationenbase_model:nvidia/Qwen3.6-27B-NVFP4base_model:quantized:nvidia/Qwen3.6-27B-NVFP4license:apache-2.0endpoints_compatibleregion:usconversational

Runs locally from ~16.63 GB disk (24 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads
27,958
Likes
18
Pipeline
text-generation
Author
CodeFault

Repository Files & Downloads

3 GGUF files detected
Direct downloads for local inference
FileTypeQuantizationSizeLink
Nvidia-Qwen3.6-27B-NVFP4-A.ggufGGUFGGUF16.63 GBDownload
Nvidia-Qwen3.6-27B-NVFP4-BF16-Attn.ggufGGUFBF1626.29 GBDownload
Nvidia-Qwen3.6-27B-NVFP4-Q8.ggufGGUFQ819.99 GBDownload

Model Details

Model IDCodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF
AuthorCodeFault
Pipelinetext-generation
Licenseapache-2.0
Base modelnvidia/Qwen3.6-27B-NVFP4
Last modified2026-07-17T00:41:42.000Z

Model README

---

license: apache-2.0

language:

  • en

base_model:

  • nvidia/Qwen3.6-27B-NVFP4

pipeline_tag: text-generation

tags:

  • chain-of-thought
  • gguf
  • nvfp4
  • qwen
  • qwen3.6
  • quantized
  • reasoning
  • llama.cpp

---

Nvidia-Qwen3.6-27B-NVFP4 - GGUF

Quantized GGUF versions of nvidia/Qwen3.6-27B-NVFP4. These were generated using llama.cpp (b9859).

  • Nvidia-Qwen3.6-27B-NVFP4-A.gguf: The FFN and attention layers are NVFP4 quantized. The MTP draft layer is preserved at BF16. This was generated from BF16-Attn using a modified llama-quantize to re-quantize the BF16-upcast attention layers to NVFP4.
  • Nvidia-Qwen3.6-27B-NVFP4-Q8.gguf: The NVFP4 FFN layers are preserved, while the FP8 attention layers are converted to Q8. This was generated using convert_hf_to_gguf.py.
  • Nvidia-Qwen3.6-27B-NVFP4-BF16-Attn.gguf: The NVFP4 FFN layers are preserved, while the FP8 attention layers are upcast to BF16. This is the default conversion for BF16 because GGUF files do not support FP8. This was generated using convert_hf_to_gguf.py.

Quantizations provided

| File | Quantization | Size |

|-----------------------------------------|---------------------------|---------|

| Nvidia-Qwen3.6-27B-NVFP4-A.gguf | NVFP4 FFN and attention | 17.9 GB |

| Nvidia-Qwen3.6-27B-NVFP4-Q8.gguf | NVFP4 FFN, Q8 attention | 21.5 GB |

| Nvidia-Qwen3.6-27B-NVFP4-BF16-Attn.gguf | NVFP4 FFN, BF16 attention | 28.2 GB |

Perplexity test

I tested perplexity using llama-perplexity and Salesforce's wikitext-2-raw-v1.

| File | Ctx | PPL |

|-----------------------------------------|-------|------------------|

| Nvidia-Qwen3.6-27B-NVFP4-A.gguf | 512 | 7.6619 ± 0.05299 |

| Nvidia-Qwen3.6-27B-NVFP4-Q8.gguf | 512 | 7.4775 ± 0.05151 |

| Nvidia-Qwen3.6-27B-NVFP4-BF16-Attn.gguf | 512 | 7.4814 ± 0.05157 |

Evaluation

The following models were evaluated for a fair comparison of capability, size and speed.

| Model | Quantization | Size | Reason |

|------------------------------|-------------------|---------|-----------------------------------------|

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | 17.9 GB | Closest non-NVFP4 in size to NVFP4. |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 26 GB | Closest non-NVFP4 in size to BF16-Attn. |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>1</sup> | 25.4 GB | Alternative NVFP4 quant. |

<small>1: unsloth/Qwen3.6-27B-NVFP4 does not provide a GGUF. I used llama.cpp's conversion which passes through Unsloth's NVFP4 tensors.</small>

| | CodeFault<br>Nvidia<br>NVFP4-A | CodeFault<br>Nvidia<br>NVFP4-Q8 | CodeFault<br>Nvidia<br>NVFP4-FB16 | Unsloth<br>NVFP4<br>(2026-05-31) | Unsloth<br>UD-Q4_K_XL | Unsloth<br>UD-Q6_K_XL |

|--------------------------|--------------------------------|---------------------------------|-----------------------------------|----------------------------------|-----------------------|-----------------------|

| Coding | | | | | | |

| HumanEval | 0.8293 ± 0.0295 | 0.8659 ± 0.0267 | 0.8354 ± 0.0290 | 0.8110 ± 0.0307 | 0.8354 ± 0.0290 | 0.8537 ± 0.0277 |

| HumanEval+ | 0.7683 ± 0.0330 | 0.7683 ± 0.0330 | 0.7927 ± 0.0318 | 0.7744 ± 0.0327 | 0.7805 ± 0.0324 | 0.7805 ± 0.0324 |

| MBPP | 0.7280 ± 0.0199 | 0.7600 ± 0.0191 | 0.7540 ± 0.0193 | 0.7420 ± 0.0196 | 0.7560 ± 0.0192 | 0.7540 ± 0.0193 |

| MBPP+ | 0.8783 ± 0.0168 | 0.8862 ± 0.0164 | 0.8836 ± 0.0165 | 0.8995 ± 0.0155 | 0.8968 ± 0.0157 | 0.8836 ± 0.0165 |

| Instruction | | | | | | |

| IFEval | 0.8558 ± 0.0151 | 0.8410 ± 0.0157 | 0.8410 ± 0.0157 | 0.8447 ± 0.0156 | 0.8410 ± 0.0157 | 0.8447 ± 0.0156 |

| Knowledge | | | | | | |

| ARC-Challenge | 0.9710 ± 0.0049 | 0.9701 ± 0.0050 | 0.9710 ± 0.0049 | 0.9710 ± 0.0049 | 0.9710 ± 0.0049 | 0.9710 ± 0.0049 |

| MMLU-Pro | 0.8250 ± 0.0034 | | 0.8285 ± 0.0034 | 0.8293 ± 0.0033 | 0.8350 ± 0.0033 | 0.8319 ± 0.0033 |

| STEM & Reasoning | | | | | | |

| BIG-Bench Hard | 0.9149 ± 0.0032 | 0.9247 ± 0.0031 | 0.9158 ± 0.0032 | 0.9131 ± 0.0033 | 0.9260 ± 0.0030 | 0.9214 ± 0.0031 |

| GPQA Diamond<br>flexible | 0.7929 ± 0.0289 | 0.7879 ± 0.0291 | 0.7828 ± 0.0294 | 0.8182 ± 0.0275 | 0.8131 ± 0.0278 | 0.7778 ± 0.0296 |

| GSM8K | 0.9196 ± 0.0075 | 0.9136 ± 0.0077 | 0.9136 ± 0.0077 | 0.9098 ± 0.0079 | 0.9083 ± 0.0080 | 0.9158 ± 0.0076 |

| Hendrycks Math | 0.3438 ± 0.0065 | 0.3530 ± 0.0065 | 0.3540 ± 0.0066 | 0.3662 ± 0.0066 | 0.3658 ± 0.0066 | 0.3876 ± 0.0067 |

These evaluations were run using lm_eval. The models were run in instruct (non-thinking) mode with the following parameters in llama-server (b9775):

ctx-size = 32768
cache-type-k = q8_0
cache-type-v = q8_0
top-p = 0.8
top-k = 20
min-p = 0
presence-penalty = 1.5
spec_type = draft-mtp
spec_draft_n_max = 2
chat-template-kwargs = {"enable_thinking":false}

Benchmarks

| Model | Quant | MTP n-max | Prompt Len | Output Len | Acceptance Rate | pp/s | tg/s |

|------------------------------------|------------|-----------|------------|------------|----------------------------|--------|-------|

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-A | | 267 | 8676 | | 2225.1 | 76.6 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-A | 1 | 267 | 5958 | 0.879 | 1877.0 | 99.5 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-A | 2 | 267 | 6708 | 0.856, 0.701 | 1897.2 | 119.1 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-A | 3 | 267 | 6900 | 0.855, 0.699, 0.559 | 1950.4 | 128.7 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-A | 4 | 267 | 5635 | 0.836, 0.681, 0.567, 0.483 | 1862.6 | 137.6 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-QA | | 267 | | | | |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-QA | 1 | 267 | | | | |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-QA | 2 | 267 | | | | |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-QA | 3 | 267 | | | | |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-QA | 4 | 267 | | | | |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-BF16 | | 267 | 5363 | | 1896.4 | 53.4 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-BF16 | 1 | 267 | 5980 | 0.881 | 1643.8 | 74.6 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-BF16 | 2 | 267 | 5152 | 0.875, 0.732 | 1704.1 | 94.1 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-BF16 | 3 | 267 | 6881 | 0.876, 0.724, 0.595 | 1675.8 | 106.1 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4 | NVFP4-BF16 | 4 | 267 | 6582 | 0.859, 0.692, 0.579, 0.476 | 1686.4 | 112.3 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4 | | 267 | 7347 | | 2056.5 | 57.6 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4 | 1 | 267 | 6826 | 0.843 | 1749.9 | 74.9 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4 | 2 | 267 | 8142 | 0.851, 0.685 | 1794.7 | 90.1 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4 | 3 | 267 | 7612 | 0.837, 0.671, 0.541 | 1787 | 96.4 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4 | 4 | 267 | 7621 | 0.817, 0.620, 0.485, 0.400 | 1772.4 | 96.5 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | | 267 | 7826 | | 1535.4 | 69.8 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | 1 | 267 | 8398 | 0.879 | 1381.1 | 107.7 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | 2 | 267 | 7363 | 0.850, 0.692 | 1276.6 | 122.1 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | 3 | 267 | 8146 | 0.852, 0.681, 0.552 | 1286.9 | 123 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q4_K_XL | 4 | 267 | 8269 | 0.830, 0.647, 0.529, 0.439 | 923.2 | 120.8 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | | 267 | 7180 | | 1257.3 | 53.7 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 1 | 267 | 5868 | 0.876 | 1249.1 | 84.8 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 2 | 267 | 6104 | 0.864, 0.701 | 1232.8 | 102.2 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 3 | 267 | 5000 | 0.847, 0.688, 0.563 | 1228.7 | 109 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 4 | 267 | 7060 | 0.852, 0.703, 0.577, 0.474 | 1052.5 | 116.8 |

These benchmarks were run on an RTX 5090 (limited to 480 W) using llama-cli (b9775) with the CUDA driver and a prompt to generate an Ansible playbook.

Serving with llama.cpp

It has a max context size of 262,114.

This can be served using:

llama-server \
-hf CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF \
-hff Nvidia-Qwen3.6-27B-NVFP4-A.gguf \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--repeat-penalty 1.1 \
--spec-type draft-mtp \
--spec-draft-n-max 3

Run CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models