GraySoft
Download guIDE →
Projects Models Compare Cloud benchmarks FAQ Download guIDE →
Model Intelligence Sheet

CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF overview

Unsloth Qwen3.6 27B NVFP4 GGUF Quantized GGUF versions of unsloth/Qwen3.6 27B NVFP4 https://huggingface.co/unsloth/Qwen3.6 27B NVFP4 from 2026 07 10. These wer…

ggufchain-of-thoughtnvfp4qwenqwen3.6quantizedreasoningllama.cpptext-generationenbase_model:unsloth/Qwen3.6-27B-NVFP4base_model:quantized:unsloth/Qwen3.6-27B-NVFP4license:apache-2.0endpoints_compatibleregion:usconversational

Runs locally from ~18.71 GB disk (24 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads
3,485
Likes
4
Pipeline
text-generation
Author
CodeFault

Repository Files & Downloads

2 GGUF files detected
Direct downloads for local inference
FileTypeQuantizationSizeLink
Unsloth-Qwen3.6-27B-NVFP4-A.ggufGGUFGGUF18.71 GBDownload
Unsloth-Qwen3.6-27B-NVFP4-Q8.ggufGGUFQ821.58 GBDownload

Model Details

Model IDCodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF
AuthorCodeFault
Pipelinetext-generation
Licenseapache-2.0
Base modelunsloth/Qwen3.6-27B-NVFP4
Last modified2026-07-15T11:32:57.000Z

Model README

---

license: apache-2.0

language:

  • en

base_model:

  • unsloth/Qwen3.6-27B-NVFP4

pipeline_tag: text-generation

tags:

  • chain-of-thought
  • gguf
  • nvfp4
  • qwen
  • qwen3.6
  • quantized
  • reasoning
  • llama.cpp

---

Unsloth-Qwen3.6-27B-NVFP4 - GGUF

Quantized GGUF versions of unsloth/Qwen3.6-27B-NVFP4 from 2026-07-10. These were generated using llama.cpp (b9859).

  • Unsloth-Qwen3.6-27B-NVFP4-A.gguf: The NVFP4 group 1 layers are preserved. The FP8 group 0 linear-attention (attn_qkv, ssm_out) and tail-MLP (layers 56–63) are converted to NVFP4, while the remaining layers are kept at Q8. The MTP draft layer is preserved at BF16. This was generated from NVFP4-Q8 using a modified llama-quantize to allow re-quantizing Q8-quantized layers to NVFP4.
  • Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf: The NVFP4 group 1 layers are preserved, while the FP8 group 0 layers are converted to Q8. This was generated using a modified convert_hf_to_gguf.py to support Unsloth's differing groups.
  • Unsloth-Qwen3.6-27B-NVFP4-BF16.gguf: The NVFP4 group 1 layers are preserved, while the FP8 group 0 layers are upcast to BF16. This version is not published because it is redundant with NVFP4-Q8.

Quantizations provided

| File | Quantization | Size |

|-------------------------------------|---------------------------------------|---------|

| Unsloth-Qwen3.6-27B-NVFP4-A.gguf | Mixed Q8/NVFP4 group 0, NVFP4 group 1 | 20.1 GB |

| Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf | Q8 group 0, NVFP4 group 1 | 23.2 GB |

Perplexity test

I tested perplexity using llama-perplexity and Salesforce's wikitext-2-raw-v1.

| File | Ctx | PPL |

|-------------------------------------|-------|-------------------|

| Unsloth-Qwen3.6-27B-NVFP4-A.gguf | 512 | 7.5789 ± 0.05260 |

| Unsloth-Qwen3.6-27B-NVFP4-Q8.gguf | 512 | 7.3792 ± 0.05057 |

| Unsloth-Qwen3.6-27B-NVFP4-BF16.gguf | 512 | 7.3702 ± 0.05052 |

Evaluation

The following models were evaluated for a fair comparison of capability, size and speed.

| Model | Quantization | Size | Reason |

|---------------------------------------|---------------------|---------|--------------------------------------------|

| CodeFault/Nvidia-Qwen3.6-27B-MTP-GGUF | NVFP4-A<sup>1</sup> | 17.9 GB | Alternative NVFP4 quant. |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | 20.4 GB | Closest non-NVFP4 in size to NVFP4-A. |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 26 GB | Closest non-NVFP4 in size to NVFP4-Q8. |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>2</sup> | 25.4 GB | Previous NVFP4 checkpoint from 2026-05-31. |

<small>1: My quantization of nvidia/Qwen3.6-27B-MTP.</small>

<small>2: unsloth/Qwen3.6-27B-NVFP4 does not provide a GGUF. I used llama.cpp's conversion which passes through Unsloth's NVFP4 tensors. This is Unsloth's previous version from 2026-05-31.</small>

| | CodeFault<br>Unsloth<br>NVFP4-A | CodeFault<br>Unsloth<br>NVFP4-Q8 | CodeFault<br>Nvidia<br>NVFP4-A | Unsloth<br>NVFP4<br>(2026-05-31) | Unsloth<br>UD-Q5_K_XL | Unsloth<br>UD-Q6_K_XL |

|--------------------------|---------------------------------|----------------------------------|--------------------------------|----------------------------------|-----------------------|-----------------------|

| Coding | | | | | | |

| HumanEval | 0.8354 ± 0.0290 | 0.8293 ± 0.0295 | 0.8293 ± 0.0295 | 0.8110 ± 0.0307 | 0.8598 ± 0.0272 | 0.8537 ± 0.0277 |

| HumanEval+ | 0.7622 ± 0.0333 | 0.7622 ± 0.0333 | 0.7683 ± 0.0330 | 0.7744 ± 0.0327 | 0.7866 ± 0.0321 | 0.7805 ± 0.0324 |

| MBPP | 0.7540 ± 0.0193 | 0.7720 ± 0.0188 | 0.7280 ± 0.0199 | 0.7420 ± 0.0196 | 0.7540 ± 0.0193 | 0.7540 ± 0.0193 |

| MBPP+ | 0.8836 ± 0.0165 | 0.8915 ± 0.0160 | 0.8783 ± 0.0168 | 0.8995 ± 0.0155 | 0.8968 ± 0.0157 | 0.8836 ± 0.0165 |

| Instruction | | | | | | |

| IFEval | 0.8540 ± 0.0152 | 0.8577 ± 0.0150 | 0.8558 ± 0.0151 | 0.8447 ± 0.0156 | 0.8558 ± 0.0151 | 0.8447 ± 0.0156 |

| Knowledge | | | | | | |

| ARC-Challenge | 0.9735 ± 0.0047 | 0.9718 ± 0.0048 | 0.9710 ± 0.0049 | 0.9710 ± 0.0049 | 0.9684 ± 0.0051 | 0.9710 ± 0.0049 |

| MMLU-Pro | 0.8291 ± 0.0034 | 0.8288 ± 0.0034 | 0.8250 ± 0.0034 | 0.8293 ± 0.0033 | | 0.8319 ± 0.0033 |

| STEM & Reasoning | | | | | | |

| BIG-Bench Hard | 0.9201 ± 0.0032 | 0.9174 ± 0.0032 | 0.9149 ± 0.0032 | 0.9131 ± 0.0033 | 0.9220 ± 0.0031 | 0.9214 ± 0.0031 |

| GPQA Diamond<br>flexible | 0.7929 ± 0.0289 | 0.7929 ± 0.0289 | 0.7929 ± 0.0289 | 0.8182 ± 0.0275 | 0.8131 ± 0.0278 | 0.7778 ± 0.0296 |

| GSM8K | 0.9151 ± 0.0077 | 0.9136 ± 0.0077 | 0.9196 ± 0.0075 | 0.9098 ± 0.0079 | 0.9128 ± 0.0078 | 0.9158 ± 0.0076 |

| Hendrycks Math | 0.3698 ± 0.0066 | 0.3624 ± 0.0066 | 0.3438 ± 0.0065 | 0.3662 ± 0.0066 | 0.3778 ± 0.0067 | 0.3876 ± 0.0067 |

NOTICE: These evals are actively running.

These evaluations were run using lm_eval. The models were run in instruct (non-thinking) mode with the following parameters in llama-server (b9775):

ctx-size = 32768
cache-type-k = q8_0
cache-type-v = q8_0
top-p = 0.8
top-k = 20
min-p = 0
presence-penalty = 1.5
spec_type = draft-mtp
spec_draft_n_max = 2
chat-template-kwargs = {"enable_thinking":false}

Benchmarks

| Model | Quant | MTP n-max | Prompt Len | Output Len | Acceptance Rate | pp/s | tg/s |

|------------------------------------------|--------------------|-----------|------------|------------|-----------------------------|---------|-------|

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | | 267 | 6654 | | 2205.4 | 70.6 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 1 | 267 | 7240 | 0.880 | 1880.7 | 91.8 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 2 | 267 | 5842 | 0.856, 0.712 | 1853.9 | 109.8 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 3 | 267 | 5711 | 0.852, 0.712, 0.596 | 1782.3 | 119.7 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 4 | 267 | 9709 | 0.841, 0.688, 0.556, 0.462 | 1738.4 | 122.7 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-Q8 | | 267 | 6544 | | 2077.4 | 63.0 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-Q8 | 1 | 267 | 6596 | 0.876 | 1742.8 | 84.3 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-Q8 | 2 | 267 | 7088 | 0.868, 0.724 | 1766.4 | 102.8 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-Q8 | 3 | 267 | 5903 | 0.831, 0.675, 0.543 | 1785.1 | 108.8 |

| CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF | NVFP4-Q8 | 4 | 267 | 5785 | 0.837, 0.684, 0.563, 0.471 | 1771.2 | 116.3 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | | 267 | 8676 | | 2225.1 | 76.6 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 1 | 267 | 5958 | 0.879 | 1877.0 | 99.5 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 2 | 267 | 6708 | 0.856, 0.701 | 1897.2 | 119.1 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 3 | 267 | 6900 | 0.855, 0.699, 0.559 | 1950.4 | 128.7 |

| CodeFault/Nvidia-Qwen3.6-27B-NVFP4-GGUF | NVFP4-A | 4 | 267 | 5635 | 0.836, 0.681, 0.567, 0.483 | 1862.6 | 137.6 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>3</sup> | | 267 | 7347 | | 2056.5 | 57.6 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>3</sup> | 1 | 267 | 6826 | 0.843 | 1749.9 | 74.9 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>3</sup> | 2 | 267 | 8142 | 0.851, 0.685 | 1794.7 | 90.1 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>3</sup> | 3 | 267 | 7612 | 0.837, 0.671, 0.541 | 1787 | 96.4 |

| unsloth/Qwen3.6-27B-NVFP4 | NVFP4<sup>3</sup> | 4 | 267 | 7621 | 0.817, 0.620, 0.485, 0.400 | 1772.4 | 96.5 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | | 267 | 7866 | | 1522.0 | 64.0 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | 1 | 267 | 6977 | 0.866 | 1296.4 | 99.0 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | 2 | 267 | 7418 | 0.850, 0.686 | 1383.8 | 115.0 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | 3 | 267 | 9204 | 0.850, 0.684, 0.565 | 1305.6 | 118.9 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q5_K_XL | 4 | 267 | 7600 | 0.855, 0.677, 0.565, 0.472 | 1294.0 | 123.5 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | | 267 | 7180 | | 1257.3 | 53.7 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 1 | 267 | 5868 | 0.876 | 1249.1 | 84.8 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 2 | 267 | 6104 | 0.864, 0.701 | 1232.8 | 102.2 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 3 | 267 | 5000 | 0.847, 0.688, 0.563 | 1228.7 | 109 |

| unsloth/Qwen3.6-27B-MTP-GGUF | UD-Q6_K_XL | 4 | 267 | 7060 | 0.852, 0.703, 0.577, 0.474 | 1052.5 | 116.8 |

<small>3: This is Unsloth's previous version from 2026-05-31.</small>

These benchmarks were run on an RTX 5090 (limited to 480 W) using llama-cli (b9775) with the CUDA driver and a prompt to generate an Ansible playbook.

Serving with llama.cpp

It has a max context size of 262,114.

This can be served using:

llama-server \
-hf CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF \
-hff Unsloth-Qwen3.6-27B-NVFP4-A.gguf \
--temp 0.6 \
--top-p 0.95 \
--top-k 20 \
--repeat-penalty 1.1 \
--spec-type draft-mtp \
--spec-draft-n-max 3

Run CodeFault/Unsloth-Qwen3.6-27B-NVFP4-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models