---

license: other

license_name: swift-open-license-1.0

license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-27b

library_name: gguf

pipeline_tag: image-text-to-text

base_model:

• ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP

base_model_relation: quantized

tags:

• gguf
• llama.cpp
• qwen3_8
• abliterated
• uncensored
• unsloth
• imatrix
• dynamic-quant
• mtp

---

Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF

GGUF quants of ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP,

an abliterated Swift-Qwen3.8-27B (UkisAI's

reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and

SGLang: NVFP4.

• Uncensored: 15/100 refusals against Swift's 98/100, KL divergence 0.0634.
• Unsloth Dynamic 3.0 layout: every tensor has the type Unsloth chose for its quant of the same

size, with Unsloth's importance matrix.

• MTP head included in every main GGUF, for self-speculative decoding in llama.cpp.
• Vision: image and video input through the included projector.
• Measured: KL divergence against the BF16 (see Quality).

Quants

| Quant | Size | Use |

| :-- | :-- | :-- |

| UD-Q2_K_XL | 9.2 GiB | Fits a 12 GB GPU. |

| UD-IQ3_XXS | 10.2 GiB | Also fits a 12 GB GPU. Prefer it to UD-Q2_K_XL if it fits: clearly closer to the BF16. |

| UD-IQ3_S | 11.2 GiB | Fits a 16 GB GPU with more context room than UD-Q3_K_XL. |

| UD-Q3_K_XL | 12.2 GiB | Fits a 16 GB GPU. |

| UD-IQ4_XS | 13.3 GiB | Smallest 4-bit. Fits a 16 GB GPU, with less context room than UD-Q3_K_XL. |

| UD-Q4_K_S | 14.3 GiB | Smaller 4-bit. Fits a 20 GB GPU, or a 16 GB GPU with short context. |

| UD-Q4_K_XL | 16.4 GiB | Fits a 24 GB GPU with room for long context. |

| UD-Q5_K_S | 17.4 GiB | Fits a 24 GB GPU with more context room than UD-Q5_K_M. |

| UD-Q5_K_M | 18.4 GiB | Fits a 24 GB GPU. |

| UD-Q6_K_XL | 23.6 GiB | Best fit for a 32 GB GPU. |

| UD-Q8_K_XL | 29.3 GiB | Near-lossless. Fits a 48 GB GPU; on a 32 GB GPU it needs partial CPU offload. |

| BF16 | 50.9 GiB | Unquantized. The source of the quants, and a reference for measuring them or making other sizes. |

Each quant is one file, Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-<quant>.gguf; the BF16 is

Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf.

Other files:

| File | Size | Use |

| :-- | :-- | :-- |

| mmproj-BF16.gguf | 0.9 GiB | Vision projector, needed only for image and video input. |

| tensor_types.tsv | 42 KB | Per-tensor quantization types for each size (see Quantization). |

| kld_results.csv | 5 KB | Every statistic behind the Quality table. |

Quality

Each quant compared with the BF16 it was made from, token by token: wikitext-2 test set,

36 × 8192 tokens, f16 KV cache, llama.cpp 94659b076. Lower KLD and higher top-1 are better.

!Mean KL divergence of each quant against file size

| Size | GiB | Mean KLD | 99 % KLD | Same top-1 |

| :-- | --: | --: | --: | --: |

| UD-Q2_K_XL | 9.2 | 0.1796 | 1.99 | 85.2 % |

| UD-IQ3_XXS | 10.2 | 0.1255 | 1.20 | 87.9 % |

| UD-IQ3_S | 11.2 | — | — | — |

| UD-Q3_K_XL | 12.2 | 0.0726 | 0.58 | 91.8 % |

| UD-IQ4_XS | 13.3 | — | — | — |

| UD-Q4_K_S | 14.3 | 0.0521 | 0.45 | 94.1 % |

| UD-Q4_K_XL | 16.4 | 0.0324 | 0.16 | 95.8 % |

| UD-Q5_K_S | 17.4 | 0.0272 | 0.11 | 96.4 % |

| UD-Q5_K_M | 18.4 | 0.0207 | 0.080 | 96.9 % |

| UD-Q6_K_XL | 23.6 | 0.0116 | 0.028 | 98.2 % |

| UD-Q8_K_XL | 29.3 | 0.0060 | 0.012 | 98.8 % |

• UD-IQ3_S and UD-IQ4_XS were added after this measurement run, so they have no numbers yet

and are absent from the chart for the same reason. Both come with the next batch.

• Mean KLD is the average divergence of the quant's next-token distribution from the BF16's.

99 % KLD is the level only the worst 1 % of tokens exceed. Same top-1 is how often both

pick the same most likely token.

• Quality follows file size with no exceptions. The knee is at UD-Q5_K_M: above it a GiB buys

little, below it each GiB saved costs about twice as much.

• The typical token is barely touched (median KLD is 0.0002 at UD-Q8_K_XL and 0.003 at

UD-Q4_K_XL); the mean is carried by a small share of tokens that change a lot. From UD-Q3_K_XL

down the error also turns one-sided, taking 0.4–1.3 % of probability from the BF16's token on

average.

This measures quantization only, against this repository's own BF16. It is a different quantity

from the 0.0634 under The model, which is the abliteration measured against Swift.

Prose only: code and chat-formatted text were not measured.

Every statistic is in kld_results.csv. It also includes perplexity, which does not rank these

files: several quants score below the BF16.

Usage (llama.cpp)

# text only
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768

# with vision (image and video input)
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --mmproj mmproj-BF16.gguf

# self-speculative decoding with the built-in MTP head
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --spec-type draft-mtp

--spec-type draft-mtp needs a llama.cpp build with MTP support for qwen35. The MTP head loads from

the main GGUF; there is no separate draft file.

Sampling, as for Swift and Qwen: temperature 1.0, top_p 0.95, top_k 20, min_p 0. The model thinks

before answering by default.

The model

The source applies the refusal direction of

orcarouter/Qwen3.8-27B-Uncensored

(Arditi et al. 2024, one direction) to Swift's weights: 131 tensors edited (attention outputs,

mlp.down_proj, embed_tokens, MTP layer included), everything else Swift's. Method, direction and

scripts are on the source model's card.

| Model | Refusals | KL divergence |

| :-- | :-: | :-: |

| BF16 source (against Swift) | 15/100 | 0.0634 |

| Swift-Qwen3.8-27B | 98/100 | 0 |

| Reference: orcarouter/Qwen3.8-27B-Uncensored (against Qwen3.8-27B) | 17/100 | 0.0621 |

Measured with Heretic on the BF16 weights (100 prompts from

mlabonne/harmful_behaviors, keyword refusal detector; first-token KL on mlabonne/harmless_alpaca;

thinking skipped). Not re-measured on the quants.

Quantization

• Conversion: the source model's BF16 safetensors (all 1199 tensors, MTP included) were converted

with llama.cpp's convert_hf_to_gguf.py, once for the language model and once with --mmproj for

the vision projector.

• Quantization: llama-quantize with Unsloth's imatrix_unsloth.gguf from

unsloth/Qwen3.8-27B-GGUF, plus a --tensor-type-file

that sets every tensor's type to the one in Unsloth's GGUF of the same size.

| Size | Tensor types (F32 norms etc. excluded) |

| :-- | :-- |

| UD-Q2_K_XL | 112×IQ3_XXS, 98×Q8_0, 67×IQ2_S, 57×IQ3_S, 48×IQ2_XXS, 34×IQ2_XS, 21×Q4_K, 20×IQ1_S, 19×IQ4_XS, 16×Q2_K, 6×Q6_K, 5×Q3_K, 2×Q5_K, 1×IQ1_M |

| UD-IQ3_XXS | 120×IQ3_XXS, 106×IQ3_S, 98×Q8_0, 45×IQ4_XS, 35×IQ2_S, 26×Q4_K, 24×IQ2_XXS, 12×IQ2_XS, 10×Q2_K, 8×IQ1_S, 7×Q5_K, 6×Q3_K, 6×Q6_K, 3×IQ1_M |

| UD-IQ3_S | 127×IQ3_S, 98×Q8_0, 88×IQ4_XS, 77×IQ3_XXS, 23×Q4_K, 21×IQ2_S, 16×Q5_K, 16×Q3_K, 12×IQ2_XS, 12×IQ2_XXS, 7×Q6_K, 6×Q2_K, 2×IQ1_S, 1×IQ4_NL |

| UD-Q3_K_XL | 156×IQ4_XS, 111×IQ3_S, 98×Q8_0, 36×Q4_K, 34×IQ3_XXS, 26×Q5_K, 15×IQ2_S, 12×Q3_K, 7×Q6_K, 4×IQ2_XS, 3×Q2_K, 2×IQ4_NL, 2×IQ2_XXS |

| UD-IQ4_XS | 211×IQ4_XS, 98×Q8_0, 55×Q4_K, 51×Q5_K, 46×IQ3_S, 17×IQ3_XXS, 9×Q3_K, 8×Q6_K, 7×IQ2_S, 2×IQ2_XS, 1×IQ4_NL, 1×Q2_K |

| UD-Q4_K_S | 172×IQ4_XS, 99×Q8_0, 95×Q4_K, 80×Q5_K, 18×Q6_K, 15×IQ3_S, 13×Q3_K, 7×IQ4_NL, 5×IQ3_XXS, 1×IQ2_S, 1×IQ2_XS |

| UD-Q4_K_XL | 191×Q5_K, 110×Q8_0, 70×IQ4_XS, 69×Q4_K, 56×Q6_K, 6×IQ4_NL, 3×Q3_K, 1×IQ3_S |

| UD-Q5_K_S | 232×Q5_K, 117×Q8_0, 84×Q6_K, 35×Q4_K, 32×IQ4_XS, 4×IQ4_NL, 2×Q3_K |

| UD-Q5_K_M | 189×Q5_K, 160×Q6_K, 124×Q8_0, 19×IQ4_XS, 12×Q4_K, 2×IQ4_NL |

| UD-Q6_K_XL | 310×Q8_0, 168×Q6_K, 27×Q5_K, 1×Q4_K |

| UD-Q8_K_XL | 453×Q8_0, 53×BF16 |

The imatrix was calibrated on the original Qwen3.8-27B; Swift's low-rank fine-tune and the rank-one

abliteration change the weights only slightly.

Checks

• Layout: each quant's 866 tensors have exactly the types and shapes of Unsloth's file of the same

size.

• MTP: every main GGUF has 65 blocks, with the MTP layer's 15 tensors in blk.64

(nextn_predict_layers = 1). llama.cpp's --spec-type draft-mtp loads it and drafts with it.

• Tokenizer and chat template:

- The vocabulary, merges and special tokens are the same as in Unsloth's Qwen3.8-27B GGUF.

- add_bos_token = false and the padding token <|endoftext|> come from Swift's own tokenizer

config.

- The embedded chat template is byte-identical to Swift's, which is Qwen3.8-27B's.

• Vision projector: all 334 tensors are identical to Qwen3.8-27B's vision projector. Neither

Swift's fine-tune nor the abliteration touches the vision tower.

• Load test: every quant loads and runs on a 32 GB GPU: fully offloaded up to UD-Q6_K_XL,

with 56 of 65 layers offloaded for UD-Q8_K_XL. Each produced sane logits on a short perplexity

run before upload; the measurement that ranks them is under Quality.

Reproduce

# 1. convert (Swift-Qwen3.8-27B-Uncensored-MTP/ is a download of the source repository)
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 --mmproj \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile mmproj-BF16.gguf

# 2. per-size --tensor-type-file from tensor_types.tsv (here UD-Q5_K_M), one anchored regex per tensor
awk -F'\t' 'NR==1 { for (i=2; i<=NF; i++) if ($i=="UD-Q5_K_M") c=i; next }
            { n=$1; gsub(/\./,"\\\\.",n); print "^" n "$=" $c }' \
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt

# 3. quantize. The last argument is the fallback type for tensors the file doesn't list:
#    UD-Q2_K_XL Q2_K, UD-IQ3_XXS IQ3_XXS, UD-IQ3_S IQ3_S, UD-Q3_K_XL Q3_K_L,
#    UD-IQ4_XS IQ4_XS, UD-Q4_K_S Q4_K_S, UD-Q4_K_XL Q4_K_M, UD-Q5_K_S Q5_K_S,
#    UD-Q5_K_M Q5_K_M, UD-Q6_K_XL Q6_K, UD-Q8_K_XL Q8_0
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
  Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf \
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q5_K_M.gguf Q5_K_M

tensor_types.tsv holds the type of every non-F32 tensor in each of Unsloth's sizes, one row per

tensor and one column per size. The regexes must be anchored (^…$). llama-quantize matches them

unanchored, so a bare output\.weight line would also match every attn_output.weight.

Not evaluated

General benchmarks, KL divergence of the quants on code or chat-formatted text, refusal behaviour

in thinking mode and on the quants, whether Swift's shorter reasoning traces survive, and MTP

acceptance against Swift.

License

Derivative of Swift-Qwen3.8-27B, under the Swift Open License v1.0

(Swift model card): free for individuals and

organizations up to US$1,000,000 annual recurring revenue, above that commercial use needs a Swift

Enterprise License from UkisAI. Qwen3.8-27B and orcarouter/Qwen3.8-27B-Uncensored are Apache 2.0.

Intended use

The model answers requests the original declines. You are responsible for how you use it and for

complying with applicable law and the license.

Credits

• Qwen for Qwen3.8-27B.
• UkisAI for Swift-Qwen3.8-27B.
• OrcaRouter for Qwen3.8-27B-Uncensored and its refusal direction.
• Unsloth for the Dynamic 3.0 quantization layout and imatrix.
• Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
• Heretic, used for evaluation.