---

license: apache-2.0

base_model:

- Lathly/Qwen3.8-27B-Samantha

- Qwen/Qwen3.8-27B

datasets:

- Lathly/samantha-nvfp4-calibration

language:

- en

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- nvfp4

- qwen3_5

- mtp

- nextn

- samantha

- conversational

- uncensored

- modelopt

---

Qwen3.8-27B-Samantha - NVFP4 GGUF with BF16 MTP

Native NVFP4 GGUF export of Lathly/Qwen3.8-27B-Samantha, a full merged Samantha 1.1 persona fine-tune of Qwen/Qwen3.8-27B.

This repository contains a llama.cpp-ready text-generation GGUF. The source model is a native vision-language model; this GGUF export is text-only and does not include a multimodal projector.

Files

| File | Size | Description |

|---|---:|---|

| Qwen3.8-27B-Samantha-NVFP4-MTP-BF16.gguf | 19.69 GB | Native NVFP4 main model with the one-layer MTP/NextN head retained in BF16 |

SHA256:

71744136f18bc053e720e021d29bd3411f2d949a52e4f5d0485e54ca976c4a85

How this GGUF was made

This is a post-training quantization export only: no additional fine-tuning or merge was performed for this repository.

1. Starting checkpoint: the merged BF16 safetensors from Lathly/Qwen3.8-27B-Samantha.
2. PTQ: NVIDIA Model Optimizer, using its native nvfp4 configuration.
3. Calibration: Lathly/samantha-nvfp4-calibration, 512 samples, sequence length 2048, batch size 1.
4. MTP: the Qwen3.5-family MTP/NextN head was intentionally excluded from NVFP4 quantization and retained in BF16 for speculative-decoding accuracy.
5. GGUF conversion: NVFP4 tensors were repacked into GGML's native NVFP4 layout, while non-quantized tensors remain BF16/F32 as appropriate.

The completed file contains 400 NVFP4 tensors, 106 BF16 tensors, and 65 total transformer blocks. GGUF metadata records qwen35.nextn_predict_layers = 1.

Running with llama.cpp

Use a recent CUDA build of llama.cpp with native NVFP4 and Qwen3.5/NextN (MTP) support:

llama-server \
  -m Qwen3.8-27B-Samantha-NVFP4-MTP-BF16.gguf \
  -ngl 999 \
  --host 0.0.0.0 --port 8080

NVFP4 acceleration requires NVIDIA Blackwell-class hardware. Other backends may not support this quantization type or may fall back to a slower path.

Validation

The finished GGUF was loaded with a fresh CUDA build of llama.cpp on an NVIDIA RTX PRO 6000 Blackwell GPU. llama.cpp detected ftype: NVFP4, loaded all layers to GPU, and completed a short Norwegian generation smoke test at approximately 75 tokens/s. This is a functional smoke test, not a formal quality or benchmark evaluation.

Original fine-tune

The source is a full merged Samantha persona fine-tune: the LoRA adapter from Lathly/Qwen3.8-27B-Samantha_Uncensored_1.1_LoRA was trained with Unsloth (QLoRA) and merged into Qwen3.8-27B. For training hyperparameters, source usage, and the original model description, see Lathly/Qwen3.8-27B-Samantha.

License, attribution, and disclaimer

Released under Apache-2.0, inheriting from the source model and its parents:

• Qwen/Qwen3.8-27B — Apache-2.0, Alibaba Cloud / Qwen Team
• Lathly/Qwen3.8-27B-Samantha — merged Samantha fine-tune
• digitalpipelines/samantha-1.1-uncensored — source training dataset, Apache-2.0

This is an "uncensored" persona fine-tune intended for open conversational use. It may generate objectionable content. Use responsibly and in accordance with applicable laws and upstream licenses.