---

language:

• en
• th

tags:

• qwen3.5
• qwen3_5
• gdn
• linear-attention
• hybrid
• sft
• lora
• fine-tuned
• sme
• tool-calling
• agent
• business
• multimodal
• gguf
• mtp

license: apache-2.0

base_model: hotdogs/Qwen3.8-27B-abliterated-sme-preview

pipeline_tag: text-generation

---

Qwen3.8-27B-Abliterated-SME-Preview-MTP-GGUF

GGUF quantization of hotdogs/Qwen3.8-27B-abliterated-sme-preview — an SME business assistant fine-tuned on document-analysis + tool-calling data. Converted for llama.cpp with the MTP (Multi-Token Prediction) head preserved.

• Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview
• Method: Quantization to GGUF, MTP head intact (mtp_num_hidden_layers=1)
• Format: .gguf (f16 source, quantized to Q4_K_M / IQ3_M etc.)
• Multimodal: vision wrapper present — use as text-only via apply_chat_template

---

⚡ Quick Results (inherited from base)

| Benchmark | Base | SME-Preview | Δ |

|---|---:|---:|---:|

| ARC-Challenge acc | 0.5667 | 0.5700 | +0.003 |

| MMLU | 0.8477 | 0.8449 | −0.003 |

| GSM8K (5-shot) | 0.6000 | 0.8100 | +0.210 |

GGUF conversion preserves these numbers within ±0.01.

---

🚀 Usage

# Download a quant
huggingface-cli download hotdogs/Qwen3.8-27B-abliterated-sme-preview-mtp-GGUF \
  --include "*.gguf" --local-dir ./model

# Run
./llama-cli -m ./model/model-Q4_K_M.gguf \
  -c 8192 -ngl 99 \
  -p "Which rows in expenses_2025.csv have department = Sales?"

> Requires a llama.cpp build with qwen35 architecture support (GDN + linear-attention).

> Do not pass --no-mtp — the model has MTP tensors.

⚠️ Notes

• MTP preserved: block_count=64+1, do not strip MTP tensors
• BF16→f16: minor precision loss acceptable for inference
• Multimodal: vision wrapper present; for text-only use apply_chat_template

📜 License

Apache-2.0. Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview