Model Intelligence Sheet
jakeatx/Qwen3.8-27B-Ternary-PQ2_0-MTP-GGUF overview
Qwen3.8 27B Ternary PQ2 0 2.13 bpw with Native MTP 2 bit / 2.13 bpw ternary quantization of Qwen3.8 27B Prism ML Ternary Bonsai 2 architecture with native 1 la…
Runs locally from ~6.99 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
1 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Qwen3.8-27B-Ternary-PQ2_0-MTP.gguf | GGUF | GGUF | 6.99 GB | Download |
Model Details
Model README
---
license: apache-2.0
tags:
- gguf
- qwen3_5
- ternary
- mtp
- speculative-decoding
- ampere
- sm86
---
Qwen3.8-27B Ternary PQ2_0 (2.13 bpw) with Native MTP
2-bit / 2.13 bpw ternary quantization of Qwen3.8-27B (Prism ML Ternary Bonsai 2 architecture) with native 1-layer MTP speculative drafting.
- Format:
PQ2_0(group 128, 34 bytes/block, ~2.13 bpw) - Model Size: 7.66 GB
- MTP Drafter: Native 1-layer MTP head (Q8_0 projection matrices fine-tuned on Bonsai 2)
- Target Hardware: Single 24 GB GPU (RTX 3090 / 3090 Ti) with 245K+ context capacity under full offload
- Runtime: Optimized for llamAmpere (branch
feature/v0.3.1-bonsai2)
Usage with llamAmpere
./build-sm86/bin/llama-server \
-m Qwen3.8-27B-Ternary-PQ2_0-MTP.gguf \
-c 32768 -b 4096 -ub 1024 -t 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0Run jakeatx/Qwen3.8-27B-Ternary-PQ2_0-MTP-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models