jakeatx/Qwen3.8-27B-GGUF overview
Qwen3.8 27B GGUF Model Hub Unified release repository for Qwen3.8 27B quantizations, fine tunes, and speculative decoding configurations optimized for llamAmpe…
Runs locally from ~13.0 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf | GGUF | IQ4_XS | 14.52 GB | Download |
| Qwen3.8-27B-ATX-4-XS.gguf | GGUF | GGUF | 14.52 GB | Download |
| Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf | GGUF | IQ4_XS | 14.64 GB | Download |
| imatrix_qwopus_flash.gguf | GGUF | GGUF | 13.0 MB | Download |
| imatrix_swift_unc.gguf | GGUF | GGUF | 13.0 MB | Download |
| qwen3.8-27b-ternary-mtp.gguf | GGUF | GGUF | 5.81 GB | Download |
Model Details
Model README
---
license: apache-2.0
tags:
- gguf
- qwen3_5
- mtp
- speculative-decoding
- ternary
- ampere
- sm86
---
Qwen3.8-27B GGUF Model Hub
Unified release repository for Qwen3.8-27B quantizations, fine-tunes, and speculative decoding configurations optimized for llamAmpere on SM86 (RTX 3090 / 3090 Ti).
Available Models
| File | Base Architecture | BPW / Quant | MTP Drafter | Target Hardware / Fit |
| :--- | :--- | :--- | :--- | :--- |
| qwen3.8-27b-ternary-mtp.gguf | Prism ML Ternary Bonsai 2 27B | 1.75 bpw (PTQ1_0) | Native 1-layer MTP | 5.81 GiB; fits 262K context under 24 GB VRAM |
| Qwen3.8-27B-ATX-4-XS.gguf | Qwen3.8-27B Base | 4.56 bpw (ATX-IQ4_XS-M) | Native 1-layer MTP | 14.52 GiB; 245K context populated under 22.6 GB ready |
| ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf | Qwen3.8-27B Swift Uncensored | 4.56 bpw (IQ4_XS-M) | Native 1-layer MTP | 15.58 GiB |
| Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf | Qwopus3.8-27B Flash | 4.56 bpw (IQ4_XS-M) | Native 1-layer MTP | 15.58 GiB |
Running with llamAmpere
Build the SM86 runtime:
cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_CUDA_FA=ON \
-DCMAKE_CUDA_ARCHITECTURES=86 -DGGML_NATIVE=ON
cmake --build build-sm86 -j8 --target llama-serverRun with native MTP speculative drafting:
./build-sm86/bin/llama-server \
-m qwen3.8-27b-ternary-mtp.gguf \
-c 32768 -b 4096 -ub 1024 -t 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0Run jakeatx/Qwen3.8-27B-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models