---

license: apache-2.0

base_model: Qwen/Qwen3.8-27B

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- imatrix

- quantization

- qwen3.8

---

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace;">

<div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // QWEN3.8-27B GGUF // AGENTIC-CALIBRATED IMATRIX</div>

<div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;">

<pre style="margin:0; font-size:9px; line-height:1.15;">

65 blocks

┌────────────────────────────┐

│ ████████████████████████ │ 48 SSM (state, no KV growth)

│ ▓ ▓ ▓ ▓ ▓ ▓ ▓ │ 16 attention (every 4th)

│ ░ │ 1 MTP head

└────────────────────────────┘

only 16 layers hold a growing KV cache

</pre>

<div style="text-align:center;">

<div style="font-size:23px; font-weight:800; letter-spacing:1px;">Qwen3.8-27B-GGUF</div>

<div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">hybrid SSM+attention</span> · <span style="white-space:nowrap;">27.32B</span> · <span style="white-space:nowrap;">262K native ctx</span> · <span style="white-space:nowrap;">agentic imatrix</span></div>

</div>

</div>

<table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-size:12px;">

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZES</div><div style="font-weight:700;">13.5–16.8 GB</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">qwen35</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">262,144</div></td>

</tr>

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">agentic domain</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LOAD-TESTED</div><div style="font-weight:700;">all 3 tiers</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RUNS ON</div><div style="font-weight:700;">llama.cpp</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">Apache-2.0</div></td>

</tr>

</table>

</div>

Qwen3.8-27B — GGUF quants with an agentic-domain imatrix

Three imatrix-guided quants of Qwen/Qwen3.8-27B.

The importance matrix is calibrated on agentic tool-use and code traces rather than on a

general-purpose corpus — the hypothesis being that preserving the weights that matter for

tool-calling helps on agentic work.

That hypothesis is not yet demonstrated. An agentic benchmark run is in progress; this card

will be updated with the number whether it is favourable or not. Until then, treat the calibration

as a design choice with a rationale, not as a proven advantage.

📦 Files

| file | size | bits |

|---|---|---|

| Qwen3.8-27B-Q3_K_M.gguf | 13.50 GB | ~3 |

| Qwen3.8-27B-IQ4_XS.gguf | 15.31 GB | ~4 |

| Qwen3.8-27B-Q4_K_M.gguf | 16.81 GB | ~4 |

| Qwen3.8-27B-agentic.imatrix | 13.6 MB | the importance matrix itself |

Q4_K_M is the one to take unless you are short on memory.

🧠 What this model is, and why the context is cheap

Of its 65 blocks: 48 are SSM / linear-attention layers, 16 are full attention (every 4th),

and 1 is an MTP head. Only those 16 attention layers hold a KV cache that grows with sequence

length, which is why a 262,144-token window is tractable here:

16 layers × 4 kv-heads × (256 + 256) × 2 bytes = 64 KB per token

A dense model with all 65 blocks attending would pay roughly 256 KB/token — four times as much.

Full arithmetic and per-context memory tables in CONTEXT.md.

⚠️ The flip side, measured: that same architecture makes prefill slow in llama.cpp today.

In multi-turn agentic loops, where the conversation is re-processed each turn, this model spends

far more time on prefill than a dense model of the same size would. Cheap in memory, expensive in

time. If you are building an agent loop on it, budget accordingly.

✅ What is verified

• All three tiers load and generate under llama.cpp (llama-bench, exit code 0). Magic checked

on every file.

• imatrix covers all 64 transformer blocks (block 64 is the MTP head, which llama.cpp does not

use and therefore has no coverage — that is expected, not a gap).

• Checksums: reports/artifact-sha256sums.txt.

❌ What is NOT verified — read before relying on it

• No agentic benchmark number yet. The run is in progress. No claim about tool use or coding

ability is made here.

• No KL-divergence or perplexity table. This release deliberately skips fidelity-vs-original

metrics; the intent is to report task performance instead, and that measurement is pending.

If you want a release where fidelity is measured tier by tier against the BF16 reference, see

Qwen3.6-27B-GGUF.

• The 262,144 context is the model's declared native window, not one we verified. No

needle-in-a-haystack run was completed.

• Text only. The base model is multimodal; this repo ships no mmproj, so vision inputs

will not work. Other publishers ship one — see the links below.

🚀 How to run it

llama-cli -m Qwen3.8-27B-Q4_K_M.gguf -c 32768 -no-cnv -p "your prompt"

-no-cnv matters: without it, recent llama-cli builds drop into conversation mode and wait on

stdin, which looks exactly like a hang when scripted.

Sampling — Qwen documents two profiles for this model, and the choice matters a great deal:

| mode | temperature | top_p | top_k | presence_penalty |

|---|---|---|---|---|

| Thinking (use this for agentic/reasoning work) | 1.0 | 0.95 | 20 | 0.0 |

| Instruct (non-thinking) | 0.7 | 0.80 | 20 | 1.5 |

The Modelfiles in configs/ use the thinking profile at 8K / 32K / 128K / 262K.

ollama create qwen3.8-27b-32k -f configs/Modelfile.32k

🔁 Provenance

• Scripts: scripts/ — the exact convert / imatrix / ladder steps used.
• Source: converted from the original safetensors to Q8_0, then quantized per tier with the

imatrix. Weights otherwise unmodified — this is a faithful quantization, not a fine-tune.

• imatrix corpus: agentic tool-use and code traces. Composition is not published.

🙏 Credit & alternatives

Model, weights and training are © Qwen —

Qwen/Qwen3.8-27B, Apache-2.0. This repo adds only the

quantization and the context configs.

Other GGUF builds of this model, several of them more complete than this one — they ship the full

ladder and the mmproj for vision: unsloth,

bartowski,

ggml-org,

lmstudio-community.

🗒️ Changelog

• 2026-08-16 — first release: Q3_K_M, IQ4_XS, Q4_K_M with an agentic-domain imatrix, context

Modelfiles, scripts and checksums. Agentic benchmark pending; card will be updated with the

result either way.