---

license: apache-2.0

tags:

- transformers

- gguf

- qwen3

- thinkingcap

- efficient-thinking

- reasoning

- token-efficient

- lora

- sft

- abliterated

- multi-token-prediction

- mtp

- llama.cpp

- text-generation

base_model:

- hotdogs/Qwen3.8-27B-thinkingcap-abliterated

model:

- hotdogs/Qwen3.8-27B-thinkingcap-abliterated-mtp-GGUF

datasets:

- hotdogs/thinkingcap-sft-qwen38-27b

pretty_name: Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

---

Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

llama.cpp GGUF from the ThinkingCap SFT-merged model

(hotdogs/Qwen3.8-27B-thinkingcap-abliterated).

Trained from the on-policy oracle-verified dataset.

All 3 quantizations preserve the MTP layer (blk.64, 15 tensors

including 4 nextn.* projection tensors). 866 tensors total, 27.3B params.

Files

| File | Size | BPW | Notes |

|------|------|-----|-------|

| Qwen3.8-27B-thinkingcap-abliterated-mtp-f16.gguf | 51 GB | 16.0 | lossless |

| Qwen3.8-27B-thinkingcap-abliterated-mtp-Q6_K.gguf | 21 GB | 6.56 | best quality/size |

| Qwen3.8-27B-thinkingcap-abliterated-mtp-Q4_K_M.gguf | 16 GB | ~5.1 | fastest |

MTP verification

import gguf
g = gguf.GGUFReader("Qwen3.8-27B-thinkingcap-abliterated-mtp-f16.gguf")
mtp = [t.name for t in g.tensors if 'blk.64' in t.name]
print(len(mtp))  # 15

Serve (llama.cpp)

CUDA_VISIBLE_DEVICES=0,1,2,3,4 llama-server \
  -m Qwen3.8-27B-thinkingcap-abliterated-mtp-Q6_K.gguf \
  --n-gpu-layers 999 --ctx-size 8192 --parallel 2 \
  --batch-size 4096 --flash-attn on \
  --temp 1 --top-k 20 --top-p 0.95 --min-p 0.0 --jinja

📊 Benchmarks

Tested with inspect_ai 0.3.260 on local llama.cpp, reasoning_effort=low,

temperature=1, top_k=20, top_p=0.95. All runs are single-epoch (no multi-epoch averaging).

| Benchmark | Q6_K (this repo) | Q4_K_M (this repo) | Qwen3.8-27B¹ | Qwen3.6-27B | Opus 4.6 Max | GPT-5.5 | Kimi K3 | Gemini 3.1 Pro |

|-----------|:--------------------:|:---------------------:|:------------:|:-----------:|:------------:|:-------:|:-------:|:--------------:|

| GPQA Diamond (198) | 87.1% | 79.5% | 89.2%¹ | 87.8%¹ | 91.3%¹ | 93.6% | 93.5% | 94.3%⁸ |

| IFEval (541, strict)² | 88.2% | 88.4% | ~91.5%⁷ | ~90.4%⁷ | 95.0%⁷ | 94.2% | 92.8% | 93.5%⁸ |

| SWE-bench Verified (50)⁴ | 69.4% (34/49)¹⁰ | 63.3% (31/49)⁹ | ~82%⁶ | ~77.2%⁶ | ~80.8%⁶ | 88.7% | 76.8% | 80.6%⁸ |

!Benchmark comparison — this repo vs frontier models

¹ Official vendor-reported scores from the Qwen3.8-27B model card, reasoning_effort=xhigh.

² Standard Google IFEval via inspect_evals/ifeval (541 prompts, prompt-level strict accuracy).

⁴ inspect_evals/swe_bench_verified_mini: 50 issues (25 django + 25 sphinx), message_limit=100 steps,

max_tokens=16384/turn, tool_timeout=210s, no internet, Docker sandbox.

⁶ External/independent evaluations (not the official model card). SWE-bench Verified (500) is a different, harder split than our 50-task mini. Treat as directional, not exact.

⁷ External comparison via serenitiesai.com (F16 reference values). Treat as directional.

⁸ From SmartScope Gemini 3.1 Pro Benchmark Analysis (2026).

GPT-5.5 and Kimi K3 SWE scores from MorphLLM SWE-bench Pro Leaderboard.

⁹ 31/49 of-scored (1 out: sphinx-doc__sphinx-9229, step-limited at 117 msgs). Run status error (APITimeoutError at scheduler level) but all 50 samples completed in summaries.

¹⁰ 34/49 of-scored (1 out: sphinx-doc__sphinx-9229, timeout at 162 msgs / 190 min). Run status cancelled — last sample hit the 3600s timeout.

> Reading it: Q6_K scores within 2.1 pp of the full BF16 Qwen3.8-27B on GPQA

> (87.1 vs 89.2) — with a quantized, abliterated, LoRA-merged GGUF running on

> consumer GPUs. On IFEval, Q6_K (88.2%) and Q4_K_M (88.4%) are near-identical and sit

> within ~3 pp of the F16 reference (~91.5% for Qwen3.8-27B, 95.0% for Opus 4.6). On

> SWE-bench, Q6_K leads at 69.4% (34/49) vs Q4_K_M 63.3% (31/49) — both timed out

> on the same hard issue (sphinx-9229); external runs of the full 500-task SWE-bench

> Verified put Qwen3.8-27B at ~82% (footnote ⁶).

> ⚠️ Hardware limitation: Two-box setup — Box A: 1× RTX 3090 24GB + 1× RTX 3060 12GB (36 GB),

> Box B: 1× RTX 4060 Ti 16GB + 1× RTX 3060 12GB (28 GB). Total 64 GB VRAM across

> both boxes. F16 (55 GB) fits on Box A with tight KV-cache; Q6_K (21 GB) and

> Q4_K_M (16 GB) are the practical daily-drivers. F16 / Q8_K benchmark runs are

> pending — this table will be updated when they land.

> ⚠️ Token & step limits: SWE-bench is the most constrained benchmark:

> message_limit=100 (agent turns) × max_tokens=16384/turn ≈ 1.6M output-token

> ceiling. The model is typically step-limited, not token-limited (agent runs

> out of turns before exhausting the token budget). GPQA is single-shot with

> max_tokens=65536, so the token ceiling is rarely hit (avg ~4K tokens/question).

Smoke test

| Problem | Think chars | Answer | Correct |

|---------|-------------|--------|---------|

| 27 x 43 | 163 | 1161 | Yes |

| bat + ball = 1.10, bat = ball + 1.00 | 161 | ball = 0.05 | Yes |

| snail 10m wall, +3m day, -2m night | 359 | day 5 | Yes |

Base model (pre-SFT) used 4000+ think-chars on the snail problem.

ThinkingCap SFT brings it to 359 (-91%).