---

license: apache-2.0

base_model: ProCreations/grug-27b

tags:

• grug
• gguf
• qat
• llama.cpp
• reasoning

language:

• en

---

grug-27b-qat-q4-gguf

2026-07-23 refresh: rock now QAT-trained from grug-27b v2.1 weights

(deep-think + agent-discipline update). bench table below measured on the

v1-era rock vs v1 control - directional guide still; v2.1 rock inherit both

the QAT recovery AND the v2.1 brain improvements.

grug put 27b brain in four-bit cave DURING training. brain feel rounding rock

before final squish. this not normal quant. this QAT recovery rock, made for

Q4 people.

recipe (same as grug-9b-qat, scaled): full-weight QAT on

grug-27b, fake int4 asymmetric

group-32 with straight-through gradient, ~3M token of grug-think data,

Adafactor LR 2e-6, 249 step on one H200. release = 25% QAT move + 75% original

anchor (full QAT overcorrect, 9b teach grug this). then fresh BF16 export,

quantize ONE time to Q4_K_M.

rocks

| file | what |

|---|---|

| grug-27b-qat-Q4_K_M.gguf | the QAT rock, ~16.5 GB |

| mmproj-grug-27b-f16.gguf | eye rock (vision), same as main gguf repo |

number. same cave, same harness, same llama.cpp build

three rock fight: ordinary Q4 (control), full-QAT Q4, and this rock (25%

QAT blend). full-QAT win MBPP big but BREAK tool hand (agent valid 96.6 ->

82.8). grug no ship broken hand. blend rock best overall:

| test | control Q4 | full-QAT Q4 | THIS ROCK |

|---|---:|---:|---:|

| MBPP-60 pass % | 81.7 | 91.7 | 88.3 (+6.6) |

| GSM8K-60 % | 96.7 | 95.0 | 98.3 (+1.6) |

| agent tool-call valid % | 96.6 | 82.8 | 96.6 (same) |

| agent right-tool % | 93.1 | 79.3 | 89.7 (-3.4) |

| agent args schema-valid % | 100 | 100 | 100 |

| identity loop rate % | 3.3 | 0.0 | 1.7 (halved) |

| greedy longform loops | 1 | 0 | 0 |

QAT feel rounding rock during training -> Q4 squish hurt less. coding and math

UP, loop sickness DOWN, tool hand intact. small right-tool dip is the honest

trade. grug show all numbers, hide nothing.

if rock act broken

single-token spam ("/" forever etc) = NOT the rock. hybrid DeltaNet brain

CANNOT survive llama.cpp context-shift: old builds shift on context overflow

and corrupt the recurrent state into token spam. fix:

• use RECENT llama.cpp (qwen3_5 support; new builds refuse instead of shift)
• agent frontends (OpenCode etc): set -c 16384 or bigger
• still broken? re-download rock (verify size) + check backend

grug re-test rock after every report: loads clean, zero spam at proper config.

how run

llama-server -m grug-27b-qat-Q4_K_M.gguf --mmproj mmproj-grug-27b-f16.gguf \
  -c 16384 --temp 0.6 --top-p 0.95 --top-k 20

need recent llama.cpp (qwen3_5 arch). grug think live in <think>, short on

purpose. tool call use XML format. main model card:

grug-27b.