---

license: apache-2.0

base_model: ProCreations/grug-35b-v2

tags: [grug, gguf, qat, llama.cpp, reasoning, moe]

language: [en]

---

grug-35b-qat-q4-gguf

35b MoE brother in QAT four-bit rock. brain feel rounding rock during

training so Q4 squish hurt less.

recipe: expert-freeze QAT on grug-35b-v2 -

ALL text linear (expert include) fake-quant int4 g32 in forward, gradient only

flow to attention/DeltaNet/shared path (1.4B trainable; expert too heavy for

one cave GPU). ~1.7M grug token, Adafactor lr 2e-6. release = 25% QAT + 75%

original anchor (grug family standard). fresh bf16 export, ONE Q4_K_M squish.

rocks

| file | what |

|---|---|

| grug-35b-qat-Q4_K_M.gguf | the QAT rock (~21 GB) |

| mmproj-grug-35b-v2-f16.gguf | eye rock (vision) |

if rock act broken

single-token spam = context-shift corruption, not rock. recent llama.cpp +

-c 16384+ for agent frontends. see main gguf card for full troubleshoot.

how run

llama-server -m grug-35b-qat-Q4_K_M.gguf --mmproj mmproj-grug-35b-v2-f16.gguf \
  -c 16384 --temp 0.6 --top-p 0.95 --top-k 20

ordinary rocks: grug-35b-v2-gguf.

27b QAT brother: grug-27b-qat-q4-gguf.

grug made by ProCreations.