---

license: apache-2.0

base_model: ProCreations/grug-3b

tags:

• grug
• gguf
• llama.cpp
• reasoning
• token-efficient

language:

• en

pipeline_tag: text-generation

---

grug-3b-qat-q4-gguf

q4 that survive the squeeze.

normal q4 round the weight after training and hope. this one train WITH the

rounding: every linear weight fake-quantized to asymmetric int4 (group 32) on

each forward, straight-through gradient update the bf16 weight underneath. model

learn weight that still work after Q4_K_M round them. same recipe as grug-9b-qat

and grug-27b-qat.

trained on same data as ProCreations/grug-3b,

so grug dialect and adaptive think length come through intact.

| file | size | note |

|---|---|---|

| grug-3b-qat-q4-Q4_K_M.gguf | 2.57 GB | the point of this repo |

| grug-3b-qat-q4-f16.gguf | 8.34 GB | qat weights unquantized, roll your own quant |

use the Q4_K_M one. plain (non-qat) quants live

here.

llama.cpp support

Nanbeige4.2 not in upstream llama.cpp yet (issue

#26086). Nanbeige team PR

#25994 add it - weight-shared

depth loop, num_loops=2. until merge, build from that branch:

git clone --depth 1 --branch nanbeige42 https://github.com/Nanbeige/llama.cpp
cd llama.cpp && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j
./build/bin/llama-cli -m grug-3b-Q4_K_M.gguf -p "What is 12 times 12?"

these gguf converted and load-probed with that branch.