---

license: mit

base_model: ProCreations/grug-9b

base_model_relation: quantized

pipeline_tag: text-generation

tags:

• code
• agent
• tool-use
• grug
• gguf
• llama.cpp

---

grug-9b-gguf

grug squish grug-9b into GGUF rocks. small rock fit small cave. all rock think 11-word grug thinks. all rock code.

grug-9b = Ornith-1.0-9B taught to reason SHORT: think tokens -94%, whole benchmark 3.3x faster, MBPP held (80->78), tool-picking better (+11). full story + honest tradeoffs on grug-9b card.

which rock for which cave

| rock | size (~) | quality | cave |

|---|---|---|---|

| Q8_0 | ~9.7 GB | basically lossless | 16GB+ VRAM or Mac |

| Q6_K | ~7.5 GB | very close | 12GB VRAM |

| Q5_K_M | ~6.5 GB | good | 10GB VRAM |

| Q4_K_M | ~5.5 GB | good, most popular rock | 8GB VRAM (RTX 4060 cave!) |

| Q3_K_M | ~4.4 GB | okay, some brain lost | small cave, phone-adjacent |

grug advice: Q4_K_M default. Q8_0 if cave big. below Q3, bird forget how code — grug not ship those.

how run

llama.cpp (need RECENT build — qwen3_5 architecture very new, old build no understand bird):

llama-cli -hf ProCreations/grug-9b-gguf:Q4_K_M

LM Studio: search grug-9b-gguf, pick rock, go.

warnings from grug

• text only. vision tower not come into GGUF cave. use full grug-9b weights if need eyes
• need llama.cpp from after qwen3_5 support land. if error say unknown architecture: update
• thinking come in <think> tag, short on purpose. that is whole point. not bug. grug proud