---

base_model: ConicCat/Gemma4-Writer-31B-F

tags:

• gguf
• llama.cpp
• gemma4
• writing

---

Gemma4-Writer-31B-F-GGUF

A local GGUF build of ConicCat/Gemma4-Writer-31B-F, a Gemma 4 31B fine-tune aimed at writing-oriented use.

This release is for people who want to run the model locally with llama.cpp-compatible software without loading the original BF16 checkpoint.

What you get

• Gemma4-Writer-31B-F-Q4_K_M.gguf
• Quantization: Q4_K_M
• Size: 17.40 GiB
• Good default for capable consumer GPUs and GPU+RAM offload setups
• Text generation and chat are supported
• This release is text-only; image/vision components are not included

Run with llama.cpp

Command line

llama-cli -m Gemma4-Writer-31B-F-Q4_K_M.gguf -ngl 999 -c 8192

OpenAI-compatible server

llama-server -m Gemma4-Writer-31B-F-Q4_K_M.gguf -ngl 999 -c 8192

Then connect any OpenAI-compatible frontend to the llama.cpp server.

The source model carries a Gemma 4 canonical chat template; the released GGUF passed llama.cpp chat-template smoke tests.

Choosing this quant

Q4_K_M is the first release because it provides a practical balance of model quality, memory use, and local inference speed for a 31B model.

If additional quantizations are published later, choose a larger quant when you have more memory and want to preserve more of the original weights.

Technical notes

The source checkpoint is multimodal, but this GGUF conversion intentionally contains the text model only. Vision/image weights and an mmproj artifact are not included.

Source: ConicCat/Gemma4-Writer-31B-F

Source revision used for conversion:

a64bd195933140ad821b906d7ec7b83bcbe8da92

GGUF SHA256:

24df949e9e11f2abf4da45651376cc51ba962957f45539188ad573b79822ac07

No MTP/NextN mismatch was detected.

The exact Q4_K_M file passed a real llama-server /v1/chat/completions smoke gate covering model load, system/user separation, exact-output compliance, arithmetic, normal stop behavior, and chat-control-token leakage.

A tokenizer metadata compatibility normalization was required during conversion for Transformers 4.57.6. It changes tokenizer configuration parsing only; model weights were not modified by that compatibility fix.

All model credit belongs to the original model authors.