---

library: gguf

language: en

license: apache-2.0

base_model: KittyLM/kittylm-qwen3-4b

pipeline_tag: text-generation

tags:

• kittylm
• gguf
• ollama
• llama-cpp
• lm-studio
• conversational

---

KittyLM-4B GGUF (qwen3-4b)

Quantized GGUF weights for KittyLM/kittylm-qwen3-4b — the kitten-speak LoRA finetune of

Qwen/Qwen3-4B. Finetune details, eval results, and license notes live on the main repo card.

Training data: KittyLM/kittylm-data.

Files

• kittylm-qwen3-4b-q4_k_m.gguf
• kittylm-qwen3-4b-q5_k_s.gguf
• kittylm-qwen3-4b-q6_k.gguf
• kittylm-qwen3-4b-q8_0.gguf

Quant roadmap: Q4_K_M (recommended daily driver), Q5_K_S (small quality bump over Q4_K_M), Q6_K (near-lossless, large), Q8_0 (near-F16 fidelity, biggest). Q4_K_M ships today and is the recommended daily driver;

higher-bit quants trade size for fidelity (see scripts/quantize_gguf.py).

Usage (Ollama, direct — recommended)

No Modelfile needed. This repo ships Ollama template (chatml family), system

(kitten persona), and params files, so the HF integration picks them up automatically:

ollama run hf.co/KittyLM/kittylm-qwen3-4b-gguf:Q4_K_M
# other quants (when present): :Q8_0, :Q6_K, :Q5_K_S

Usage (Ollama, classic Modelfile)

A ready-made Modelfile lives in this repo (FROM points at the Q4_K_M file):

hf download KittyLM/kittylm-qwen3-4b-gguf --include "*.gguf" --local-dir ./gguf
hf download KittyLM/kittylm-qwen3-4b-gguf Modelfile --local-dir ./gguf
ollama create kittylm-qwen3-4b -f ./gguf/Modelfile
ollama run kittylm-qwen3-4b "Good night!"

Usage (llama.cpp / LM Studio)

Point llama.cpp (llama-cli -m <file>.gguf) or LM Studio at the .gguf file directly.

Apply the model's chat template and keep the KittyLM system prompt (also shipped as the

system file in this repo and as system_prompt.txt in KittyLM/kittylm-data).