---

license: apache-2.0

base_model: prism-ml/Ternary-Bonsai-4B-gguf

---

Ternary-Bonsai-4B-TQ2_0-GGUF

Requantized from prism-ml/Ternary-Bonsai-4B-gguf's F16 baseline to GGML's native TQ2_0 ternary type (~2.06 bits/weight), using stock llama.cpp (tag b9498) — not PrismML's custom fork/packing.

Sanity-checked via llama-server's OpenAI-compatible endpoint on CPU (Metal has no TQ1_0/TQ2_0 matmul kernels at this llama.cpp version — this model must run with -ngl 0 / CPU-only until upstream adds Metal support):

• Chat coherence: pass ("What is the capital of France?" -> "The capital of France is Paris.")
• Native tool-calling (Qwen3 <tool_call> format): FAIL, reproducible across 3 trials at temp=0. The model emits the correct tool-call JSON payload but consistently drops the opening <tool_call> tag (only emits the closing </tool_call>), breaking llama.cpp's native tool-call parser (finish_reason: stop instead of tool_calls). Do not enable native/strict tool-call parsing for this specific quantized model without a more lenient parser or further investigation.

License: Apache 2.0 (carried over from base model).