---

license: apache-2.0

base_model:

- prism-ml/Ternary-Bonsai-27B-gguf

tags:

- gguf

- ternary

- 2-bit

- multimodal

- neurogrid

---

Ternary-Bonsai-27B GGUF (NeuroGrid mirror)

A mirror of the prism-ml/Ternary-Bonsai-27B-gguf

weights (Apache-2.0, base Qwen/Qwen3.6-27B), kept here because these are the

exact files the NeuroGrid engine

serves in production and pinning them makes a deployment reproducible. The

weights are not ours and are not modified. Credit and licence belong upstream.

Which file to download

Three quantized files with two different block geometries share the same

2-bit weights. Picking the wrong one gets you a load error, not bad output.

| File | Block | bpw | Loads on |

|---|---|---|---|

| Ternary-Bonsai-27B-Q2_g64.gguf (7.59 GB) | 64 | 2.25 | upstream llama.cpp (GGML_TYPE_Q2_0 = 42) and NeuroGrid |

| Ternary-Bonsai-27B-Q2_0.gguf (7.17 GB) | 128 | 2.125 | the PrismML fork and NeuroGrid |

| Ternary-Bonsai-27B-PQ2_0.gguf (7.17 GB) | 128 | 2.125 | the same group-128 layout under its own name |

| Ternary-Bonsai-27B-mmproj-Q8_0.gguf (0.63 GB) | | | the CLIP vision tower, pair it with any of the above |

The Q2_0 name means two different byte layouts depending on who wrote the

file: upstream settled on group 64, the PrismML fork uses group 128. Both are

here so you do not have to guess. NeuroGrid reads both and refuses a file its

device kernel cannot run at load time rather than serving it wrong.

• Serving on NeuroGrid: take Q2_0 (group 128), which is what the device

GEMV kernel wants.

• Comparing against llama.cpp: take Q2_g64.

Running it on NeuroGrid

One process serves chat, vision and embeddings:

neurogrid -solo \
  -bonsai-gguf   Ternary-Bonsai-27B-Q2_0.gguf \
  -bonsai-mmproj Ternary-Bonsai-27B-mmproj-Q8_0.gguf \
  -bonsai-max-seq 16384 -max-seq-len 16384 \
  -http-port 8080 -host 127.0.0.1

Vision is enabled by the tower being loaded, not by the model name you

advertise, so -bonsai-mmproj is the whole switch. Images arrive as

OpenAI-style image_url content parts on /v1/chat/completions.

On CUDA the useful environment is NG_QWEN35_GPU=1,

NG_QWEN35_GPU_LMHEAD=1, NG_QWEN35_GRAPH=1 and NG_CLIP_GPU=1; add

NG_QWEN35_BATCH=8 with NG_QWEN35_BATCH_GRAPH=1 to run up to 8 concurrent

sequences through one decode step.

Running it on llama.cpp

llama-server -m Ternary-Bonsai-27B-Q2_g64.gguf \
  --mmproj Ternary-Bonsai-27B-mmproj-Q8_0.gguf -c 16384

Group-64 only. The 7.17 GB files will not load on ggml-org/llama.cpp master.