---

license: apache-2.0

base_model: prism-ml/Ternary-Bonsai-2-27B-gguf

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- ternary

- mtp

- speculative-decoding

- qwen3.5

- opencoti

---

Ternary Bonsai 2 27B — MTP bundles (GGUF)

PrismML's Ternary Bonsai 2 27B

(a 27B dense model with end-to-end ternary weights, Qwen3.8-27B hybrid backbone,

262K context, Apache-2.0) with a NextN / MTP draft head bundled in, so the model

self-speculates — no second file, no external drafter.

The published PrismML GGUFs carry no NextN layer. These bundles add the upstream

Qwen/Qwen3.8-27B MTP head (its mtp.*

tensors, exported by upstream llama.cpp convert_hf_to_gguf.py --mtp --outtype q8_0)

as one extra layer; every other key and tensor is the PrismML file, byte for byte.

The head stays in the normal basis (it is not among the Hadamard-rotated tensors);

its token-embedding lookup on the rotated table gets the inverse transform in the

engine (PrismML's 518ad108).

| File | Weights | Size | sha256 |

|---|---|---|---|

| Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf | PQ2_0 (ternary, ~1.72 bpw) + q8_0 head | 7.66 GB | 211653d24a0bfdc3d1c5f65711b995f9311ce6b24363cfed3b9b113087de18b3 |

| Ternary-Bonsai-2-27B-PTQ1_0-MTP.gguf | PTQ1_0 (packed ternary) + q8_0 head | 6.40 GB | 174c9c6584e118766d5511e00a84fd8a31d146248915a726cfac8e6d140c6204 |

The vision projector is not duplicated here: use PrismML's

Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf (600 MiB) from the base repo with --mmproj.

Running

These files need an engine that runs ternary weights and a bundled NextN head

on a qwen35 hybrid: opencoti's llamafile

(from the c11 cut) does; PrismML's own llama.cpp fork runs the ternary weights but

not the bundled head.

# opencoti llamafile — NextN self-speculation; the draft depth for the ternary
# file types is a built-in table row (depth 2), no flag needed.
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
    -ngl 99 -fa on --spec-type draft-mtp

# with image input
./opencoti-llamafile --server -m Ternary-Bonsai-2-27B-PQ2_0-MTP.gguf \
    --mmproj Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf -ngl 99 -fa on --spec-type draft-mtp

The opencoti setup wizard lists this repo under the Ternary Bonsai 2 family and

offers the projector as an optional download (it is loaded at the first image

request, nothing is reserved for it at boot).

What MTP buys (greedy, 256-token answers, 12 chat prompts, medians)

| Card | PQ2_0, MTP off | PQ2_0, MTP on (depth 2) | acceptance |

|---|---|---|---|

| RTX PRO 6000 (Blackwell) | 128 tok/s | 161 tok/s | ~2.3 tokens / step |

| RTX 3090 | 46 tok/s | 57 tok/s | ~2.3 tokens / step |

Speculation preserves the model's distribution, not the argmax at near-ties: with

greedy sampling an answer can differ from the unspeculated one at a position where

the top-2 tokens are within a few hundredths of a nat (the verify batch rounds

differently from single-token decode). No degeneration was measured (distinct

4-gram ratio identical on and off).

Reproducing the bundle

1. Export the head: convert_hf_to_gguf.py --mtp --outtype q8_0 Qwen/Qwen3.8-27B

(upstream llama.cpp) → mtp-Qwen3.8-27B-Q8_0.gguf

(c9e5c641de0c9896641c07a400ed92075024fe2c623549d427b1a6e44075bc0a).

1. Bundle it into the PrismML GGUF with opencoti's bonsai2-mtp-bundle.py: the head

becomes layer block_count, nextn_predict_layers = 1; the script refuses unless

every shared qwen35.* hyperparameter and the vocab agree and no tensor collides.

The result is byte-identical across machines (the two sha256 above).

License

Apache-2.0, as the base model and the Qwen3.8-27B head. Credits: PrismML for Bonsai 2,

the Qwen team for Qwen3.8-27B.