---

license: mit

language: en

library_name: transformers

pipeline_tag: text-generation

tags:

- causal-lm

- mixture-of-experts

- reasoning

- ternary

- custom-code

---

Maple-Preview-GGUFs

Custom llama.cpp fork and setup instructions: github.com/deepgrove-ai/llama.cpp

We include the following gguf variants:

| Variant | GGUF size |

| --- | ---: |

| TQ1_0 + Q4_K head | 4.64 GiB |

| TQ1_0 + FP16 head | 5.06 GiB |

| TQ2_0 + Q4_K head | 5.50 GiB |

| TQ2_0 + FP16 head | 5.91 GiB |

TQ1_0 and TQ2_0 are different ternary packing schemes. Use TQ2_0 for generally faster speeds but slightly higher memory.

LM-head is kept in higher precision - either Q4_k or FP16.

Speed

M5 Pro, CPU-only, 16 threads, 512 prompt tokens, 128 generated tokens, 3 repetitions:

| Matrix weights | LM head | GGUF size | Prefill (pp512) | Decode (tg128) |

| --- | --- | ---: | ---: | ---: |

| TQ1_0 | FP16 | 5.06 GiB | 515.41 ± 0.28 tokens/s | 161.06 ± 0.57 tokens/s |

| TQ1_0 | Q4_K | 4.64 GiB | 513.33 ± 3.62 tokens/s | 231.13 ± 0.13 tokens/s |

| TQ2_0 | FP16 | 5.91 GiB | 618.57 ± 2.12 tokens/s | 169.81 ± 2.94 tokens/s |

| TQ2_0 | Q4_K | 5.50 GiB | 610.48 ± 3.76 tokens/s | 252.74 ± 0.37 tokens/s |

Architecture

Maple-Preview is a 20B-A1B reasoning model designed from the start for efficient on-device inference. It utilizes a 24-layer, 256-expert (8 active) configuration with 3:1 SWA-512:GA attention.

Evaluation

On benchmarks, Maple-Preview sets a new point on the Pareto frontier for both memory-to-performance and speed-to-performance, demonstrating its strong reasoning capabilities. However, we note that this preview is focused primarily on raw reasoning and, as such, may underperform on agentic benchmarks. We intend to continue improving general performance through extended training before Maple's full release.

License

Maple-Preview is released under the MIT License.