thread13/OrionLLM_GRM-3.2-Sky-GGUF-el4-MTP overview

---

quantized_by: el4

pipeline_tag: image-text-to-text

tags:

• qwen
• qwen3_5_moe
• gguf
• mtp
• text-to-text
• conversational
• text-generation-inference

base_model:

• OrionLLM/GRM-3.2-Sky
• a4lg/Qwen3.5-35B-A3B-MTP-ONLY-GGUF

notes:

• based on local GGUF quantizations by el4, MTP GGUF layers by a4lg and (modified) conversion code by buzz and tooltd

---

TL;DR

Recently I was running some perplexity calculations (extending initial benchmarks started here),

and one thing that stood out was the quality of el4 ONYX models, especially its "quality" edition.

(Another thing that stood out was somewhat unfairly ignored quality of Kwaipilot/KAT-Coder-V2.5-Dev and OrionLLM/GRM-3.2-Sky.)

Now while el4 has an xspark draft model in their repository,

I am somehow more used to the now-classic MTP one -- or may be I just somehow had more luck with mtp draft models on my specific hardware.

Either way, I used a slightly corrected version of tooltd's 'grm_mtp_graft.py to do the same trick and graft a [Qwen3.6 MTP draft model][a4lg] by a4lg to the ONYX set made by el4, and am uploading these here.

_I shall probably also upload here the modified Python code and respective perplexity measurements, which could be of general interest._

[orion-sky]: https://huggingface.co/OrionLLM/GRM-3.2-Sky

[el4-onyx]: https://huggingface.co/el4/GRM-3.2-Sky-ONYX-GGUF

[tooltd-mtp]: https://huggingface.co/tooltd/GRM-3.2-Sky-MTP-GGUF

[apex]: https://github.com/localai-org/apex-quant

[unsloth]: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/blob/main/Qwen3.6-35B-A3B-UD-Q8_K_XL.gguf

[buzz]: https://gist.github.com/buzz/1c439684d5e3f36492ae9f64ef7e3f67

[tooltd-conv-py]: https://huggingface.co/tooltd/GRM-3.2-Sky-MTP-GGUF/blob/main/grm_mtp_graft.py

[a4lg]: https://huggingface.co/a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF

[orion-sky]: https://huggingface.co/OrionLLM/GRM-3.2-Sky

[sky-perf-img]: https://huggingface.co/OrionLLM/GRM-3.2-Sky/resolve/main/assets/agentic-performance-evaluation.png

[iq4-xs]: https://github.com/ggml-org/llama.cpp/pull/5747

[ik1-iqk]: https://github.com/ikawrakow/ik_llama.cpp/discussions/8

[ik2-imatrix]: https://github.com/ggml-org/llama.cpp/pull/4861

[code-contests-orig]: https://huggingface.co/datasets/deepmind/code_contests

[boutell-kv-q8-0-directory-naming-issues]: https://www.reddit.com/r/LocalLLaMA/comments/1sq94qx/is_anyone_getting_real_coding_work_done_with/

[garage-ppl-tests]: https://huggingface.co/thread13/endless-frontier_BigBang-v1-GGUF-stub/blob/main/perplexity_comparison.md