What license applies to NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF?

License: apache-2.0. Verify terms on Hugging Face before commercial use.

How do I run NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF locally?

Download a GGUF file from this page and load it in guIDE or llama.cpp. Pipeline task: image-text-to-text.

Model Intelligence Sheet

NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF overview

Quality first GGUF quants of OrionLLM/GRM 2.6 Plus 0628 Source model: https://huggingface.co/OrionLLM/GRM 2.6 Plus 0628 These quants were produced by my experi…

ggufquantizedimatrixmtpllama.cppqwen3_5reasoningagenticconversationalimage-text-to-textbase_model:OrionLLM/GRM-2.6-Plus-0628base_model:quantized:OrionLLM/GRM-2.6-Plus-0628license:apache-2.0endpoints_compatibleregion:us

Runs locally from ~8.30 GB disk (12 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads

2,868

Likes

Pipeline

image-text-to-text

Author

NikiKrutan

Repository Files & Downloads

16 GGUF files detected

Direct downloads for local inference

File	Type	Quantization	Size	Link
GRM-2.6-Plus-0628-MTP-10400-NIKI_IQ3_XXS.gguf	GGUF	IQ3_XXS	10.16 GB	Download
GRM-2.6-Plus-0628-MTP-11100-NIKI_IQ3_S.gguf	GGUF	IQ3_S	10.84 GB	Download
GRM-2.6-Plus-0628-MTP-11800-NIKI_IQ3_S.gguf	GGUF	IQ3_S	11.52 GB	Download
GRM-2.6-Plus-0628-MTP-12700-NIKI_IQ4_XS.gguf	GGUF	IQ4_XS	12.40 GB	Download
GRM-2.6-Plus-0628-MTP-13500-NIKI_IQ4_XS.gguf	GGUF	IQ4_XS	13.18 GB	Download
GRM-2.6-Plus-0628-MTP-14500-NIKI_IQ4_XS.gguf	GGUF	IQ4_XS	14.16 GB	Download
GRM-2.6-Plus-0628-MTP-15400-NIKI_IQ4_XS.gguf	GGUF	IQ4_XS	15.04 GB	Download
GRM-2.6-Plus-0628-MTP-16500-NIKI_Q5_K_M.gguf	GGUF	Q5_K_M	16.11 GB	Download
GRM-2.6-Plus-0628-MTP-17600-NIKI_Q5_K_M.gguf	GGUF	Q5_K_M	17.19 GB	Download
GRM-2.6-Plus-0628-MTP-18800-NIKI_Q5_K_M.gguf	GGUF	Q5_K_M	18.36 GB	Download
GRM-2.6-Plus-0628-MTP-20100-NIKI_Q6_K.gguf	GGUF	Q6_K	19.63 GB	Download
GRM-2.6-Plus-0628-MTP-21500-NIKI_Q6_K.gguf	GGUF	Q6_K	21.00 GB	Download
GRM-2.6-Plus-0628-MTP-23000-NIKI_Q6_K.gguf	GGUF	Q6_K	22.46 GB	Download
GRM-2.6-Plus-0628-MTP-8500-NIKI_IQ2_S.gguf	GGUF	IQ2_S	8.30 GB	Download
GRM-2.6-Plus-0628-MTP-9100-NIKI_IQ2_S.gguf	GGUF	IQ2_S	8.89 GB	Download
GRM-2.6-Plus-0628-MTP-9700-NIKI_IQ3_XXS.gguf	GGUF	IQ3_XXS	9.47 GB	Download

Model Details

Model ID	NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF
Author	NikiKrutan
Pipeline	image-text-to-text
License	apache-2.0
Base model	OrionLLM/GRM-2.6-Plus-0628
Last modified	2026-08-11T18:00:00.000Z

Model README

---

license: apache-2.0

base_model:

OrionLLM/GRM-2.6-Plus-0628

tags:

gguf
quantized
imatrix
mtp
llama.cpp
qwen3_5
reasoning
agentic
conversational

pipeline_tag: image-text-to-text

---

Quality-first GGUF quants of OrionLLM/GRM-2.6-Plus-0628

Source model: https://huggingface.co/OrionLLM/GRM-2.6-Plus-0628

These quants were produced by my experimental tool, niki-allocator.

The main difference of my method is per-tensor type allocation (not per tensor groups, layers, community lore etc.)

Each tensor gets what is predicted to be best for it.

The tool is under active development and testing, and has not been published yet.

Quantization quality comparison

To prove my method I've performed comparison with quants from other providers:

Quants for comparison were downloaded on August 4, 2026.

Lower is better

!Mean KLD, wikitext2 test corpus

!Mean KLD, niki-small-plus corpus

!p95 KLD, wikitext2 test corpus

!p95 KLD, niki-small-plus corpus

!p99 KLD, wikitext2 test corpus

!p99 KLD, niki-small-plus corpus

!Same top probability, wikitext2 test corpus

!Same top probability, niki-small-plus corpus

!RMS Δp, wikitext2 test corpus

!RMS Δp, niki-small-plus corpus

Sizes

My quant sizes were chosen to fill 8500-23000 MiB with geometric progression and rounded to 100 MiB.

No special size choosing was performed (there may be exact sizes that are better, but they are left on the table due to hardware constraints).

niki-allocator is able to produce allocation with ~1 MiB precision.

Which one to choose?

The bigger - the better. So it is always a compromise of what you can afford considering context size and other constraints. Choose the biggest you can fit.

Quant names

Hugging Face requires quant type in the filename to properly show it. But all my quants are mixed by design. I decided to include dominating quant type (by size, not by tensor count) to the name therefore. It may easily be less than 50% of the model though. So choose by size, ignore listed quant type. For my quants: bigger --> better. Always.

Methodology

niki-allocator

It is quality-first per-tensor quantization allocator for llama.cpp quantize.

Allocator decides, per tensor, which quant type to use so that the resulting GGUF hits an exact target file size with the best estimated quality.

It is not direct measurement of KLD-metrics, it is only estimation. But as you can see from real KLD evaluations estimation is good enough.

Raw results are already good. But I added some calibration levers on top to make it better (for Qwen3.5-0.8B I used uncalibrated estimation). These levers need series of KLD evaluations to move estimation closer to reality.

niki-allocator is work in progress. Not published yet.

Corpora

imatrix: niki-small corpus - ~330K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 16K chars) dropped. Used for imatrix only.
Calibration of niki-allocator and evaluation: niki-small-plus corpus - ~405K tokens from my own agentic sessions (English, Russian, code, reasoning). Deduped, cleaned, filtered by simhash similarity to include only really different turns, short turns (less than 12K chars) dropped. Does not include any exact or similar (by simhash) agentic turns from niki-small.
Independent evaluation: wikitext-2 test corpus - ~290 K tokens standard corpus. It wasn't seen during calibration or imatrix at all. Only for final evaluation before publishing. So my quants are not anyhow tuned for this dataset.

Chunk sizes

imatrix (niki-small): 8064 tokens (384 x 21). To catch reasoning and long turn traces, not just basic chat.
niki-small-plus evaluation: 6144 tokens (384 x 16). 66 chunks total. To catch reasoning and long turns, but to keep chunks amount somewhat reasonable (that's why lower than for imatrix).
wikitext-2 test evaluation: 768 tokens (384 x 2). 386 chunks total. It is not directly comparable with popular 512 tokens chunks!

Why 384 magic number?

My hardware is fastest with ubatch 384, so I use it.

Metrics

All metrics were evaluated with llama-perplexity against Q8_0 standard quant (without imatrix) due to hardware constraints.

Quantization was performed with patched llama-quantize from BF_16 source. Fully compatible with original llama.cpp.

All my and foreign quants were evaluated against the same base.

llama.cpp commit 876a4321163249c43ca4e986818fab5ab081f282.

The main metric during calibration of niki-allocator was mean KLD. Also I used p95 KLD, PPL and same top p as secondary metrics.

Better PPL doesn't always mean better other more important metrics. So it can't be used as a main metric (some people still use). It is just fast indicator (no need for logits base) to check for big problems.

Why not p99 KLD, p99.9 KLD, max KLD?

With such a small corpus (well, almost everybody seems to use wikitext-2 test with only 290K tokens) these metrics are not very robust. While p99 KLD can give some plausible comparison for the allocation method as a whole (that's why I include graphs of it), p99.9 is essentially a corpus-driven noise (too little data), max KLD is just random at any corpus size (one hardest token of the whole dataset, saying nothing about other hard tokens). Please correct me if I am wrong.

Evaluation results

niki-small-plus corpus

<table>

<tr>

<th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Source</th>

<th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Label</th>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

</table>

wikitext-2 test corpus

<table>

<tr>

<th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Source</th>

<th style="background: rgba(128, 128, 128, 0.18); text-align: center;">Label</th>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

<tr>

<td style="background: rgba(26, 14, 0, 0.8); text-align: left;">DAXZEIT</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 31, 12, 0.6); text-align: left;">bartowski</td>

</tr>

<tr>

<td style="background: rgba(0, 11, 38, 0.8); text-align: left;">NikiKrutan</td>

</tr>

</table>

MTP

I've tested MTP performance and discovered that there is no point to keep MTP head higher than IQ4_XS. Q3_K is near but slightly worse (used for lower quants). Exact allocation of IQ4_XS/Q3_K is chosen by allocator (not pinned). Lower than that there is considerable quality drop, so not used in my quants.

|---|---:|---:|---:|---:|

| — (no MTP) | — | 28.71 | n/a | 1.0 |

| IQ3_S | 172.2 | 48.61 | 0.9584 | 3.69 |

| Q3_K | 172.2 | 50.37 | 0.9597 | 3.95 |

| IQ4_XS | 216.8 | 52.63 | 0.9640 | 4.04 |

| Q4_0 | 227.9 | 51.47 | 0.9715 | 4.00 |

| Q4_K | 227.9 | 51.68 | 0.9569 | 4.07 |

| Q5_K | 279.2 | 51.46 | 0.9732 | 4.00 |

| Q6_K | 332.4 | 51.56 | 0.9743 | 4.03 |

| Q8_0 | 430.5 | 50.55 | 0.9755 | 4.02 |

Speculation parameters: --spec-draft-n-max 5 --spec-draft-p-min 0.8.

Standard llama.cpp speed-bench with all 11 categories x 5 samples, osl 128, concurrency 1, temperature 0.

GPUs: 2 x 5060Ti

Chat template

Chat template included with gguf's is v16 from here: https://huggingface.co/Moore2877/Qwen-Fixed-Chat-Templates-llamacpp

I believe it is good.

Multimodality

It works fine with original Qwen mmproj. I use this one: https://huggingface.co/DAXZEIT/Qwen3.6-27B-mmproj-hybrid-Q8_0-F16-gguf

Acknowledgments

OrionLLM - great source model
Qwen - great base model which we all are tinkering with
DAXZEIT - quants for comparison
bartowski - quants for comparison
llama.cpp - well, you know them already
Moore2877 - chat template
michaelw9999 - some ideas to start with

Run NikiKrutan/GRM-2.6-Plus-0628-MTP-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models