---

license: apache-2.0

base_model: nex-agi/Nex-N2.5-mini

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• qwen3.5
• qwen3.5-moe
• quantized
• text-generation
• image-text-to-text
• multimodal
• moe
• long-context
• tool-calling

---

Nex-N2.5-mini GGUF

Community GGUF quantizations of nex-agi/Nex-N2.5-mini.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About Nex-N2.5-mini

Nex-N2.5-mini is Nex-AGI's

multimodal, agent-oriented model for long-horizon tasks. The upstream card

describes the Nex-N2.5 family as supporting computer use, web browsing, visual

grounding, coding, reasoning, and tool calling. It also documents image and

video inputs through the official multimodal processor and chat template.

The upstream configuration identifies a Qwen3.5 Mixture-of-Experts model with

256 experts and 8 active experts per token, 40 text layers, and a 262,144-token

text context configuration. The upstream repository presents the mini variant

as a 35B-parameter BF16 model. The upstream deployment and benchmark details

are available in the official model card.

This release contains text GGUF files plus a separate

mmproj-Nex-N2.5-mini-F16.gguf vision projector. The local validation below

uses a 4,096-token context and does not claim that the full configured context

has been validated by this package.


*Upstream Nex-N2.5 benchmark overview; the image and scores belong to the

official model card.*

This is a quantization-only release. No training, fine-tuning, merging, or

weight modification other than GGUF conversion and quantization was performed.

The Q8_0 file was quantized directly from the converted BF16 GGUF; the other

published files used the same BF16 source and a model-specific importance

matrix. No GGUF file was used as the source for another quantization.

Fidelity measurements

The table below compares every published text GGUF with the BF16 reference on

a held-out WikiText pilot: eight chunks from wiki.test.raw and eight chunks

from wiki.valid.raw, with a 4,096-token context, 512 batch/ubatch, 64 CPU

threads, and the same Qwen3.5-compatible llama.cpp runtime. Values are averaged

across the two splits. Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1

agreement, indicate closer next-token behavior to BF16. The BF16 reference

mean PPL was 6.684743 in this pilot.

| File | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|

| Nex-N2.5-mini-Q8_0.gguf | 0.023277 | 94.523% | +0.986% | 4.132% |

| Nex-N2.5-mini-Q6_K.gguf | 0.027186 | 93.811% | -0.858% | 4.676% |

| Nex-N2.5-mini-Q5_K_M.gguf | 0.038910 | 92.636% | +0.036% | 5.262% |

| Nex-N2.5-mini-Q4_K_M.gguf | 0.063583 | 90.315% | +3.230% | 6.732% |

| Nex-N2.5-mini-IQ4_NL.gguf | 0.062792 | 90.273% | +1.378% | 6.774% |

| Nex-N2.5-mini-IQ4_XS.gguf | 0.065216 | 90.071% | +0.801% | 6.854% |

| Nex-N2.5-mini-Q3_K_L.gguf | 0.122060 | 86.334% | +3.083% | 9.158% |

| Nex-N2.5-mini-Q3_K_M.gguf | 0.129316 | 85.906% | +4.044% | 9.427% |

| Nex-N2.5-mini-IQ3_M.gguf | 0.172758 | 83.735% | +13.676% | 11.555% |

| Nex-N2.5-mini-IQ3_S.gguf | 0.154186 | 84.600% | +10.201% | 10.701% |

| Nex-N2.5-mini-Q2_K.gguf | 0.233133 | 80.709% | +10.839% | 12.770% |

| Nex-N2.5-mini-Q2_K_S.gguf | 0.278980 | 78.709% | +15.232% | 14.023% |

| Nex-N2.5-mini-IQ2_XS.gguf | 0.516971 | 71.590% | +51.851% | 19.371% |

| Nex-N2.5-mini-IQ1_M.gguf | 0.695877 | 66.064% | +69.116% | 24.275% |

| Nex-N2.5-mini-Q1_0.gguf | 8.616836 | 4.879% | +445003.665% | 61.752% |

For a general local profile, Q4_K_M is the practical starting point in this

pilot. IQ4_NL and IQ4_XS are compact Q4-region alternatives. Q5_K_M and Q6_K

are stronger quality/size choices, while Q8_0 is the highest-bit option.

Q3_K_L, Q3_K_M, IQ3_M, and IQ3_S are lower-memory Q3-region compromises.

Q2_K, Q2_K_S, IQ2_XS, IQ1_M, and Q1_0 are memory-constrained experimental

profiles and should be checked against the intended workload.

These measurements describe next-token fidelity relative to BF16; they are not

a direct percentage of capabilities retained. Instruction following,

reasoning, multilingual behavior, formatting, vision, and tool-calling quality

can vary by workload and should be evaluated separately when they matter.

The compact machine-readable results are available in

reproducibility/quality-summary.tsv.

Corpus hashes, conversion details, evaluation settings, runtime provenance,

and artifact hashes are recorded in

reproducibility/manifest.md.

Quick start

./llama-cli \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99 --cpu-moe --fit on --fit-target 1024

--cpu-moe keeps the MoE weights on the CPU and is useful when the available

GPU memory is smaller than the model working set. Omit it when the target

machine has enough memory and the runtime configuration has been tested for

that setup.

For the multimodal path, keep the text GGUF and the separate projector beside

the executable:

./llama-mtmd-cli \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --mmproj mmproj-Nex-N2.5-mini-F16.gguf \
  --image path/to/image.jpg \
  --jinja \
  -p 'Answer briefly in English: What is the main subject of this image?' \
  -n 64 -c 4096 -ngl 99 --cpu-moe --fit on --fit-target 1024

Reproducibility and validation

The source was locked to upstream revision

87420286149d9cce9bd46cd335ef9bda33c37c1b and converted directly from the

upstream BF16 safetensors. The text converter used --no-mtp because this

revision advertises MTP configuration but does not contain MTP tensors. The

vision projector was converted separately to F16.

All fifteen published text GGUF files passed llama.cpp tensor checks, load, and

English generation smoke tests. The BF16 reference also passed the same text

smoke profile. The Q4_K_M text file and the included F16 projector passed a

multimodal image smoke test. Runtime throughput is supplementary and is

available as the compact

reproducibility/runtime-summary.tsv;

it is not a quality score or a replacement for the fidelity table.

Raw conversion, calibration, quantization, smoke-test, fidelity, and benchmark

logs remain local and are intentionally not uploaded. Published artifact

checksums are in SHA256SUMS.txt.

License and attribution

The upstream model metadata specifies Apache License 2.0. Preserve the

upstream attribution and license when redistributing these derivative GGUF

artifacts. These are community GGUF quantizations, not an official

nex-agi/Nex-N2.5-mini release or endorsement.