---

quantized_by: tarruda

pipeline_tag: image-text-to-text

base_model: nex-agi/Nex-N2-Pro

base_model_relation: quantized

license: apache-2.0

---

Nex N2 Pro GGUF

My custom IQ2_XXS/IQ3_XXS GGUF quantization for

nex-agi/Nex-N2-Pro

The chat template adds preserve_thinking option to improve experience with

coding agents on platforms with slow prompt processing.

Quant Recipes

| Recipe | Quant Size | Default type | Tensor-specific overrides |

| --- | --- | --- | --- |

| IQ3_XXS | 119093.00 MiB (2.52 BPW) | Q6_K | .attn=q8_0, ffn_down_exps=iq3_xxs, ffn_gate_exps=iq2_xxs, ffn_up_exps=iq2_xxs |

Usage

Here's an example script:

#!/bin/sh -e
model=./IQ3_XXS/Nex-N2-Pro-IQ3_XXS-00001-of-00004.gguf
mmproj=./IQ3_XXS/Nex-N2-Pro-mmproj-Q8_0.gguf

parallel=1
ctx=262144

reasoning_budget_message=". Need answer.
Let's draft final."

ctx_size=$((ctx * parallel))
llama-server --no-warmup --no-mmap \
  --model $model --mmproj $mmproj \
  -ctk q8_0 -ctv q8_0 \
  --ctx-size $ctx_size -np $parallel \
  --temp 0.7 --top-p 0.95 --top-k 40 \
  --repeat-penalty 1.0  --presence-penalty 0.0 \
  -cram 0 \
  --reasoning-budget-message "$reasoning_budget_message" \
  --reasoning-preserve \
  --ctx-checkpoints 2 --checkpoint-min-step 1024 \
  --host 0.0.0.0