SC117/Nex-N2-mini-APEX-GGUF overview

---

library_name: transformers

license: apache-2.0

license_link: https://huggingface.co/nex-agi/Nex-N2-mini/blob/main/LICENSE

pipeline_tag: image-text-to-text

tags:

• qwen3_5_moe
• qwen3_5
• agentic
• vision
• moe
• apex
• quantization

base_model:

• nex-agi/Nex-N2-mini

---

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 24px;">

<div style="background: #f5f5f7; border-radius: 20px; padding: 36px 32px; margin-bottom: 20px; text-align: center; position: relative; overflow: hidden;">

<div style="position: absolute; top: -30px; right: -30px; width: 120px; height: 120px; background: #ffedd5; border-radius: 50%;"></div>

<div style="position: absolute; bottom: -20px; left: 40px; width: 80px; height: 80px; background: #fed7aa; border-radius: 50%;"></div>

<div style="position: absolute; top: 50%; left: -15px; width: 60px; height: 60px; background: #ffedd5; border-radius: 50%;"></div>

<div style="display: inline-flex; gap: 8px; margin-bottom: 16px; position: relative; z-index: 1;">

<span style="background: #007aff; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">APEX</span>

<span style="background: #5856d6; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">Vision</span>

<span style="background: #34c759; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">Agentic</span>

</div>

<h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Nex-N2-mini</h1>

<p style="margin: 8px 0 0 0; font-size: 14px; color: #86868b; position: relative; z-index: 1;"><a href="https://huggingface.co/SC117/Nex-N2-mini-APEX-GGUF/blob/main/README_zh.md" style="color: #007aff; text-decoration: none;">📖 中文文档</a></p>

<p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">Agentic Vision MoE — APEX Quantized GGUF</p>

</div>

</div>

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; display: flex; flex-direction: column; gap: 20px; margin-bottom: 30px;">

<div style="border: 2px solid #f59e0b; border-radius: 12px; overflow: hidden; background: #fffbeb; box-shadow: 0 2px 8px rgba(245,158,11,0.1);">

<div style="background: linear-gradient(135deg, #f59e0b 0%, #d97706 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚡</span> Thinking Mode Requires Nex's Patched llama.cpp</div>

<div style="padding: 16px; font-size: 13px; color: #92400e; line-height: 1.8;">

<div style="background: #eff6ff; border: 1px solid #93c5fd; border-radius: 6px; padding: 10px 14px; margin-bottom: 12px; font-weight: 700;">👉 Using stock llama.cpp (without Nex's patch)? → <a href="https://huggingface.co/SC117/Nex-N2-mini-template-fix-APEX-GGUF" target="_blank" style="color: #1d4ed8; text-decoration: underline; font-weight: 900;">SC117/Nex-N2-mini-template-fix-APEX-GGUF</a> — works out of the box, no <code>--chat-template-file</code> needed</div>

<p style="margin: 0 0 12px 0;">Nex-N2-mini's original chat template uses complex vision processing macros that stock llama.cpp's Jinja parser cannot handle correctly. This causes <code><think></code> thinking tags to not be injected, breaking <code>--reasoning-format</code>.</p>

<p style="margin: 0 0 12px 0;"><b>The official fix:</b> Use <a href="https://github.com/nex-agi/llama.cpp/releases/tag/nex-b9596-fix-b9599-9cd1771" target="_blank" style="color: #92400e; text-decoration: none; font-weight: 700;">Nex's patched llama.cpp</a>, which works with the <b>unmodified GGUF and unmodified template</b>. Once Nex's upstream patch is merged into stock llama.cpp, this workaround will no longer be needed.</p>

<p style="margin: 0;"><b>⚠️ Do NOT modify <code>chat_template.jinja</code>.</b> The model was trained strictly on the current template — editing the tags deviates from the training-time format and <b>may degrade output quality</b>. See <a href="https://huggingface.co/nex-agi/Nex-N2-Pro/discussions/3" target="_blank" style="color: #92400e; text-decoration: none; font-weight: 700;">discussion #3</a>.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>💡</span> What is APEX?</div>

<div style="padding: 16px;">

<p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">These GGUF files are quantized using <a href="https://github.com/mudler/apex-quant" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">APEX</a>, a MoE-aware mixed-precision quantization technique that outperforms standard quantization methods while being significantly smaller.</p>

<p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7; font-weight: 700;">APEX beats Q8_0 perplexity at half the size — and even beats F16.</p>

<p style="margin: 0; font-size: 13px; color: #334155; line-height: 1.7;">APEX classifies every tensor by its role — routed expert, shared expert, or attention — and applies a layer-wise precision gradient, giving the most sensitive edge layers higher precision and compressing the redundant middle layers more aggressively.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📦</span> Available Files</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 38%;">File</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 12%;">Size</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 12%;">BPW</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 38%;">Note</th></tr></thead><tbody>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini.BF16.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">64.6 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">16.0</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Full precision reference</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini-APEX-Quality.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">21.3 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">5.23</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Highest quality, best accuracy</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini-APEX-Balanced.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">23.6 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">5.85</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Best all-rounder, recommended</td></tr>

<tr><td style="padding: 8px 10px; color: #334155;"><code>Nex-N2-mini-APEX-Compact.gguf</code></td><td style="padding: 8px 10px; color: #334155;">15.4 GB</td><td style="padding: 8px 10px; color: #334155;">3.81</td><td style="padding: 8px 10px; color: #334155;">Best quality/size ratio, 16GB VRAM</td></tr>

<tr><td style="padding: 8px 10px; color: #334155;"><code>mmproj-Nex-N2-mini.F16.gguf</code></td><td style="padding: 8px 10px; color: #334155;">858 MB</td><td style="padding: 8px 10px; color: #334155;">-</td><td style="padding: 8px 10px; color: #334155;">Vision projector (required for image/video)</td></tr>

</tbody></table>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧠</span> Model Details</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Architecture</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Qwen3.5 MoE (GatedDeltaNet + Full Attention) + Vision Encoder</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Parameters</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">35B total, 3B active per token</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Experts</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">256 routed experts, 8 active per token</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Layers</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">40 layers (30 linear_attn + 10 full_attn)</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Context</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">262,144 tokens</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Vision</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Image support (mmproj 858MB)</td></tr>

<tr><td style="padding: 6px 10px; font-weight: bold; color: #334155; background: white;">Thinking</td><td style="padding: 6px 10px; color: #4b5563; background: white;">Qwen3-style think tags — requires Nex's patched llama.cpp (see above)</td></tr>

</tbody></table>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage</div>

<div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">

<p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">Download Nex's patched llama.cpp</p>

<p style="margin: 0 0 12px 0; font-size: 12px; color: #64748b;">Binaries: <a href="https://github.com/nex-agi/llama.cpp/releases/tag/nex-b9596-fix-b9599-9cd1771" style="color: #c2410c; text-decoration: none;">nex-agi/llama.cpp</a> &nbsp;|&nbsp; Docker: <code>ghcr.io/nex-agi/llama.cpp:server-cuda-nex-b9596-fix-b9598-8c0d5c9</code></p>

<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server \

-m Nex-N2-mini-APEX-Quality.gguf \

-ngl 99 -ncmoe 19 -c 32768 \

--host 0.0.0.0 --port 8081</p>

<p style="margin: 12px 0 0 0; font-size: 12px; color: #64748b; font-style: italic;">Replace <code>Nex-N2-mini-APEX-Quality.gguf</code> with your preferred quantization tier. Add <code>--mmproj mmproj-Nex-N2-mini.F16.gguf</code> for vision. Recommended sampling: temperature 0.7, top_p 0.95, top_k 40, min_p 0.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📋</span> Original Model Benchmarks</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Benchmark</th><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Score</th><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Category</th></tr></thead><tbody>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">BrowseComp</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">74.1</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Agent</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">SWE-Bench Verified</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">74.4</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Coding</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Terminal-Bench 2.1</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">60.7</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Coding</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">GPQA Diamond</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">82.6</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Reasoning</td></tr>

<tr><td style="padding: 6px 10px; color: #334155;">IFEval</td><td style="padding: 6px 10px; color: #334155;">89.1</td><td style="padding: 6px 10px; color: #334155;">Instruction</td></tr>

</tbody></table>

<p style="margin: 12px 0 0 0; font-size: 12px; color: #64748b; font-style: italic;">From the original Nex-N2-mini model card (BF16, full precision).</p>

</div>

</div>

</div>

Using with Stock llama.cpp

If you cannot use Nex's patched llama.cpp, a template-fixed version is available at <a href="https://huggingface.co/SC117/Nex-N2-mini-template-fix-APEX-GGUF" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">SC117/Nex-N2-mini-template-fix-APEX-GGUF</a>. These GGUFs have a modified <code>chat_template.jinja</code> embedded so that <code>--reasoning-format</code> works on stock llama.cpp without <code>--chat-template-file</code>.

> ⚠️ The Nex team explicitly advises against modifying the chat template — the model was trained strictly on the original template, and deviating from the training-time format may degrade output quality. See <a href="https://huggingface.co/nex-agi/Nex-N2-Pro/discussions/3" target="_blank" style="color: #92400e; text-decoration: none; font-weight: 700;">discussion #3</a>. Use the template-fixed version only if you have no alternative, and be aware of the potential quality trade-off.

Links

• APEX Quantization: https://github.com/mudler/apex-quant
• Original Model: https://huggingface.co/nex-agi/Nex-N2-mini
• Nex's Patched llama.cpp (required for thinking mode): https://github.com/nex-agi/llama.cpp/releases/tag/nex-b9596-fix-b9599-9cd1771
• Template-Fixed GGUF (stock llama.cpp fallback): https://huggingface.co/SC117/Nex-N2-mini-template-fix-APEX-GGUF
• Chat Template Discussion: https://huggingface.co/nex-agi/Nex-N2-Pro/discussions/3
• Qwen3.5-35B-A3B: https://huggingface.co/Qwen/Qwen3.5-35B-A3B

<!-- updated -->