SC117/Nex-N2-mini-template-fix-APEX-GGUF overview

---

library_name: transformers

license: apache-2.0

license_link: https://huggingface.co/nex-agi/Nex-N2-mini/blob/main/LICENSE

pipeline_tag: image-text-to-text

tags:

• qwen3_5_moe
• qwen3_5
• agentic
• vision
• moe
• apex
• quantization

base_model:

• nex-agi/Nex-N2-mini

---

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 24px;">

<div style="background: #f5f5f7; border-radius: 20px; padding: 36px 32px; margin-bottom: 20px; text-align: center; position: relative; overflow: hidden;">

<div style="position: absolute; top: -30px; right: -30px; width: 120px; height: 120px; background: #ffedd5; border-radius: 50%;"></div>

<div style="position: absolute; bottom: -20px; left: 40px; width: 80px; height: 80px; background: #fed7aa; border-radius: 50%;"></div>

<div style="position: absolute; top: 50%; left: -15px; width: 60px; height: 60px; background: #ffedd5; border-radius: 50%;"></div>

<div style="display: inline-flex; gap: 8px; margin-bottom: 16px; position: relative; z-index: 1;">

<span style="background: #007aff; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">APEX</span>

<span style="background: #5856d6; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">Vision</span>

<span style="background: #34c759; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">Agentic</span>

<span style="background: #ff9500; color: white; font-size: 11px; font-weight: 600; padding: 5px 14px; border-radius: 20px;">⚠ Template Fix</span>

</div>

<h1 style="margin: 0 0 8px 0; font-size: 32px; font-weight: 700; color: #1d1d1f; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">Nex-N2-mini</h1>

<p style="margin: 8px 0 0 0; font-size: 14px; color: #86868b; position: relative; z-index: 1;"><a href="https://huggingface.co/SC117/Nex-N2-mini-template-fix-APEX-GGUF/blob/main/README_zh.md" style="color: #007aff; text-decoration: none;">📖 中文文档</a></p>

<p style="margin: 0; font-size: 15px; color: #86868b; position: relative; z-index: 1;">Agentic Vision MoE — APEX Quantized GGUF (Stock llama.cpp)</p>

</div>

</div>

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; display: flex; flex-direction: column; gap: 20px; margin-bottom: 30px;">

<div style="border: 2px solid #ef4444; border-radius: 12px; overflow: hidden; background: #fff5f5; box-shadow: 0 2px 8px rgba(239,68,68,0.1);">

<div style="background: linear-gradient(135deg, #ef4444 0%, #dc2626 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚠️</span> Temporary Workaround — Not Official</div>

<div style="padding: 16px; font-size: 13px; color: #991b1b; line-height: 1.8;">

<p style="margin: 0 0 12px 0; background: #fef2f2; border: 1px solid #fca5a5; border-radius: 6px; padding: 8px 12px; font-weight: 700;">👉 Want the unmodified, recommended version? Go to <a href="https://huggingface.co/SC117/Nex-N2-mini-APEX-GGUF" target="_blank" style="color: #991b1b; text-decoration: underline; font-weight: 900;">SC117/Nex-N2-mini-APEX-GGUF</a> (use with Nex's patched llama.cpp)</p>

<p style="margin: 0 0 12px 0;"><b>This is a temporary, unofficial workaround for stock llama.cpp users.</b> The original <code>chat_template.jinja</code> has been replaced with a <a href="https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates" target="_blank" style="color: #991b1b; text-decoration: none; font-weight: 700;">fixed version</a> so that <code>--reasoning-format</code> works without <code>--chat-template-file</code>.</p>

<p style="margin: 0 0 12px 0;"><b>⚠️ The Nex team explicitly recommends against modifying the chat template.</b> The model was trained strictly on the original template — deviating from the training-time format <b>may degrade output quality</b>. See <a href="https://huggingface.co/nex-agi/Nex-N2-Pro/discussions/3" target="_blank" style="color: #991b1b; text-decoration: none; font-weight: 700;">discussion #3</a> for details.</p>

<p style="margin: 0 0 12px 0;"><b>The recommended approach</b> is to use <a href="https://github.com/nex-agi/llama.cpp/releases/tag/nex-b9596-fix-b9599-9cd1771" target="_blank" style="color: #991b1b; text-decoration: none; font-weight: 700;">Nex's patched llama.cpp</a> with the <a href="https://huggingface.co/SC117/Nex-N2-mini-APEX-GGUF" target="_blank" style="color: #991b1b; text-decoration: none; font-weight: 700;">unmodified GGUF</a>. Once Nex's upstream patch is merged into stock llama.cpp, these template-fixed GGUFs will be superseded.</p>

<p style="margin: 0;"><b>Use this only if</b> you cannot use Nex's patched llama.cpp and need thinking mode to work on stock builds. Be aware that output quality may differ from the original model.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>💡</span> What is APEX?</div>

<div style="padding: 16px;">

<p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7;">These GGUF files are quantized using <a href="https://github.com/mudler/apex-quant" target="_blank" style="color: #c2410c; text-decoration: none; font-weight: 700;">APEX</a>, a MoE-aware mixed-precision quantization technique that outperforms standard quantization methods while being significantly smaller.</p>

<p style="margin: 0 0 12px 0; font-size: 13px; color: #334155; line-height: 1.7; font-weight: 700;">APEX beats Q8_0 perplexity at half the size — and even beats F16.</p>

<p style="margin: 0; font-size: 13px; color: #334155; line-height: 1.7;">APEX classifies every tensor by its role — routed expert, shared expert, or attention — and applies a layer-wise precision gradient, giving the most sensitive edge layers higher precision and compressing the redundant middle layers more aggressively.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📦</span> Available Files</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 38%;">File</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 12%;">Size</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 12%;">BPW</th><th style="padding: 8px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold; width: 38%;">Note</th></tr></thead><tbody>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini.BF16.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">64.6 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">16.0</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Full precision reference</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini-APEX-Quality.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">21.3 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">5.23</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Highest quality, best accuracy</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini-APEX-Balanced.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">23.6 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">5.85</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Best all-rounder, recommended</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>Nex-N2-mini-APEX-Compact.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">15.4 GB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">3.81</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Best quality/size ratio, 16GB VRAM</td></tr>

<tr><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;"><code>mmproj-Nex-N2-mini.F16.gguf</code></td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">858 MB</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">-</td><td style="padding: 8px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Vision projector (required for image/video)</td></tr>

<tr><td style="padding: 8px 10px; color: #334155;"><code>original-chat-template.jinja</code></td><td style="padding: 8px 10px; color: #334155;">7.9 KB</td><td style="padding: 8px 10px; color: #334155;">-</td><td style="padding: 8px 10px; color: #334155;">Original unmodified template — for reference / use with Nex's patched llama.cpp</td></tr>

</tbody></table>

<p style="margin: 12px 0 0 0; font-size: 12px; color: #991b1b; font-style: italic;">⚠ All GGUF files above (except mmproj) contain a modified <code>chat_template.jinja</code>. See warning above.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧠</span> Model Details</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Architecture</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Qwen3.5 MoE (GatedDeltaNet + Full Attention) + Vision Encoder</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Parameters</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">35B total, 3B active per token</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Experts</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">256 routed experts, 8 active per token</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Layers</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">40 layers (30 linear_attn + 10 full_attn)</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Context</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">262,144 tokens</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155; background: white;">Vision</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563; background: white;">Image support (mmproj 858MB)</td></tr>

<tr><td style="padding: 6px 10px; font-weight: bold; color: #334155; background: white;">Thinking</td><td style="padding: 6px 10px; color: #4b5563; background: white;">Qwen3-style think tags — works on stock llama.cpp via modified template</td></tr>

</tbody></table>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Usage (Stock llama.cpp)</div>

<div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;">

<p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">Text only</p>

<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server \

-m Nex-N2-mini-APEX-Quality.gguf \

-ngl 99 -ncmoe 19 -c 32768 \

--host 0.0.0.0 --port 8081</p>

<p style="margin: 12px 0 8px 0; font-weight: bold; color: #1e293b;">With vision</p>

<p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server \

-m Nex-N2-mini-APEX-Quality.gguf \

--mmproj mmproj-Nex-N2-mini.F16.gguf \

-ngl 99 -ncmoe 19 -c 32768 \

--host 0.0.0.0 --port 8081</p>

<p style="margin: 12px 0 0 0; font-size: 12px; color: #64748b; font-style: italic;">No <code>--chat-template-file</code> needed — the fixed template is embedded in the GGUF. Thinking mode works out of the box. Add <code>--mmproj mmproj-Nex-N2-mini.F16.gguf</code> for vision. Replace <code>Nex-N2-mini-APEX-Quality.gguf</code> with your preferred quantization tier (Quality / Balanced / Compact). Recommended sampling: temperature 0.7, top_p 0.95, top_k 40, min_p 0.</p>

</div>

</div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);">

<div style="background: linear-gradient(135deg, #ff6b35 0%, #f7931e 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📋</span> Original Model Benchmarks</div>

<div style="padding: 16px;">

<table style="width: 100%; border-collapse: collapse; font-size: 13px;"><thead><tr style="background: rgba(255,107,53,0.05);"><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Benchmark</th><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Score</th><th style="padding: 7px 10px; border-bottom: 2px solid #ff6b35; text-align: left; color: #c2410c; font-weight: bold;">Category</th></tr></thead><tbody>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">BrowseComp</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">74.1</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Agent</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">SWE-Bench Verified</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">74.4</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Coding</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Terminal-Bench 2.1</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">60.7</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Coding</td></tr>

<tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">GPQA Diamond</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">82.6</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #334155; background: white;">Reasoning</td></tr>

<tr><td style="padding: 6px 10px; color: #334155;">IFEval</td><td style="padding: 6px 10px; color: #334155;">89.1</td><td style="padding: 6px 10px; color: #334155;">Instruction</td></tr>

</tbody></table>

<p style="margin: 12px 0 0 0; font-size: 12px; color: #64748b; font-style: italic;">From the original Nex-N2-mini model card (BF16, full precision).</p>

</div>

</div>

</div>

Links

• APEX Quantization: https://github.com/mudler/apex-quant
• Original Model: https://huggingface.co/nex-agi/Nex-N2-mini
• Nex's Patched llama.cpp (recommended): https://github.com/nex-agi/llama.cpp/releases/tag/nex-b9596-fix-b9599-9cd1771
• Unmodified GGUF (recommended with Nex's llama.cpp): https://huggingface.co/SC117/Nex-N2-mini-APEX-GGUF
• Original Chat Template: https://huggingface.co/nex-agi/Nex-N2-mini/blob/main/chat_template.jinja
• Fixed Chat Template (used in this GGUF): https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
• Qwen3.5-35B-A3B: https://huggingface.co/Qwen/Qwen3.5-35B-A3B

<!-- updated -->