SC117/MiniCPM5-2B-abliterated-FIT-GGUF overview

---

base_model: openbmb/MiniCPM5-2B

base_model_relation: quantized

license: apache-2.0

library_name: gguf

pipeline_tag: text-generation

language:

• en
• zh

tags:

• gguf
• llama-cpp
• quantization
• minicpm
• minicpm5
• fit-gguf
• uncensored
• abliterated
• on-device

---

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; margin-bottom: 24px;">

<div style="background: #0f172a; border-radius: 20px; padding: 40px 32px 30px 32px; text-align: center; position: relative; overflow: hidden;">

<div style="position: absolute; top: -40px; right: -40px; width: 160px; height: 160px; background: rgba(139,92,246,0.18); border-radius: 50%;"></div>

<div style="position: absolute; bottom: -50px; left: -30px; width: 140px; height: 140px; background: rgba(59,130,246,0.15); border-radius: 50%;"></div>

<div style="position: absolute; top: 30%; right: 12%; width: 56px; height: 56px; background: rgba(59,130,246,0.22); border-radius: 50%;"></div>

<div style="display: inline-flex; flex-wrap: wrap; justify-content: center; gap: 8px; margin-bottom: 18px; position: relative; z-index: 1;"><span style="background: #3b82f6; color: white; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">FIT-GGUF v0.3.3</span><span style="background: #8b5cf6; color: white; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">GATE-VERIFIED TIERS</span><span style="background: #10b981; color: white; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">4 TIERS · 1.14–1.46 GiB</span><span style="background: #ef4444; color: white; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">ABLITERATED (TRIAL #68)</span><span style="background: #f59e0b; color: #0f172a; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">MEASURED KL + SAME-TOP</span><span style="background: #1e293b; color: #94a3b8; font-size: 11px; font-weight: 700; padding: 5px 14px; border-radius: 20px;">APACHE-2.0</span></div>

<h1 style="margin: 0 0 10px 0; font-size: 34px; font-weight: 800; color: #f8fafc; letter-spacing: -0.5px; border: none; position: relative; z-index: 1;">MiniCPM5-2B-abliterated · FIT-GGUF</h1>

<p style="margin: 0 0 20px 0; font-size: 15px; color: #cbd5e1; position: relative; z-index: 1;">Four fidelity tiers of a 128K-context 2.5B on-device model — every shipped file re-verified against its own BF16, byte-exact and reproducible.</p>

<div style="position: relative; z-index: 1; margin: 0 auto; width: 72%; max-width: 540px;">

<div style="height: 8px; border-radius: 8px; background: #334155; position: relative; overflow: visible;">

<div style="height: 8px; width: 62%; border-radius: 8px; background: linear-gradient(90deg, #3b82f6, #8b5cf6);"></div>

<div style="position: absolute; top: 50%; left: 62%; transform: translate(-50%, -50%); width: 18px; height: 18px; border-radius: 50%; background: #ffffff; box-shadow: 0 0 0 4px rgba(139,92,246,0.35);"></div>

</div>

<div style="display: flex; justify-content: space-between; margin-top: 8px; font-size: 10px; color: #94a3b8; font-weight: 600;"><span>1.14 GiB</span><span style="color: #c4b5fd;">verified minimum at each fidelity</span><span>1.46 GiB</span></div>

</div>

<p style="margin: 18px 0 0 0; font-size: 13px; position: relative; z-index: 1;"><span style="color: #94a3b8;">English</span><span style="color: #475569;"> · </span><a href="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/blob/main/README.zh-CN.md" style="color: #93c5fd; text-decoration: none; font-weight: 600;">简体中文 📖</a></p>

</div>

</div>

<p align="center"><img src="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/resolve/main/assets/fit-gguf-banner.png" alt="FIT-GGUF" width="760"></p>

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; display: flex; flex-direction: column; gap: 18px; margin-bottom: 30px;">

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧭</span> About FIT-GGUF — the tool behind these files</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 10px 0;">Every file in this repository was planned, executed and verified by <a href="https://github.com/Scorp1o117/FIT-GGUF" style="color: #1d4ed8; text-decoration: none; font-weight: 700;">FIT-GGUF</a>, an open-source, deterministic tensor-level planning layer on top of llama.cpp quantization. Standard GGUF quantization asks you to pick one of a handful of presets; FIT-GGUF instead asks <i>what quality do you want</i>, then finds and <b>verifies</b> the smallest GGUF that demonstrably meets it.</p><p style="margin: 0 0 12px 0; padding: 10px 14px; background: #f5f3ff; border-left: 4px solid #7c3aed; border-radius: 6px; color: #4c1d95; font-weight: 600;">Traditional GGUF gives you presets. FIT gives you a fidelity contract: macro KL ≤ tier anchor ∧ same-top ≥ model-calibrated floor.</p><table style="width: 100%; border-collapse: collapse; font-size: 13px;"><tbody><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155;">Deterministic size prediction &amp; byte-exact delivery</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #166534; font-weight: 700;">✅ Validated (G2 gate, delta = 0)</td></tr><tr><td style="padding: 6px 10px; font-weight: bold; color: #334155;">Universally optimal tensor allocation</td><td style="padding: 6px 10px; color: #92400e; font-weight: 700;">⚠️ Not established — FIT claims verified fidelity contracts, not a universal quality optimum</td></tr></tbody></table><p style="margin: 12px 0 0 0;">Method, pre-registered research record and the <code>fit</code> CLI are all open source: <a href="https://github.com/Scorp1o117/FIT-GGUF" style="color: #1d4ed8; text-decoration: none; font-weight: 700;">github.com/Scorp1o117/FIT-GGUF</a></p></div></div>

<div style="border: 1px solid #fca5a5; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #dc2626 0%, #ea580c 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>⚠️</span> Safety notice / 安全提示</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 8px 0;">The source model is an <b>abliterated, refusal-removed</b> model with no meaningful built-in guardrails, and may comply with harmful, illegal or unsafe requests. Use it only where you can provide appropriate moderation, access control and legal review. Do not deploy it to end users without your own safety layer.</p><p style="margin: 0; color: #64748b;">源模型经过拒答方向消融，不具备可靠的内置安全护栏；请仅在合法、受控、具备审核与滥用防护的环境中使用，使用者自行承担部署责任。</p></div></div>

<div style="border: 1px solid #c4b5fd; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #7c3aed 0%, #a855f7 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧬</span> Abliteration — trial #68, documented</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 10px 0;">The refusal-direction ablation was performed locally with the <b>abliterix</b> pipeline: <b>mean-LoRA steering</b> — rank-3 full-norm LoRA adapters built from per-layer mean refusal directions, with <b>projected abliteration</b>, a <b>linear decay kernel</b> over depth, winsorized direction vectors (q = 0.995), and per-layer vector scope. Only <code>attn.o_proj</code> and <code>mlp.down_proj</code> are steered; <code>attn.q/k/v</code> are disabled. The exported weights are Optuna trial <b>#68</b> of an 80-trial study (<code>checkpoints_minicpm5_2b_full</code>), selected from the measured Pareto front.</p><table style="width: 100%; border-collapse: collapse; font-size: 12.5px;"><thead><tr style="background: rgba(124,58,237,0.05);"><th style="padding: 7px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #6d28d9;">Metric</th><th style="padding: 7px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #6d28d9;">Trial #68</th></tr></thead><tbody><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 600;">Refusals (100 held-out harmful prompts)</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 700; color: #166534;">7</td></tr><tr><td style="padding: 6px 10px; font-weight: 600;">3-token full-distribution KL vs original (nats/token)</td><td style="padding: 6px 10px; font-weight: 700;">0.0628</td></tr><tr><td style="padding: 6px 10px; font-weight: 600;">Validation KL, 100 held-out prompts (mean / p95)</td><td style="padding: 6px 10px;">0.0633 / 0.1959</td></tr><tr><td style="padding: 6px 10px; font-weight: 600;">Top-1 disagreement rate</td><td style="padding: 6px 10px;">3.0%</td></tr><tr><td style="padding: 6px 10px; font-weight: 600;">Generation health (repetition / collapse check)</td><td style="padding: 6px 10px; color: #166534; font-weight: 700;">PASSED</td></tr></tbody></table><p style="margin: 10px 0 0 0; padding: 9px 13px; background: #fffbeb; border-left: 4px solid #f59e0b; border-radius: 6px; color: #92400e;"><b>Why #68.</b> It is the lowest-perturbation member of the best refusal bucket: every trial that reaches ≤7 refusals costs at least KL 0.0628, and the two trials that reach 6 refusals sit at KL 0.227 and 0.255 — 3.6× and 4× the damage. #68 is the front's knee, not a hand-picked favourite.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📦</span> Pick a tier</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 10px 0; padding: 9px 13px; background: #fffbeb; border-left: 4px solid #f59e0b; border-radius: 6px; color: #92400e;"><b>File size ≠ RAM/VRAM usage.</b> KV cache, compute buffers and runtime overhead are separate — at the full 128K context a 2.5B model still needs several GiB of KV. Pick a sane <code>-c</code>. Naming: <code>minicpm5-2b-abliterated-FIT-&lt;tier&gt;-&lt;size&gt;GiB-&lt;type&gt;.gguf</code>, where <code>&lt;type&gt;</code> is always a <b>nameable GGUF preset</b> — the native preset when the recipe overrides no tensor, the element-weighted dominant type when that is itself a preset name, and the recipe's base preset otherwise. The three bare tensor types (<code>Q4_K</code>, <code>Q3_K</code>, <code>Q5_K</code>) are deliberately never used as suffixes: llama.cpp only ships the <code>_S</code>/<code>_M</code>/<code>_L</code> variants, and a filename with no recognisable quantisation token is dropped from Hugging Face's own variant panel.</p><table style="width: 100%; border-collapse: collapse; font-size: 12.5px;"><thead><tr style="background: rgba(37,99,235,0.05);"><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Tier</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">GiB</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Recipe</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Macro KL ↓</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Same-top ↑</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Gates (KL ≤ / top ≥)</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Measured positioning</th></tr></thead><tbody><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 700; color: #1d4ed8;">QUALITY</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">1.459</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">FIT recipe on <b>Q4_K_M</b> (67 tensor overrides)</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 700;">0.0493</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">89.90%</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #7c3aed;">0.05 / 89.52%</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">At Q4_K_M's size, slightly cleaner than it (0.0526)</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">BALANCED</td><td style="padding: 6px 10px;">1.283</td><td style="padding: 6px 10px;">FIT recipe, dominant <b>IQ4_XS</b> (174 tensor overrides over Q3_K_M)</td><td style="padding: 6px 10px; font-weight: 700;">0.0929</td><td style="padding: 6px 10px;">86.27%</td><td style="padding: 6px 10px; color: #7c3aed;">0.10 / 85.48%</td><td style="padding: 6px 10px;">Fills the Q3_K_M (0.163) → IQ4_XS (0.068) gap</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">COMPACT</td><td style="padding: 6px 10px;">1.212</td><td style="padding: 6px 10px;">FIT recipe on <b>Q3_K_M</b> (94 tensor overrides)</td><td style="padding: 6px 10px; font-weight: 700;">0.1495</td><td style="padding: 6px 10px;">83.02%</td><td style="padding: 6px 10px; color: #7c3aed;">0.15 / 82.42%</td><td style="padding: 6px 10px;">Same size class as Q3_K_M, markedly cleaner</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">MINI</td><td style="padding: 6px 10px;">1.142</td><td style="padding: 6px 10px;">native <b>IQ3_M</b> preset (no tensor overrides)</td><td style="padding: 6px 10px; font-weight: 700;">0.1855</td><td style="padding: 6px 10px;">81.13%</td><td style="padding: 6px 10px; color: #7c3aed;">0.20 / 79.55%</td><td style="padding: 6px 10px;">Native presets win at this size; MINI is the verified one</td></tr></tbody></table><p style="margin: 12px 0 8px 0;"><b>The four tiers are all KL-bound</b> — KL, not the Same-top floor, is what separates PASS from FAIL on this model, so the recorded margins are the KL ones above.</p><table style="width: 100%; border-collapse: collapse; font-size: 12.5px;"><thead><tr style="background: rgba(37,99,235,0.05);"><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Also shipped</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">GiB</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Macro KL ↓</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Same-top ↑</th><th style="padding: 7px 10px; border-bottom: 2px solid #2563eb; text-align: left; color: #1d4ed8;">Why it is here</th></tr></thead><tbody><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 700; color: #1d4ed8;">Q8_0</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">2.496</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: 700;">0.0013</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">98.25%</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15);">Highest fidelity that is still a quantized file</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">Q6_K</td><td style="padding: 6px 10px;">1.928</td><td style="padding: 6px 10px; font-weight: 700;">0.0079</td><td style="padding: 6px 10px;">95.54%</td><td style="padding: 6px 10px;">Best quality-per-byte on this model's ladder</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">BF16</td><td style="padding: 6px 10px;">4.693</td><td style="padding: 6px 10px; font-weight: 700;">0</td><td style="padding: 6px 10px;">100%</td><td style="padding: 6px 10px;">The reference every KL / Same-top number here is measured against</td></tr><tr><td style="padding: 6px 10px; font-weight: 700; color: #1d4ed8;">imatrix</td><td style="padding: 6px 10px; color: #64748b;">0.003</td><td style="padding: 6px 10px; color: #64748b;">—</td><td style="padding: 6px 10px; color: #64748b;">—</td><td style="padding: 6px 10px;">500×512 chunks of <code>APEX-imatrix-Small.txt</code>, so you can re-quantize this model yourself</td></tr></tbody></table><p style="margin: 10px 0 0 0; padding: 9px 13px; background: #f0f9ff; border-left: 4px solid #2563eb; border-radius: 6px; color: #1e3a8a;"><b>Read the curve, not just the tiers.</b> On a 2.5B model the fidelity tiers are deliberately capped at KL 0.05–0.20, so the two big native presets are <i>cleaner</i> than every tier — Q6_K at 1.93 GiB is 6× cleaner than QUALITY at 1.46 GiB. If you want maximum fidelity rather than minimum size, take Q8_0 or Q6_K.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📈</span> Measured quality</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><img src="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/resolve/main/results/kl-curve-en.png" alt="Macro KL against main GGUF size, with every native preset and the four shipped tiers labelled" style="width: 100%; border-radius: 8px; border: 1px solid #e2e8f0;"><p style="margin: 12px 0 8px 0;">Quality improves monotonically across the healthy preset ladder. The 2-bit region is reported as measured: <b>IQ2_XXS / IQ2_XS / IQ2_M / IQ3_XXS collapse</b> (macro KL 3.92 / 1.93 / 0.81 / 0.40) and are excluded by design. COMPACT and BALANCED fill the Q3_K_M → IQ4_XS gap where no native preset exists; MINI <i>is</i> the native IQ3_M preset, which is why its diamond and its tier label sit on the same point.</p><img src="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/resolve/main/results/sametop-curve-en.png" alt="Macro Same-top against main GGUF size" style="width: 100%; border-radius: 8px; border: 1px solid #e2e8f0; margin-top: 8px;"><p style="margin: 12px 0 0 0;">Same-top is the share of positions where the quantized model and the aligned BF16 reference pick the same highest-logit token. On this model it tracks KL closely and never becomes the binding gate in the tier range.</p><p style="margin: 10px 0 8px 0; padding: 9px 13px; background: #fffbeb; border-left: 4px solid #f59e0b; border-radius: 6px; color: #92400e; font-size: 12.5px;"><b>Measurement basis, stated plainly.</b> These charts plot the numbers recorded by the calibration and delivery run. In that run the 12-preset ladder was evaluated on <b>CPU</b>, while the four shipped tiers were verified on <b>CUDA</b> — the calibration was started before a CUDA-runtime-loading fix and the tier searches ran after it, and <code>fit calibrate</code> resumes by skipping already-recorded presets, so both measurement eras ended up in one bundle. Re-measuring the shipped files on CUDA reproduces their verdicts with modestly different numbers (macro KL 0.0493 / <b>0.0918</b> / <b>0.1484</b> / <b>0.1831</b> for QUALITY / BALANCED / COMPACT / MINI). Every one of the 20 per-domain pairs moved the same way — the CPU-basis figures are slightly <i>conservative</i> (higher KL); the largest difference is MINI · Chinese, 0.1723 → 0.1689 (−2.0%), and no domain got worse. <b>All four tiers PASS either way.</b> Within one backend the evaluation is bit-reproducible: a fresh CUDA re-run returned the delivery values to the last digit.</p><p style="margin: 0; font-size: 12px;"><a href="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/resolve/main/results/kl-curve-en.png" style="color: #1d4ed8; text-decoration: none;">Full-size EN</a> · <a href="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/resolve/main/results/kl-curve-zh.png" style="color: #1d4ed8; text-decoration: none;">中文大图</a> · <a href="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/blob/main/results/four-tier-results.json" style="color: #1d4ed8; text-decoration: none;">raw results JSON</a></p></div></div>

<div style="border: 1px solid #fca5a5; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #dc2626 0%, #ea580c 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🚀</span> Run it</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 10px 0; padding: 9px 13px; background: #f0fdf4; border-left: 4px solid #16a34a; border-radius: 6px; color: #14532d;"><b>Good news: this is plain <code>llama</code> architecture.</b> Unlike models that need a patched llama.cpp, these files load in any reasonably recent upstream build — no PR branch, no custom fork. The measurements here used <b>b10690</b>.</p><p style="margin: 0 0 8px 0; font-weight: bold; color: #1e293b;">llama.cpp</p><p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">./llama-server \

-m minicpm5-2b-abliterated-FIT-BALANCED-1.28GiB-IQ4_XS.gguf \

-ngl 99 -c 8192

./llama-cli \

-m minicpm5-2b-abliterated-FIT-QUALITY-1.46GiB-Q4_K_M.gguf \

-ngl 99 -c 4096 -p "Hello" -n 256</p><p style="margin: 10px 0 0 0;">The chat template is embedded in every file — including the tool-calling and reasoning-content handling — so LM Studio, KoboldCpp, Jan and friends load them without extra configuration. This is a dense 2.5B text model: no vision projector, no MTP head, no MoE. The full 131,072-token context is available, but KV cache grows with it; start at <code>-c 8192</code> and work up.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🔬</span> Evaluation protocol &amp; honest scope</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><table style="width: 100%; border-collapse: collapse; font-size: 12.5px;"><tbody><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155;">Runtime</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563;">llama.cpp b10690 · Windows x86_64 · CUDA 13.3 (RTX 5090-class)</td></tr><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155;">Command shape</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563;"><code>llama-perplexity -ngl 99 -t 16 -c 512 -b 512 --kl-divergence …</code></td></tr><tr><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); font-weight: bold; color: #334155;">Reference</td><td style="padding: 6px 10px; box-shadow: 0 1px 0 0 rgba(128,128,128,0.15); color: #4b5563;">This model's own BF16 logits (the BF16 GGUF shipped here)</td></tr><tr><td style="padding: 6px 10px; font-weight: bold; color: #334155;">Domains</td><td style="padding: 6px 10px; color: #4b5563;">wiki_test · wiki_valid · Chinese · code · agent_chat (five fixed 64 KiB slices, macro mean)</td></tr><tr><td style="padding: 6px 10px; font-weight: bold; color: #334155;">Calibration</td><td style="padding: 6px 10px; color: #4b5563;">12-point standard preset ladder + 10 gap probes; guard profile <code>guard-minicpm5-2b-abliterated-exact-v1</code>, scope <code>exact_model</code>, floors by <code>empirical_p5</code> (n = 3)</td></tr><tr><td style="padding: 6px 10px; font-weight: bold; color: #334155;">Weights binding</td><td style="padding: 6px 10px; color: #4b5563;"><code>source_weights_sha256 = f161afcf…c6e12</code> — the registry entry is keyed to these exact weights</td></tr></tbody></table><p style="margin: 10px 0 0 0;"><b>Tier verification:</b> a tier ships only if macro KL ≤ its anchor <b>and</b> Same-top ≥ a model-calibrated guard floor. Floors are <code>exact_model</code>-scoped and are deliberately <i>not</i> transferable to other models. Each shipped file was re-evaluated on its own bytes before delivery (G2 delta +0 on all four — the delivered byte count equals the re-finalized prediction exactly).</p><p style="margin: 8px 0 0 0;"><b>Allocator scope:</b> the stock <code>balanced</code> v0.3 policy with the APEX-imatrix-Small importance matrix; no model-specific refine profile, because this model did not get one. What is claimed is deterministic size planning plus measured, double-gate verification of these specific artifacts — not a universally optimal allocation.</p><p style="margin: 8px 0 0 0;"><b>Sizes in the file names are measured bytes, not budgets.</b> The tool refuses to emit an artifact whose size it cannot predict to the byte.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🧩</span> Included — and not included</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 6px 0;">✅ 4 gate-verified tier GGUFs · ✅ <b>BF16 reference GGUF</b> (the exact weights every measurement above is taken against) · ✅ <b>Q6_K and Q8_0</b> native presets · ✅ the <b>importance matrix</b> used for quantization · ✅ chat template embedded in every file · ✅ checksums (<code>SHA256SUMS.txt</code>) · ✅ labelled quality curves (<code>results/</code>) · ✅ tier manifest (<code>FIT-TIERS.md</code>) · ✅ the exact numbers as JSON (<code>results/four-tier-results.json</code>)</p><p style="margin: 0 0 6px 0;">❌ 2-bit quantization classes — measured collapse on this model (macro KL 0.40–3.92), excluded by design · ❌ no vision projector / no MTP head (dense text model) · ❌ no GGUF above Q8_0; take the BF16 file if you need the exact weights</p><p style="margin: 0;">The abliteration (trial #68) was performed locally from <a href="https://huggingface.co/openbmb/MiniCPM5-2B" style="color: #1d4ed8; text-decoration: none; font-weight: 600;">openbmb/MiniCPM5-2B</a>; this repository contributes the abliterated weights, the FIT quantization plans and the artifacts.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>🔍</span> Verify &amp; reproduce</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0; font-family: monospace; background: #f8fafc; padding: 10px 14px; border-radius: 6px; border: 1px solid #e2e8f0; font-size: 12px; color: #1e293b; white-space: pre-wrap;">sha256sum -c SHA256SUMS.txt</p><p style="margin: 10px 0 0 0;">The evaluation slices are public in the <a href="https://github.com/Scorp1o117/FIT-GGUF" style="color: #1d4ed8; text-decoration: none; font-weight: 700;">FIT-GGUF repository</a>; the calibration ladder, gap probes, search audit and guard profile for this release are retained in the FIT-GGUF experiment record <a href="https://github.com/Scorp1o117/FIT-GGUF/tree/main/experiments/2026-09-10-minicpm5-2b-4tier" style="color: #1d4ed8; text-decoration: none; font-weight: 700;"><code>experiments/2026-09-10-minicpm5-2b-4tier/</code></a>. The five-domain <code>.kld</code> reference logits are not stored there (9.6 GiB) — they regenerate from the BF16 GGUF shipped here and are checked against <code>reference-manifest.json</code>, which pins each domain's hash and byte count. Because the importance-matrix path is embedded in the GGUF metadata, the <i>byte count</i> of a re-quantization depends on where you put the imatrix file — the weights do not. Exact-size behaviour is scoped to the recorded source metadata and the recorded llama.cpp build; changing the converter, runtime, source layout or metadata requires revalidation.</p></div></div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; background: #ffffff; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"><div style="background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); padding: 12px 16px; color: white; font-weight: 700; font-size: 14px; display: flex; align-items: center; gap: 8px;"><span>📄</span> License &amp; credits</div><div style="padding: 16px; font-size: 13px; color: #334155; line-height: 1.7;"><p style="margin: 0 0 8px 0;">Apache-2.0, inherited from the base model — follow the upstream license and model-card requirements.</p><p style="margin: 0;"><a href="https://huggingface.co/openbmb" style="color: #1d4ed8; text-decoration: none; font-weight: 600;">OpenBMB</a> — the MiniCPM5 family · <a href="https://github.com/ggml-org/llama.cpp" style="color: #1d4ed8; text-decoration: none; font-weight: 600;">llama.cpp</a> — quantization and the KL/perplexity evaluator · <b>abliterix</b> — the local ablation pipeline (trial #68) · <a href="https://github.com/Scorp1o117/FIT-GGUF" style="color: #1d4ed8; text-decoration: none; font-weight: 600;">FIT-GGUF</a> — verified size-exact quantization. FIT-GGUF is an independent project, not affiliated with OpenBMB or llama.cpp.</p></div></div>

</div>

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; text-align: center; margin-bottom: 30px;"><a href="https://github.com/Scorp1o117/FIT-GGUF" style="display: inline-block; background: linear-gradient(135deg, #2563eb 0%, #7c3aed 100%); color: white; font-weight: 700; font-size: 14px; padding: 12px 28px; border-radius: 24px; text-decoration: none;">⭐ FIT-GGUF on GitHub — the tool, the method, the full research record</a><p style="margin: 14px 0 0 0; font-size: 13px;"><span style="color: #86868b;">English</span><span style="color: #cbd5e1;"> · </span><a href="https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/blob/main/README.zh-CN.md" style="color: #1d4ed8; text-decoration: none; font-weight: 600;">简体中文</a></p></div>

<!-- SC117_SUPPORT_START -->

<div align="center">

<a href="https://ko-fi.com/sc117">

<img

src="https://huggingface.co/SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/resolve/main/assets/support-sc117.png"

alt="Support SC117 on Ko-fi — support is optional and all releases remain freely available."

width="100%"

/>

</a>

</div>

<!-- SC117_SUPPORT_END -->