---

license: apache-2.0

base_model: Jiunsong/SuperHY3-abliterated-NVFP4

base_model_relation: quantized

library_name: gguf

tags:

- gguf

- hy_v3

- moe

- abliterated

- uncensored

- mtp

---

SuperHY3-abliterated-GGUF

GGUF conversions of Jiunsong/SuperHY3-abliterated-NVFP4

(295B-A32B MoE, hy_v3 architecture, 80 layers + MTP layer, 192 experts).

⚠️ Provenance: dequantized from NVFP4 — read this first

No BF16 source of this abliteration exists anywhere. The upstream model is

published ONLY as an NVFP4 (compressed-tensors, FP4-E2M1 group-16) checkpoint.

These GGUFs were produced by exactly dequantizing that NVFP4 checkpoint to

BF16, then converting/quantizing as normal:

• The dequantization step is lossless with respect to the NVFP4 checkpoint

(bit-exact against the reference compressed-tensors implementation;

round-trip verified on the actual shards).

• BUT the expert FFN weights (the bulk of the parameters) carry the **FP4

ceiling (~4.25 bpw effective information)** of the source into every tier

below. Attention, dense layers, embeddings, and router weights were never

quantized upstream and are true BF16.

• Practical consequence: tiers up to ~Q4 lose essentially nothing vs a

hypothetical BF16 source. Q5/Q6/Q8/BF16 buy fidelity only on the

attention/dense tensors. The BF16 tier is provided as a conversion-faithful

archival source, not because it contains BF16-grade expert weights.

Dequantization tool: llm-dequant

(streaming NVFP4 → BF16 safetensors, byte-exact round-trip verification).

Requirements

hy_v3 support is not yet in llama.cpp master. Build from

PR #25395

(this repo was produced at commit cecbf5fb0):

git fetch origin pull/25395/head:hy3 && git checkout hy3
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release && cmake --build build -j

Files

| Tier | Size | BPW | Notes |

|---|---|---|---|

| BF16 | 597.7 GB | 16 | conversion-faithful source (expert weights carry FP4 ceiling — see above) |

Quantized tiers (IQ1_S … Q8_0, imatrix) are being produced with the same

verified pipeline and will appear here as they upload.

MTP / speculative decoding

The MTP layer (blk.80, eh_proj/enorm) is present in the source and bundled

in these GGUFs. PR #25395 supports --spec-type draft-mtp. On the same

architecture (Hy3 IQ2_M, one GB10) we measured +27% decode speed with:

--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75 --parallel 1

--spec-draft-p-min 0.75 matters: the head is trained single-depth and the

p_min=0 default makes speculation a net loss. MTP acceptance on this

fine-tune has not been separately measured yet; numbers will be added with the

quant tiers.

Known quirks (inherited from the hy_v3 family)

• Native OpenAI-style tools API fails on llama-server (unsupported

<tool_calls:opensource> markup) — use prompt-injected tools; the model

tool-calls well, the server-side parser is what's missing.

• EOG metadata warning at load (special_eos_id is not in special_eog_ids)

— if you see looping, add an explicit stop on the EOS token.

• Chat template requires --jinja.

Provenance chain

tencent/Hy3 (BF16)
  └─ Jiunsong abliteration + fine-tune, released as NVFP4 only
       └─ llm-dequant: exact NVFP4 → BF16 dequantization
            └─ convert_hf_to_gguf.py (llama.cpp PR #25395 @ cecbf5fb0) → BF16 GGUF
                 └─ llama-quantize (+imatrix) → quant tiers