---

license: apache-2.0

base_model: tencent/Hy3

tags:

• hy3
• hunyuan
• hy_v3
• gguf
• mixture-of-experts
• moe

language:

• en
• zh

library_name: hy3

pipeline_tag: text-generation

---

hy3-gguf — GGUF weights for Tencent Hy3 (tencent/Hy3)

GGUF-format weights for tencent/Hy3 (HYV3ForCausalLM, model_type: hy_v3), a

295B-parameter / 21B-active-parameter Mixture-of-Experts model from

Tencent's Hunyuan ("Hy") team (tencent/Hy3).

These files were produced by the hy3

converter (hy3-convert) and are meant to be run with the **hy3 inference

engine**, a from-scratch C/Metal/CUDA implementation.

> ## ⚠️ This GGUF does NOT work with llama.cpp

>

> Despite the .gguf extension, these files are **only usable by the

> hy3 engine**. llama.cpp,

> ollama, LM Studio, text-generation-webui, koboldcpp, and any other

> llama.cpp-based tool cannot load these files. Three independent reasons:

>

> 1. Unknown architecture. The metadata declares

> general.architecture = "hy_v3". llama.cpp only knows hunyuan-moe,

> hunyuan-dense, hunyuan_vl — loading aborts with

> unknown model architecture: 'hy_v3'.

> 2. Custom metadata keys. All hyperparameters use the hy_v3.* prefix

> (hy_v3.block_count, hy_v3.expert_count, …), which llama.cpp does not

> look up.

> 3. Non-fused expert tensors. Experts are stored one tensor per expert

> (blk.N.ffn_gate_exps.0.gate_proj.weight, …1…, … — 46080 tensors),

> whereas llama.cpp expects experts fused into a single stacked 3D tensor per

> layer. This is a fundamentally different on-disk layout.

>

> This is a custom GGUF readable only by the hy3 loader. Do not open

> issues against llama.cpp for these files.

How to run

Use the hy3 engine: <https://github.com/yuhai-china/hy3>

git clone https://github.com/yuhai-china/hy3
cd hy3
make            # macOS builds the Metal backend automatically

# download a GGUF from this repo, then:
./run_metal.sh -m /path/to/hy3_q4k_mixed.gguf -p "The capital of France is" -experts 8

> Testing scope: the hy3 engine's performance work and benchmarks were

> developed and verified only on macOS / Apple Silicon (Metal backend),

> measured on an M2 Ultra (~20–27 tok/s decode depending on -experts). The

> CPU and CUDA backends exist in the source but were not exercised as part of

> that work — treat them as untested.

Files / quantization

The mixed-precision GGUF follows this scheme (see hy3_convert.c):

| Tensor group | Type |

|---|---|

| Routed experts (ffn_{gate,up,down}_exps) — the bulk of the model | Q4_K |

| Attention q/k/v/o projections, shared-expert & dense FFN, output.weight | Q8_0 |

| Norms, router (ffn_gate_inp), biases | F32 |

| token_embd.weight | F16 |

Model facts

| | |

|---|---|

| Architecture | HYV3ForCausalLM (hy_v3) |

| Layers | 80 (layer 0 dense, layers 1–79 MoE) |

| Hidden size | 4096 |

| Attention | 64 heads, GQA with 8 KV heads, head_dim 128 |

| Experts | 192 routed (top-8 activated) + 1 shared (always active) |

| Expert intermediate size | 1536 |

| Dense (layer 0) intermediate size | 13312 |

| Vocab size | 120832 (120818 real tokens + padding) |

| RoPE | theta 11158840, rotate_half pairing |

| QK norm | per-head RMSNorm on Q and K, before RoPE |

| MoE routing | sigmoid(router_logits); top-8 by sigmoid + expert_bias, combined using unbiased sigmoid weights, renormalized to sum 1, scaled by router_scaling_factor = 2.826 |

The engine supports a runtime top-k experts override (-experts 1..8) to

trade quality for speed. On a small 13-question code/reasoning eval (greedy,

no-think): experts=8 → 10/13, experts=4 → 7/13. Default is 8.

Chat template

Hy3 is instruction-tuned and expects the Hunyuan V3 chat format (the hy3

engine applies it automatically; use --raw to bypass). Single user turn,

no-think:

<｜hy_begin_of_sentence:opensource｜><｜reasoning_mode:opensource｜>reasoning_effort:no_think<｜hy_User:opensource｜>{prompt}<｜hy_Assistant:opensource｜><think:opensource></think:opensource>

Generation stops on <｜hy_eos:opensource｜> (120025),

<｜hy_endofsentence｜> (120001), or <｜hy_EOT｜> (120008).

License & attribution

Weights derive from tencent/Hy3; refer

to the upstream repository for the governing model license. This is an

unofficial community conversion, not affiliated with or endorsed by Tencent.