Model Intelligence Sheet
tarruda/Hy3-GGUF overview
Hy3 GGUF My custom IQ3 XXS GGUF quantization of tencent/Hy3 https://huggingface.co/tencent/Hy3 . Quantization recipe | Recipe | Quant size | BPW | Default type…
Runs locally from ~4.8 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
6 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Hy3-MTP-Q8_0.gguf | GGUF | Q8_0 | 4.70 GB | Download |
| IQ3_XXS/Hy3-IQ3_XXS-00001-of-00004.gguf | GGUF | IQ3_XXS | 4.8 MB | Download |
| IQ3_XXS/Hy3-IQ3_XXS-00002-of-00004.gguf | GGUF | IQ3_XXS | 46.54 GB | Download |
| IQ3_XXS/Hy3-IQ3_XXS-00003-of-00004.gguf | GGUF | IQ3_XXS | 46.48 GB | Download |
| IQ3_XXS/Hy3-IQ3_XXS-00004-of-00004.gguf | GGUF | IQ3_XXS | 15.91 GB | Download |
| imatrix.gguf | GGUF | GGUF | 573.7 MB | Download |
Model Details
| Model ID | tarruda/Hy3-GGUF |
|---|---|
| Author | tarruda |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | tencent/Hy3 |
| Last modified | 2026-07-14T12:44:15.000Z |
Model README
---
quantized_by: tarruda
pipeline_tag: text-generation
base_model: tencent/Hy3
base_model_relation: quantized
license: apache-2.0
---
Hy3 GGUF
My custom IQ3_XXS GGUF quantization of
Quantization recipe
| Recipe | Quant size | BPW | Default type | Tensor-specific overrides |
| --- | ---: | ---: | --- | --- |
| IQ3_XXS | 111550.48 MiB | 3.17 | Q6_K | ffn_down_exps=iq3_xxs, ffn_gate_exps=iq3_xxs, ffn_up_exps=iq3_xxs |
Usage
#!/bin/sh -e
model=./IQ3_XXS/Hy3-IQ3_XXS-00001-of-00004.gguf
ctx=80000
parallel=1
ctx_size=$((ctx * parallel))
llama-server --no-mmap --no-warmup \
--model "$model" --ctx-size "$ctx_size" -np "$parallel" \
-ctk q8_0 -ctv q8_0 \
--temp 0.9 --top-p 1.0 \
--chat-template-kwargs '{"reasoning_effort":"high"}' \
--cache-ram 0 -ctxcp 2 \
--host 0.0.0.0Run tarruda/Hy3-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models