---

license: other

license_name: nvidia-open-model-license

base_model: TaichuAI/ZDTaichu5.0-9B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: image-text-to-text

language:

• en
• zh

tags:

• gguf
• llama.cpp
• zdtaichu5.0
• quantized
• multimodal
• vision-language-model
• spatial-reasoning
• long-context

---

ZDTaichu5.0-9B GGUF

Community GGUF quantizations of TaichuAI/ZDTaichu5.0-9B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About ZDTaichu5.0-9B

ZDTaichu5.0-9B is a multimodal foundation model for visual understanding,

spatial reasoning, agentic tool use, and embodied-AI workloads. The upstream

model combines a Qwen3.5-9B language backbone with the C-RADIOv4-H vision

encoder and accepts text, one or more images, and video. The upstream card

describes any-resolution visual input and up to 128K tokens of context; the

GGUF metadata preserves the converted model context configuration, while

practical context length depends on the runtime and available memory.

This release is for llama.cpp. The

language model is provided as quantized GGUF files and the visual encoder /

projector is provided separately as mmproj-ZDTaichu5.0-9B-BF16.gguf. Use the

same mmproj file with each language-model quant when passing images or video.

The official upstream model card and Transformers/vLLM instructions are

available here.

Files

| File | Use |

|---|---|

| ZDTaichu5.0-9B-*.gguf | Quantized language-model variants |

| mmproj-ZDTaichu5.0-9B-BF16.gguf | BF16 vision encoder/projector for multimodal input |

| chat_template.jinja | Upstream chat template used by the quickstart |

Fidelity measurements

The table reports next-token fidelity against the converted BF16 language-model

GGUF on the hold-out wiki.valid.raw corpus, using one 2048-token chunk at

context length 2048. The qwen4exp llama-perplexity runtime was used with GPU0

only. Each row is the mean reported for that evaluated chunk; this is a compact

fidelity smoke measurement, not a task or visual-understanding benchmark.

| File | Size (GB) | Mean KLD ↓ | Top-1 vs BF16 ↑ | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|---:|

| ZDTaichu5.0-9B-Q8_0.gguf | 9.53 | 0.000476 | 99.218% | +0.179% | 0.674% |

| ZDTaichu5.0-9B-Q6_K.gguf | 7.36 | 0.002051 | 97.947% | +0.567% | 1.517% |

| ZDTaichu5.0-9B-Q5_K_M.gguf | 6.47 | 0.007224 | 97.165% | +1.552% | 2.831% |

| ZDTaichu5.0-9B-Q5_K_S.gguf | 6.31 | 0.008165 | 97.067% | +1.129% | 2.770% |

| ZDTaichu5.0-9B-Q4_K_M.gguf | 5.63 | 0.019637 | 92.669% | +0.488% | 4.486% |

| ZDTaichu5.0-9B-IQ4_NL.gguf | 5.42 | 0.022016 | 94.233% | +1.512% | 3.968% |

| ZDTaichu5.0-9B-Q4_K_S.gguf | 5.35 | 0.022511 | 92.669% | +1.160% | 4.803% |

| ZDTaichu5.0-9B-IQ4_XS.gguf | 5.20 | 0.024556 | 94.135% | +2.000% | 4.549% |

| ZDTaichu5.0-9B-Q3_K_L.gguf | 4.93 | 0.052946 | 90.714% | +6.806% | 6.753% |

| ZDTaichu5.0-9B-Q3_K_M.gguf | 4.62 | 0.058086 | 89.932% | +7.424% | 7.015% |

| ZDTaichu5.0-9B-IQ3_M.gguf | 4.42 | 0.069806 | 89.247% | +5.129% | 7.907% |

| ZDTaichu5.0-9B-IQ3_S.gguf | 4.37 | 0.069117 | 89.834% | +5.055% | 7.810% |

| ZDTaichu5.0-9B-Q3_K_S.gguf | 4.26 | 0.103701 | 87.488% | +11.029% | 10.096% |

| ZDTaichu5.0-9B-Q2_K.gguf | 3.83 | 0.198499 | 81.623% | +15.439% | 14.454% |

| ZDTaichu5.0-9B-Q2_K_S.gguf | 3.70 | 0.273838 | 79.081% | +27.421% | 16.831% |

| ZDTaichu5.0-9B-IQ2_S.gguf | 3.43 | 0.308748 | 78.104% | +26.755% | 19.314% |

| ZDTaichu5.0-9B-IQ2_XS.gguf | 3.29 | 0.338933 | 78.104% | +26.174% | 19.820% |

| ZDTaichu5.0-9B-IQ1_M.gguf | 2.88 | 0.950750 | 62.170% | +123.781% | 31.852% |

| ZDTaichu5.0-9B-Q1_0.gguf | 1.97 | 9.816482 | 0.098% | +1316447.597% | 65.432% |

Lower Mean KLD, ΔPPL, and RMS Δp, and higher Top-1 agreement, indicate closer

next-token behavior to BF16. The lower tiers are included for constrained

hardware and should be treated as experimental; IQ1_M and especially Q1_0 are

not recommended when capability retention matters. Task-specific and

multimodal behavior can differ from this text-only measurement. The compact

machine-readable source is

reproducibility/quality-summary.tsv,

with commands and hashes in reproducibility/manifest.md.

Quick start

Text-only smoke test (the measured sweet-spot candidate):

./llama-cli \
  -m ZDTaichu5.0-9B-IQ4_NL.gguf \
  --no-mmproj \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Multimodal input uses the BF16 projector alongside the language model:

./llama-cli \
  -m ZDTaichu5.0-9B-Q4_K_M.gguf \
  --mmproj mmproj-ZDTaichu5.0-9B-BF16.gguf \
  --image image.jpg \
  --chat-template-file chat_template.jinja \
  --jinja \
  --reasoning off \
  -p 'Describe the visible objects and their spatial relationships in this image.' \
  -n 128 -c 4096 -ngl 99

Adjust -ngl, -c, batch sizes, and projector offload options for the target

hardware. Image/video support depends on a llama.cpp build with the mtmd

runtime enabled.

Reproducibility and validation

The conversion source is locked to upstream revision

a22afd15a3f85659f103caa659ec4aa9500a998e. Quantization was performed

directly from the converted BF16 language-model GGUF, never from another

quantized file. The model-specific imatrix combines WikiText-2 train text and

the included English/Chinese spatial, JSON/tool, code, and agent calibration

text. wiki.valid.raw and wiki.test.raw were held out from imatrix creation.

All nineteen text variants passed deterministic load/generate smoke tests. Q4_K_M

also passed a multimodal smoke with the BF16 mmproj and an image; the runtime

reported text, vision, and video modalities and generated a grounded

description.

The compact commands, hashes, calibration split information, converter/runtime

commits, and validation profile are in

reproducibility/manifest.md. Artifact

checksums are in SHA256SUMS.txt. Raw conversion,

quantization, smoke-test, and evaluation logs are kept locally under

reports/ and are not part of this public package.

License and attribution

The upstream weights are distributed under the [NVIDIA Open Model License

Agreement](https://huggingface.co/TaichuAI/ZDTaichu5.0-9B/blob/main/LICENSE).

The Qwen3.5 language backbone and C-RADIOv4-H components have additional

notices; see the included NOTICE,

THIRD_PARTY_LICENSES.md, and LICENSE.

Redistribution should retain those files and their attribution requirements.

These are community GGUF quantizations, not an official ZDTaichu5.0-9B

release or endorsement.