---

license: apache-2.0

base_model: Agnes-AI/Agnes-3.0-Flash

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

• en
• zh

tags:

• gguf
• llama.cpp
• agnes
• quantized
• text-generation
• multimodal
• long-context

---

Agnes-3.0-Flash GGUF

Community GGUF quantizations of Agnes-AI/Agnes-3.0-Flash.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About Agnes-3.0-Flash

Agnes-3.0-Flash is an Apache-2.0 open-weight preview model from Agnes AI. The upstream checkpoint is a 33B-parameter hybrid-attention multimodal model for English and Chinese, with text, image, and video understanding and a 262,144-token context window. See the official model card for the original Transformers and SGLang instructions.

This release converts the native Agnes decoder to the compatible Qwen3.5 GGUF graph. The additional parallel SwiGLU branch is preserved by an algebraic concatenation of its gate/up and down matrices; no weights are trained or fine-tuned. The text GGUF and, when published and validated, the matching F16 vision projector are intended for recent llama.cpp builds with Qwen3.5 hybrid-attention support.

Quantization

Each published quantization is converted from the locked upstream BF16 checkpoint. Q8_0 is produced directly from the source conversion; lower-bit files use the Agnes-specific importance matrix and are quantized independently from the same BF16 GGUF reference. Files are uploaded incrementally after their load and generation smoke tests pass.

Fidelity measurements will be added only after a fixed hold-out comparison against the BF16 reference has completed. Until then, this card makes no quality-retention claim for any quantization.

Quick start

Use a recent llama.cpp build with Qwen3.5 support. Replace the filename with any published quantization in this repository:

./llama-cli \
  -m Agnes-3.0-Flash-Q4_K_M.gguf \
  --jinja \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

Start with a 4K context and one request while checking available memory. The upstream model supports longer contexts, but the practical limit depends on the selected quantization, KV-cache settings, projector, and available hardware.

Reproducibility and validation

• Locked upstream repository: Agnes-AI/Agnes-3.0-Flash
• Locked upstream revision: 24f712ce59379b54c4a141d2708c35daf5ff613b
• Source: upstream BF16 safetensors; no published quantization is used as an input
• Converter: reproducibility/convert_agnes.py
• Runtime and quantizer details: reproducibility/manifest.md
• Checksums: SHA256SUMS.txt

Raw conversion, calibration, quantization, smoke-test, and benchmark logs remain local under reports/ and are not uploaded to this repository.

License and attribution

The upstream checkpoint is released under the Apache License 2.0. These are community GGUF quantizations, not an official Agnes AI release or endorsement.