---

base_model: Agnes-AI/Agnes-3.0-Flash

license: apache-2.0

library_name: gguf

tags:

- gguf

- agnes

- mtp

---

Agnes 3.0 Flash GGUF

> [!IMPORTANT]

> These GGUFs do not load on stock llama.cpp yet. Agnes runs a second, narrower

> SwiGLU in parallel with the main FFN on every layer, and upstream has no support for

> it, so a stock build stops with:

>

> ```

> error loading model: done_getting_tensors: wrong number of tensors; expected 1188, got 972

> ```

>

> (the 216 missing tensors are the parallel branch; the 15 MTP tensors are dropped too).

> Until upstream lands support, use our fork:

>

> ```bash

> git clone -b agnes-3.0-flash https://github.com/JakeATX/llamAmpere.git

> cd llamAmpere && cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server

> ```

>

> Details: docs/agnes-3.0-flash.md

Quantization build for the Preview release of Agnes-AI/Agnes-3.0-Flash,

pinned to source revision 24f712ce59379b54c4a141d2708c35daf5ff613b.

Build status

The source safetensors are downloaded and verified locally. Conversion and

uploads are in progress; quant folders will be published independently as they

finish:

• ATX-Q3_K_XL/
• ATX-Q4_K_XL/
• ATX-Q4_K_M/
• ATX-Q5_K_XL/
• ATX-Q6_K_XL/
• ATX-Q8_K_XL/
• ATX-IQ4_XS-M/

Every target GGUF includes Agnes' available one-layer MTP/NextN head. The

GGUFs also retain the checkpoint's second 2,048-wide parallel FFN branch.

All Agnes model conversion, model-specific recipe adaptation, quantization,

validation, and publication in this repository were performed by ATX. Unsloth

did not produce these Agnes quants. Where an Unsloth Qwen3.8 calibration asset

is named in GGUF metadata, that records an input to ATX's layer-role transfer;

it is not an authorship or publisher claim for these artifacts.

Compatibility

Agnes' parallel FFN is not supported by stock llama.cpp at publication time.

The artifacts are built from official ggml-org/llama.cpp commit

3057bb66c86c46d5781e50e85462a760ba7d1feb plus the narrowly scoped patch in

build-info/. The patch commit

5bb7de853836418731d655a287af645bdf8257a5 has that official commit as its

direct parent. These artifacts contain the text model and native MTP head. The

vision projector is not included because the current converter does not

support an Agnes mmproj export.

The model is dense rather than MoE. The dynamic quant recipes therefore apply

role-aware bit allocation across recurrent attention, global attention, main

FFN, and parallel FFN tensors instead of routed-expert formulas. The parallel

FFN follows the corresponding main FFN tensor type in each layer.