---

license: apache-2.0

language:

• en

tags:

• agentic
• tool-use
• coding-agent
• function-calling
• gguf
• llama.cpp
• imatrix

library_name: llama.cpp

pipeline_tag: text-generation

---

gl-agent-1-27b-GGUF

GGUF builds of a 27B model fine-tuned for long horizon agentic tasks. The quant,

the MTP sidecar and the vision projector are separate downloads, so you take

only what you use.

Which file

Perplexity is measured on held-out agentic traces that the calibration data

never saw. The absolute values mean nothing outside this table; they are here to

show where the curve bends. BF16 reference is 2.6333.

| file | size | bpw | PPL | |

|---|---|---|---|---|

| UD-Q8_K_XL | 28.60 GB | 8.41 | 2.6333 | reference |

| UD-Q6_K_XL | 23.71 GB | 6.97 | 2.6339 | |

| UD-Q5_K_XL | 20.09 GB | 5.91 | 2.6455 | |

| UD-Q4_K_XL | 18.00 GB | 5.29 | 2.6668 | good default |

| UD-IQ3_XXS | 13.16 GB | 3.87 | 2.8505 | best value at the low end |

| UD-Q2_K_XL | 11.97 GB | 3.52 | 3.0600 | |

| UD-IQ2_XXS | 10.23 GB | 3.01 | 3.5451 | |

| UD-IQ1_M | 9.26 GB | 2.72 | 4.3435 | |

| UD-IQ1_S | 8.69 GB | 2.55 | 4.9578 | smallest recommended |

| UD-IQ1_S_XS | 6.04 GB | 1.78 | 6.4027 | experimental, see below |

Q8 down to Q4 costs 0.033 perplexity, so there is little reason to run anything

above UD-Q4_K_XL. UD-IQ3_XXS is where the size saving stops being free.

UD-IQ1_S_XS is here because sub-2-bit is interesting, not because it works. It

writes coherent prose on some prompts and drops into a repetition loop on

others. Do not put an agent behind it.

Running it

llama-cli -hf Loke-60000/gl-agent-1-27b-GGUF:UD-Q4_K_XL \
  -ngl 99 -c 32768 --jinja

Needs a recent llama.cpp. This architecture mixes full attention with linear

attention layers and support for it is new; build b1-5de25a7 or later works.

Set -c to what your memory allows, since the model's own maximum is far larger

than most single machines can hold.

Thinking is on by default. enable_thinking, preserve_thinking and

reasoning_effort are settable per request through the chat template.

MTP sidecar, optional

The multi-token-prediction block ships as its own file. llama.cpp ignores those

tensors entirely during ordinary decoding, so folding them into every quant

would make everyone carry weight that most people never execute.

llama-cli -m gl-agent-1-27b-UD-Q4_K_XL.gguf \
  -md mtp-gl-agent-1-27b-q8_0.gguf \
  --spec-type draft-mtp -ngl 99 -c 32768 --jinja

Measured on UD-Q4_K_XL at a fixed seed: 11.4 t/s without it, 17.4 t/s with it.

Take the sidecar nearest your quant. q8_0 is 3.16 GB, q4_K is 2.03 GB, and

the BF16 copy is there if you want to quantise it yourself. The mtp- prefix is

the sidecar convention, so -hf together with --spec-type draft-mtp resolves

it without being told where to look.

The sidecar carries its own embedding and output tensors so it can load as a

draft model, which means the pair comes to a little more than a fused build

would. That is the price of not imposing it on everyone.

Vision, optional

llama-mtmd-cli -m gl-agent-1-27b-UD-Q4_K_XL.gguf \
  --mmproj mmproj-gl-agent-1-27b-F16.gguf -ngl 99

How these were built

The importance matrix was computed over real agentic trajectories rather than

generic web text, so the tensors that carry tool loops are the ones weighted

during quantisation.

Bit depth is not uniform. FFN tensors are 64% of the weights and that is where

the bits come from. Token embeddings, the output head and the full-attention

layers sit one to several steps higher, and the small state tensors of the

linear-attention layers stay at full precision. Those are what break first and

they are cheap to keep.

Every file above was checked for a valid header, scored on the held-out set, and

made to generate before being published.

Benchmarks

Coming soon.