---

license: apache-2.0

base_model: Blockway/Agens-Volundr-32B-v1

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language: [en, zh, yue]

tags: [gguf, llama.cpp, agens, volundr, blockway, agent, tool-use, code, linear-attention, sparse-attention, cantonese]

---

Agens Volundr 32B v1 — GGUF

Main model card, benchmarks and discussion: Blockway/Agens-Volundr-32B-v1

GGUF files of Agens Volundr 32B v1 for llama.cpp: BF16,

Q8_0 and Q4_K_M. Text only. For what the model is, its benchmarks and its chat format, see the

main model card.

> Needs the volundr branch of llama.cpp: github.com/BlockWayz/llama.cpp.

> Volundr is a new architecture (KDA linear attention, BCSA sparse attention, Engram memory, mHC hyper-connections).

> Stock llama.cpp, and apps built on it, will refuse these files with unknown model architecture: 'volundr'

> until support is merged upstream.

Files

| file | size | contents |

|---|---:|---|

| Agens-Volundr-32B-v1-Q4_K_M.gguf | 26.6 GB | Q4_K_M for the feed-forward and BCSA/dense attention projections; every KDA projection, the Engram value/gate projections, the mHC dynamic projections, the token embedding and the output layer at Q8_0; Engram tables and BCSA indexer BF16. Recommended. |

| Agens-Volundr-32B-v1-Q8_0.gguf | 36.3 GB | all matrices Q8_0; Engram tables and BCSA indexer BF16 |

| Agens-Volundr-32B-v1-BF16.gguf | 64.5 GB | reference: the text model's weights unquantised (BF16) |

Norms, convolution, decay and mHC static parameters are F32 in every file. The linear-attention (KDA) weights are kept

at Q8_0 in the 4-bit file on purpose: an earlier model of ours degraded badly when its linear-attention projections

were quantised to 4 bit. SHA256SUMS lists the checksums.

Build llama.cpp (volundr branch)

git clone --branch volundr https://github.com/BlockWayz/llama.cpp && cd llama.cpp

# CPU
cmake -B build -DCMAKE_BUILD_TYPE=Release
# or NVIDIA GPU
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON

cmake --build build -j --target llama-server llama-cli

If CMake cannot find libcurl, add -DLLAMA_CURL=OFF. More detail in

README-volundr.md.

Run

pip install -U huggingface_hub
hf download Blockway/Agens-Volundr-32B-v1-GGUF Agens-Volundr-32B-v1-Q4_K_M.gguf --local-dir .

# OpenAI-compatible server on :8080 (thinking, tool calls and reasoning_content parsing work out of the box)
./build/bin/llama-server -m Agens-Volundr-32B-v1-Q4_K_M.gguf --jinja -c 32768 -ngl 99

# interactive chat in the terminal
./build/bin/llama-cli -m Agens-Volundr-32B-v1-Q4_K_M.gguf --jinja -c 32768 -ngl 99

Drop -ngl 99 to run on the CPU only. When offloading to a GPU, the four Engram tables (4.3 GB of pure lookups) can

stay in host RAM: --override-tensor "engram_embd=CPU".

Thinking is on by default. Per request:

curl http://127.0.0.1:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is 17 * 23 + 9?"}],
  "chat_template_kwargs": {"reasoning_effort": "low"}
}'
# "chat_template_kwargs": {"enable_thinking": false}      -> no thinking
# "chat_template_kwargs": {"reasoning_effort": "medium"}   -> low | medium | xhigh (default xhigh)

Tool calls use the standard OpenAI tools field and come back as tool_calls. Sampling defaults stored in the GGUF:

temperature 1.0, top-k 20, top-p 0.95. The chat template is the model's own and is embedded in every file.

Multi-turn chat: prompt cache

llama-server reuses the cached state across the turns of a conversation, so only the new tokens of each turn are

processed (the recurrent KDA/Engram state is rolled back through context checkpoints taken during prompt processing;

nothing has to be enabled). Measured with the Q4_K_M file on one 48 GB GPU, -np 1, greedy:

| history | turn 2, cold (full prefill) | turn 2, warm (cache reuse) | tokens processed when warm |

|---:|---:|---:|---:|

| 3K tokens, no thinking | 2.1 s | 0.6 s | 37 |

| 8K tokens, no thinking | 7.1 s | 0.5 s | 36 |

| 8K tokens, thinking (reasoning dropped from history: checkpoint restore) | 6.3 s | 0.7 s | 123 |

| 16K tokens, no thinking | 18.4 s | 0.7 s | 37 |

Restoring a checkpoint is exact (bit-identical logits, llama-volundr-state-test). Flags: --ctx-checkpoints N

(default 32, 168 MiB each in host RAM; 8 is plenty for a chat), --cache-ram (host prompt cache for idle

conversations), one slot per concurrent conversation (-np). --cache-reuse (KV shifting) is not supported.

Speed

Q4_K_M on one 48 GB GPU, full offload (-ngl 99), flash attention, F16 KV cache:

| context | 1K | 8K | 32K |

|---|---:|---:|---:|

| decode | 32 tok/s | 27 tok/s | 25 tok/s |

(Preview measurements; the v1 files have the same tensor layout and types. In this build's checks: 26.5 tok/s at a 1.1K

prompt with a 4-slot server, prompt processing 1,480 tok/s at 3K and 920 tok/s at 16K.)

Quantisation quality

llama-perplexity on held-out chat text (mostly Cantonese and Chinese), 12 chunks of 4,096 tokens; KLD and top-1 are

measured against the BF16 file.

| file | PPL | vs BF16 | mean KLD | same top-1 |

|---|---:|---:|---:|---:|

| BF16 | 3.3654 | — | — | — |

| Q8_0 | 3.3664 | +0.06 % | 0.00040 | 99.17 % |

| Q4_K_M | 3.3936 | +0.87 % | 0.0092 | 96.15 % |

With an 8,192-token context scored on positions 4,096–8,191 (where the BCSA far field is active) the picture is the

same: 3.1078 / 3.1086 / 3.1337.

Parity with the reference implementation

The llama.cpp graph was checked against the checkpoint's own PyTorch code (fp32) on all-position logits: the BF16 file

agrees on 100 % of top-1 tokens (mean KL 1–2e-5) on an English prose prompt and a chat-templated prompt, and the

64-token greedy continuation is identical on the first and diverges on the second only at a 0.02-logit near-tie.

Restoring a saved sequence state or a context checkpoint and re-processing the same tokens gives bit-identical logits.

Unused-token mask

The output layer has 248,320 rows but the tokenizer ends at id 248,076. The ids 248,077–248,319 are never valid output,

yet the model can put real probability on some of them: where a tool call opens, sampling sometimes picks one instead of

the tool-call token and the call comes back as plain text. The volundr branch therefore masks that range (a -inf

logit bias, applied automatically for this architecture in llama-server and llama-cli). With the mask, sampled tool

calls with the Q4_K_M file parsed 104/104 (no-think at temperature 0.6 and 1.0, thinking at 1.0).

VOLUNDR_MASK_UNUSED_TOKENS=0 turns the mask off. If you call the llama.cpp library directly (bindings that do not use

llama.cpp's common sampling code), apply the same bias yourself.

Known limits

• Text only. The vision tower is not converted; use the main repository for image input.
• No speculative decoding in llama.cpp.
• CUDA and CPU only for the two Volundr-specific ops (Engram hashing, mHC Sinkhorn). On Metal, Vulkan or ROCm the

scheduler should run those two ops on the CPU; those backends have not been tested.

• BCSA prefill cost. Prompt processing scores every query against every cached position before pooling and

selecting blocks, so it slows down with prompt length (about 620 tok/s for a 32K-token prompt on the same GPU).

Decoding switches to a sparse path above 12,288 cached positions.

• Tokenizer. llama.cpp applies no Unicode normalisation (use NFC input, as the HF tokenizer does), and for scripts

with combining marks (Devanagari, Thai, Burmese, Tamil, ...) its Qwen3.5 pre-tokenizer keeps marks inside letter

runs where the HF tokenizers library splits them, so those languages tokenise differently from the HF/sglang model.

English, Chinese, Cantonese, Vietnamese, Japanese, Korean, Arabic and code tokenise identically.

• No context shift and no partial KV-cache removal (recurrent state), as for llama.cpp's other hybrid models.
• Multi-GPU --split-mode layer works but is no faster than one 48 GB GPU; --split-mode row does not load.

Licence

Apache-2.0. These files are a format conversion of the weights in

Blockway/Agens-Volundr-32B-v1; LICENSE, NOTICE

and the files they reference apply.