---

license: apache-2.0

base_model: swiss-ai/Apertus-v1.1-4B-Instruct

base_model_relation: quantized

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- apertus

- swiss-ai

- multilingual

language:

- multilingual

---

Apertus v1.1 4B Instruct - GGUF

Quantized from swiss-ai/Apertus-v1.1-4B-Instruct.

All credit to Swiss AI - this is only a

format conversion.

Why another one

The other quantized 4B GGUFs available lost their chat template somewhere in conversion.

Without it llama.cpp falls back to ChatML, whose <|im_start|> and <|im_end|>

are not tokens in this vocabulary, so nothing ever ends a turn - the model

writes both sides of the conversation until it hits your token limit.

Files

| | Size | |

|---|---|---|

| apertus-v1.1-4b-instruct-q8_0.gguf | 4.1 GB | basically lossless, start here |

| apertus-v1.1-4b-instruct-q4_k_m.gguf | 2.4 GB | smaller and quicker, a bit worse |

Running it

llama-server -m apertus-v1.1-4b-instruct-q8_0.gguf --ctx-size 4096

4096 is what it was trained on.

Made with

docker run --rm -v ./source:/src:ro -v ./out:/models \
    ghcr.io/ggml-org/llama.cpp:full \
    --convert /src --outfile /models/f16.gguf --outtype f16

docker run --rm -v ./out:/models ghcr.io/ggml-org/llama.cpp:full \
    --quantize /models/f16.gguf /models/apertus-v1.1-4b-instruct-q8_0.gguf Q8_0

llama.cpp build FILL IN.