---

license: apache-2.0

base_model: Aleph-Alpha/Kolibri-1

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

language:

• en
• de

tags:

• gguf
• llama.cpp
• kolibri1
• mixture-of-experts

---

Kolibri-1 GGUF by Prompt48

Independent GGUF conversion and quantization of Aleph Alpha's Kolibri-1. Aleph Alpha created the original model; Prompt48 produced these GGUF artifacts from the original checkpoint. No further training was performed.

Compatibility

These files require Kolibri architecture support. This release uses llama.cpp at 836d57176 with the Kolibri patch by Hob-forge/Seraphiel102, credited and available at Hob-forge/Kolibri-1-GGUF. Ordinary llama.cpp, Ollama and LM Studio builds without that support cannot load these files.

The implementation patch is MIT; the original model weights are Apache-2.0. The original model license is included with this release.

Conversion

The source is Aleph Alpha's block-scaled FP8 checkpoint, converted to a BF16 GGUF intermediate. Dequantizing to BF16 does not recover precision lost in the original FP8 checkpoint. Every quantization is generated directly from that intermediate, rather than from another low-bit quantization.

Source revision: e52eb4627d11516b0c01de49210ab5a4e4061444.

python convert_hf_to_gguf.py /path/to/source --outfile Kolibri-1-BF16.gguf --outtype bf16
./build/bin/llama-quantize Kolibri-1-BF16.gguf Kolibri-1-Q4_K_M.gguf Q4_K_M 16

Files and independent validation

| File | Size (decimal GB) |

|---|---|

| Kolibri-1-Q4_K_M.gguf | 47.45 |

Validation on this release: CPU-only patched llama.cpp, 4,096-token configured context; four short prompts.

• english (2.0 seconds, including prompt processing): Sunlight is made of many colors, and shorter blue wavelengths scatter more easily in the atmosphere. This scattered blue light reaches our eyes from all directions, making the sky appear blue.
• german (1.9 seconds, including prompt processing): Der Himmel ist blau, weil die Luftmoleküle kurzwelliges blaues Licht stärker streuen als langwelliges rotes Licht. Dieses gestreute blaue Licht erreicht unsere Augen aus allen Richtungen des Himmels.
• arithmetic (0.6 seconds, including prompt processing): 391
• tool_call (3.4 seconds, including prompt processing): [{"type": "function", "function": {"name": "get_weather", "arguments": "{\n \"city\": \"Heidelberg\"\n}"}, "id": "hogICM5zctvNGp2B9LeCMOhJz2V2L0Dw"}]

See validation/smoke-tests.json for the exact requests and responses, and SHA256SUMS for file integrity. No importance matrix was used.

Short smoke tests demonstrate basic operation; they do not establish benchmark equivalence to the original FP8 model. Safety behavior and broad task quality are unverified. CUDA on an A100 80GB and long-context needle retrieval (Q4_K_M) are reported in the context-length section below; other GPUs and backends are unverified.

Running

Apply the included patch to the specified llama.cpp revision and build before running:

./build/bin/llama-server -m Kolibri-1-Q4_K_M.gguf -c 4096 --jinja --host 127.0.0.1

The model has 78B total parameters and 3.46B active per token. Memory requirements depend on the full weights plus runtime buffers and KV cache. See the original model card for architecture, intended use, recommended sampling and limitations.

Context length: enable the full 1,048,576 tokens

The GGUF metadata stores kolibri1.context_length = 262144 (copied from max_position_embeddings in the source config.json). llama.cpp therefore caps every slot at 262,144 tokens, even if you pass a larger -c:

the slot context (1048576) exceeds the training context of the model (262144) - capping
n_ctx_slot = 262144

The original model card advertises 1,048,576 tokens and, for vLLM, uses a max_position_embeddings override to reach it. The llama.cpp equivalent needs no reconversion, one extra flag:

./build/bin/llama-server -m Kolibri-1-Q4_K_M.gguf -ngl 99 -c 1048576 \
  --override-kv kolibri1.context_length=int:1048576 --jinja

After this the log shows n_ctx_slot = 1048576 and no capping warning.

Measured on a RunPod A100 80GB, Q4_K_M, patched llama.cpp 836d571 built with CUDA 12.8 (sm_80):

| Item | Result |

|---|---|

| llama-bench -ngl 99 | pp512 2,496.89 ± 257.03 tok/s, tg128 126.45 ± 0.92 tok/s |

| VRAM, model loaded, 8,192 ctx | 45,929 MiB |

| VRAM, 262,144 ctx | 51,137 MiB |

| VRAM, 1,048,576 ctx | 67,267 MiB |

| Needle (46% depth), 195,935 tokens | exact, 122 s |

| Needle (46% depth), 718,803 tokens, override on | exact, 914 s |

Notes: GPU (CUDA) inference is now verified on this one setup; other GPUs, backends and the Q6_K / Q8_0 files remain untested here. At medium/high reasoning effort the model can spend the entire token budget thinking, so give it a large max_tokens or use reasoning_effort: low for code. Full video: https://youtu.be/rx4-l8e9R5c

Q6_K

Total weight-file size: 64.23 GB, split into 2 files. Download all parts into the same folder and point llama.cpp at the first part.

Generated directly from the same BF16 intermediate and tested with the same four CPU smoke prompts; see validation/Q6_K/smoke-tests.json. These limited tests are not a quality benchmark.

• english (2.8 seconds): Sunlight is made of many colors, and the atmosphere scatters shorter blue wavelengths more than longer red ones. This scattered blue light reaches our eyes from all directions, making the sky appear blue.

• german (2.8 seconds): Der Himmel ist blau, weil die Luftmoleküle kurzwelliges blaues Licht stärker streuen als langwelliges rotes Licht. Dieses gestreute blaue Licht erreicht unsere Augen aus allen Richtungen des Himmels.

• arithmetic (0.8 seconds): 391

• tool_call (4.5 seconds): [{"type": "function", "function": {"name": "get_weather", "arguments": "{\n \"city\": \"Heidelberg\"\n}"}, "id": "vVWvDHq1kl189YeBvzSjVPijF5rhPDmn"}]

Q8_0

Total weight-file size: 83.14 GB, split into 2 files. Download all parts into the same folder and point llama.cpp at the first part.

Generated directly from the same BF16 intermediate and tested with the same four CPU smoke prompts; see validation/Q8_0/smoke-tests.json. These limited tests are not a quality benchmark.

• english (2.5 seconds): Sunlight is made of many colors, and shorter blue wavelengths scatter more easily in the atmosphere. This scattered blue light reaches our eyes from all directions, making the sky appear blue.

• german (2.7 seconds): Der Himmel ist blau, weil die Luftmoleküle kurzwelliges blaues Licht stärker streuen als langwelliges rotes Licht. Dieses gestreute blaue Licht erreicht unsere Augen aus allen Richtungen des Himmels.

• arithmetic (0.7 seconds): 391

• tool_call (4.4 seconds): [{"type": "function", "function": {"name": "get_weather", "arguments": "{\n \"city\": \"Heidelberg\"\n}"}, "id": "jPGNNnztXtvGrXn4X4AnvdQ3tP7Z63hZ"}]

Build the tested CPU runtime

Download kolibri1-llama.cpp.patch from this repository. In the folder containing that patch:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 836d571
git apply ../kolibri1-llama.cpp.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=OFF
cmake --build build -j --target llama-server
./build/bin/llama-server -m /path/to/Kolibri-1-Q4_K_M.gguf -c 4096 -np 1 --jinja --host 127.0.0.1

For Q6_K or Q8_0, replace the model path with the first split file and keep the second part beside it. These CPU smoke tests ran on a RunPod A100 host with approximately 250 GB assigned system RAM and 12 CPU threads. The tests used temperature=0, max_tokens=192, and chat_template_kwargs={"reasoning_effort":"none","enable_thinking":false}. These are test settings; consult the original model card for recommended sampling settings.

No GPU acceleration was used for the four smoke tests above (CPU only). A later CUDA run is reported in the context-length section: CUDA verified on one A100 80GB with Q4_K_M; other backends remain unverified.

Complete RunPod tutorial and viewer code

The complete conversion walkthrough and code are available in tutorial/README.md. Download the viewer code ZIP for numbered portable scripts, the RunPod helper, original session code, conversion evidence and the three fresh Q4 exercises with executed checks. Code contains no private credentials. The tutorial explains the CPU backend, pinned build, storage requirements, attribution, uploading and pod cleanup.