---

base_model: IFM/K2-Horizon-7B

pipeline_tag: text-generation

library_name: transformers

license: apache-2.0

language:

- en

tags:

- gguf

- llama.cpp

- k2-horizon

- imatrix

- quantized

---

K2-Horizon-7B GGUF — imatrix quants

GGUF quantizations of IFM/K2-Horizon-7B, made directly from the publisher's BF16 GGUF rather than from the FP8 checkpoint.

Files

| File | Quant | Size | Guidance |

|---|---:|---:|---|

| K2-Horizon-7B-Q8_0.gguf | Q8_0 | 8.92 GiB | Near-BF16 quality; largest and safest quant |

| K2-Horizon-7B-Q6_K.gguf | Q6_K | 6.89 GiB | Strong quality/size trade-off |

| K2-Horizon-7B-Q4_K_M.gguf | Q4_K_M | 5.21 GiB | Recommended compact quant |

| K2-Horizon-7B-imatrix-WikiText2-100K.gguf | imatrix | 5.10 MiB | Reproducibility artifact |

Q6_K is the llama.cpp quant name corresponding to “Q6 K”; there is no Q6_K_M format.

SHA-256 checksums:

5a98a289aba5c8c99ef05c9261287f19e8f586fd679bab45b632eb86b47809bf  K2-Horizon-7B-Q8_0.gguf
4f0449e4582118df22e401b1dd31dcece63d0e431695836e0c5812901e7c51ac  K2-Horizon-7B-Q6_K.gguf
66371e17b1ee10b8ae36c6c1c9c8ebbb29ae80d006f8951e15dfc4bea082fb4a  K2-Horizon-7B-Q4_K_M.gguf
4095ee9e7d91bc46e73955fc5139196c2fcae65099eef058b75a6f3b5fa25136  K2-Horizon-7B-imatrix-WikiText2-100K.gguf

Quantization quality

Q6_K and Q4_K_M were produced with a 252-tensor importance matrix collected over 200 × 512-token chunks (~100k tokens) from the WikiText-2 training split. The imatrix was collected with the Q8_0 model because the 16.76 GiB BF16 graph exceeded the local Metal working-set limit. Q8_0 is a close proxy to BF16 here (measured mean KLD 0.000467). Output and token embedding tensors intentionally use llama.cpp's normal mixed-quant fallback because imatrix collection does not process output.weight by default.

Small held-out WikiText-2 guardrail, measured against BF16 logits over two 256-token chunks with llama.cpp commit 35999d101cf2233fc54f09c3c8d599da7303ce02:

| Quant | PPL | Mean KLD vs BF16 | RMS Δp | Same top token |

|---|---:|---:|---:|---:|

| BF16 | 11.508836 ± 2.043381 | — | — | — |

| Q8_0 | 11.508721 ± 2.041464 | 0.000467 ± 0.000052 | 0.476% | 98.031% |

| Q6_K | 11.500250 ± 2.039763 | 0.004487 ± 0.000333 | 1.767% | 93.701% |

| Q4_K_M | 11.884325 ± 2.101731 | 0.025116 ± 0.002599 | 3.922% | 88.189% |

This is a deliberately small local regression test, not a general capability benchmark. Task-specific evaluation is still recommended.

Compatibility

K2 Horizon support is not yet in a released upstream llama.cpp build at the time of publication. Use the publisher's model/K2Horizon fork or a future upstream build that includes K2 Horizon support. These files were generated and validated with fork commit 35999d101cf2233fc54f09c3c8d599da7303ce02.

Example:

llama-cli \
  -m K2-Horizon-7B-Q4_K_M.gguf \
  -ngl 99 \
  -c 32768 \
  --temp 1.0 \
  --top-p 0.95 \
  -n 32768

The embedded chat template defaults to high reasoning effort, matching the publisher's recommendation. Context length is 524,288 tokens, but practical context depends on available memory and KV-cache settings.

Reproducibility

• Source repository revision: 7d69990f2d27e67a3f52aaf2e24d34c1dcfdc630
• Source file: K2-Horizon-7B-BF16.gguf
• Source SHA-256: 088c5d0814ef955d137fd1073ee3a68f6a53411fca44b75dc58a320640000444
• llama.cpp fork commit: 35999d101cf2233fc54f09c3c8d599da7303ce02
• Calibration corpus: WikiText-2 raw training split, first 200 chunks at context 512
• Evaluation corpus: WikiText-2 raw test split, first two chunks at context 256

Quantization commands:

llama-imatrix \
  -m K2-Horizon-7B-Q8_0.gguf \
  -f wiki.train.raw \
  -o K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  -ngl 99 -c 512 -b 512 -ub 512 \
  --chunks 200 --no-ppl

llama-quantize --imatrix K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  K2-Horizon-7B-BF16.gguf K2-Horizon-7B-Q6_K.gguf Q6_K

llama-quantize --imatrix K2-Horizon-7B-imatrix-WikiText2-100K.gguf \
  K2-Horizon-7B-BF16.gguf K2-Horizon-7B-Q4_K_M.gguf Q4_K_M

License and credit

The original model is released by IFM under Apache-2.0. See the upstream model card for architecture, training, usage guidance, limitations, and citation details.