---

license: apache-2.0

base_model: IFM/K2-Horizon-MoVA-36B-A4B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

• en

tags:

• gguf
• llama.cpp
• k2-horizon
• mova
• moe
• long-context
• quantized
• text-generation

---

K2-Horizon-MoVA-36B-A4B GGUF

Community GGUF quantizations of IFM/K2-Horizon-MoVA-36B-A4B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About K2-Horizon-MoVA-36B-A4B

K2-Horizon-MoVA-36B-A4B

is a sparse Mixture-of-Experts model with Mixture-of-Values attention (MoVA).

The upstream checkpoint has 36B total parameters and activates about 4B

parameters per token. It advertises a native 524,288-token context window and

is intended for reasoning, coding, agentic, and long-context workloads. See the

official model card for

the original model, serving recipes, prompt conventions, and upstream results.


*Upstream K2-Horizon-MoVA-36B-A4B benchmark results; the image and scores are

from the official model card.*

Fidelity measurements

The table below compares every published GGUF file against the

BF16 reference on a held-out WikiText pilot: eight chunks from wiki.test.raw and eight chunks from

wiki.valid.raw, using a 4,096-token context and the same K2 llama.cpp runtime.

Values are averaged across the two splits. Size (GB) uses decimal GB

(file_size_bytes / 1,000,000,000). Lower Mean KLD, ΔPPL, and RMS Δp, and

higher Top-1 agreement, indicate closer next-token behavior to BF16. The BF16

reference mean PPL was 6.815612 in this pilot. These are next-token fidelity

measurements, not a direct percentage of capabilities retained and not a

substitute for task-specific evaluation.

| File | Size (GB) | Mean KLD (↓) | Top-1 vs BF16 (↑) | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|---:|

| K2-Horizon-MoVA-36B-A4B-Q8_0.gguf | 39.831 | 0.004015 | 97.270% | +0.071% | 1.822% |

| K2-Horizon-MoVA-36B-A4B-Q6_K.gguf | 30.765 | 0.005739 | 96.281% | +0.180% | 2.191% |

| K2-Horizon-MoVA-36B-A4B-Q5_K_M.gguf | 26.439 | 0.010129 | 95.185% | +0.527% | 2.865% |

| K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf | 22.368 | 0.017449 | 93.564% | +0.855% | 3.702% |

| K2-Horizon-MoVA-36B-A4B-Q4_K_S.gguf | 21.361 | 0.019009 | 93.454% | +0.904% | 3.738% |

| K2-Horizon-MoVA-36B-A4B-Q4_1.gguf | 23.580 | 0.018516 | 93.472% | +0.906% | 3.710% |

| K2-Horizon-MoVA-36B-A4B-Q4_0.gguf | 21.320 | 0.027218 | 92.184% | +1.244% | 4.603% |

| K2-Horizon-MoVA-36B-A4B-IQ4_NL.gguf | 21.281 | 0.019676 | 93.350% | +1.055% | 3.909% |

| K2-Horizon-MoVA-36B-A4B-IQ4_XS.gguf | 20.132 | 0.019835 | 93.558% | +0.957% | 3.909% |

| K2-Horizon-MoVA-36B-A4B-Q3_K_L.gguf | 18.841 | 0.047255 | 89.686% | +2.760% | 5.925% |

| K2-Horizon-MoVA-36B-A4B-Q3_K_M.gguf | 17.661 | 0.051186 | 89.265% | +2.920% | 6.213% |

| K2-Horizon-MoVA-36B-A4B-IQ3_M.gguf | 16.547 | 0.054490 | 89.066% | +4.063% | 6.483% |

| K2-Horizon-MoVA-36B-A4B-IQ3_S.gguf | 16.395 | 0.055718 | 89.060% | +4.034% | 6.482% |

| K2-Horizon-MoVA-36B-A4B-Q2_K.gguf | 13.622 | 0.116123 | 84.810% | +9.201% | 9.529% |

| K2-Horizon-MoVA-36B-A4B-Q2_K_S.gguf | 12.810 | 0.159896 | 82.334% | +13.465% | 11.167% |

| K2-Horizon-MoVA-36B-A4B-IQ2_XS.gguf | 11.179 | 0.236500 | 77.873% | +20.785% | 13.879% |

| K2-Horizon-MoVA-36B-A4B-IQ1_M.gguf | 8.689 | 0.604200 | 66.142% | +69.948% | 22.709% |

| K2-Horizon-MoVA-36B-A4B-Q1_0.gguf | 5.759 | 10.019445 | 1.478% | +2,069,546.308% | 59.829% |

Bold rows mark practical sweet spots: Q5_K_M for extra fidelity headroom,

Q4_K_M as the balanced default, and IQ3_M as the compact importance-aware

option. Q8_0 remains the high-bit reference profile.

In this pilot, Q1_0 shows very large divergence from the BF16 reference;

the extreme-compression profiles should therefore be treated as experimental.

The machine-readable results are available in

reproducibility/quality-summary.tsv,

with corpus hashes, evaluation settings, and runtime provenance in

reproducibility/manifest.md.

Quick start

Use a K2-Horizon-capable llama.cpp build. The explicit partial offload and

CPU MoE options in this example keep the model within the available accelerator

memory when full offload is not possible; adjust the GPU-layer count for your

hardware.

./llama-cli \
  -m K2-Horizon-MoVA-36B-A4B-Q6_K.gguf \
  --chat-template-file reproducibility/chat_template_smoke_user.jinja \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF, and why is it useful for running language models locally?' \
  -n 128 -c 4096 \
  --device CUDA0 \
  --gpu-layers 8 \
  --cpu-moe

The included template is the compatible single-turn template used by the

release smoke test. The upstream full tool-aware Jinja template is not claimed

as certified by this package. Long-context use may require substantially more

memory than the short example above.

Reproducibility and validation

The GGUF files were quantized directly from the locked upstream BF16 input. Q8_0

was generated without an imatrix; the other ladder members used the

model-specific combined importance matrix. Every published file passed the

load/generate smoke test before upload. Runtime throughput data, when present,

is supplementary and is recorded separately from the BF16 fidelity comparison

above in reproducibility/runtime-summary.tsv.

The public package includes compact reproduction inputs and scripts. Raw

conversion, imatrix, quantization, smoke-test, fidelity, and benchmark logs are

kept locally under reports/ and are intentionally not uploaded.

Checksums for all published artifacts and public reproduction files are in

SHA256SUMS.txt. The locked source revision, BF16 checksum,

runtime commit, calibration inputs, and validation settings are in

reproducibility/manifest.md.

License and attribution

The upstream model is released under the Apache License 2.0. Preserve upstream

attribution and the included LICENSE when redistributing these

derivative artifacts.

These are community GGUF quantizations, not an official IFM release or

endorsement.