---

base_model: IFM/K2-Horizon-MoVA-36B-A4B

base_model_relation: quantized

language:

- en

license: apache-2.0

tags:

- llama-cpp

- gguf

- moe

- mixture-of-experts

- mova

- mixture-of-values

- k2-horizon

- reasoning

- long-context

- ifm

- mbzuai

pipeline_tag: text-generation

---

K2-Horizon-MoVA-36B-A4B — GGUF

GGUF quantizations of IFM/K2-Horizon-MoVA-36B-A4B,

produced with the MBZUAI-IFM fork of llama.cpp

(model/K2Horizon branch) that adds native k2-horizon architecture support.

> Architecture note: Upstream llama.cpp does not yet support the k2_horizon /

> K2HorizonForCausalLM architecture (PR in progress as of September 2026).

> You must use the MBZUAI-IFM fork linked above — or upstream once the PR merges —

> to load these GGUFs.

---

Model Overview

K2-Horizon-MoVA-36B-A4B is the sparse member of the K2-Horizon family released by the

Institute of Foundation Models (IFM) at MBZUAI on 3 September 2026. It is a

Mixture-of-Experts (MoE) model augmented with a novel Mixture-of-Values (MoVA)

attention mechanism, storing ~36 B parameters while activating only ~4 B per token.

| Property | Value |

|---|---|

| Architecture | k2_horizon (K2HorizonForCausalLM) |

| Total parameters | ~36 B |

| Active parameters / token | ~4 B |

| Layers | 48 (3 leading dense + 45 MoE) |

| Hidden size | 2 560 |

| MoE routing | 100 routed experts / 8 active + 1 shared expert per MoE layer |

| MoVA attention | 64 value experts / 4 active per attention layer |

| Context window | 512 K (524 288 tokens, native) |

| Embedding tying | Untied (separate input / output embeddings) |

| Precision (source) | BF16 |

| License | Apache 2.0 |

---

What Is MoVA?

Mixture-of-Values (MoVA) applies the same sparse routing philosophy as MoE, but

inside the attention mechanism: the value projections are replaced by a learned pool of

value experts, and only a small subset is activated per token. This lets the model store

a much richer set of representational "styles" in the value space without paying for all

of them at inference time, complementing the feed-forward MoE sparsity.

---

Quantization Details

| File | Quant type | Approx. size | Notes |

|---|---|---|---|

| K2-Horizon-MoVA-36B-A4B-BF16.gguf | BF16 | ~70 GiB | Original precision; quantization source |

| K2-Horizon-MoVA-36B-A4B-Q8_0.gguf | Q8\_0 | ~35 GiB | Near-lossless |

| K2-Horizon-MoVA-36B-A4B-Q6_K.gguf | Q6\_K | ~27 GiB | Very high quality |

| K2-Horizon-MoVA-36B-A4B-Q5_K_M.gguf | Q5\_K\_M | ~23 GiB | High quality, recommended |

| K2-Horizon-MoVA-36B-A4B-Q5_K_S.gguf | Q5\_K\_S | ~22 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q5_1.gguf | Q5\_1 | ~22 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q5_0.gguf | Q5\_0 | ~21 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf | Q4\_K\_M | ~20 GiB | Good balance of size and quality |

| K2-Horizon-MoVA-36B-A4B-Q4_K_S.gguf | Q4\_K\_S | ~19 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q4_1.gguf | Q4\_1 | ~20 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q4_0.gguf | Q4\_0 | ~19 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q3_K_L.gguf | Q3\_K\_L | ~16 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q3_K_M.gguf | Q3\_K\_M | ~15 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q3_K_S.gguf | Q3\_K\_S | ~14 GiB | |

| K2-Horizon-MoVA-36B-A4B-Q2_K.gguf | Q2\_K | ~10 GiB | Smallest; used as imatrix source |

| K2-Horizon-MoVA-36B-A4B-IQ4_NL.gguf | IQ4\_NL | ~20 GiB | imatrix-calibrated |

| K2-Horizon-MoVA-36B-A4B-IQ4_XS.gguf | IQ4\_XS | ~19 GiB | imatrix-calibrated |

| K2-Horizon-MoVA-36B-A4B-IQ3_M.gguf | IQ3\_M | ~16 GiB | imatrix-calibrated |

| K2-Horizon-MoVA-36B-A4B-IQ3_XS.gguf | IQ3\_XS | ~14 GiB | imatrix-calibrated |

| K2-Horizon-MoVA-36B-A4B-IQ3_XXS.gguf | IQ3\_XXS | ~13 GiB | imatrix-calibrated |

| K2-Horizon-MoVA-36B-A4B-IQ2_M.gguf | IQ2\_M | ~11 GiB | imatrix-calibrated |

IQ quants (those beginning with IQ) were generated using an importance matrix

computed on the Q2\_K quant with 8 × 512-token chunks of WikiText-2 calibration data

(Salesforce/wikitext).

All quants were produced from the BF16 GGUF published by IFM at

IFM/K2-Horizon-MoVA-36B-A4B-GGUF.

No safetensors conversion was performed; the IFM BF16 GGUF already includes tokenizer

metadata and a llama.cpp-compatible chat template.

---

Hardware Requirements (Rough Guide)

These figures are weight-only estimates and exclude KV cache, runtime overhead, and

context length. They are not minimum VRAM/RAM requirements.

| Quant | Est. weight size | Practical minimum RAM/VRAM |

|---|---|---|

| BF16 | ~70 GiB | 2 × 48 GB GPU (A6000 / H100 80 GB) |

| Q8\_0 | ~35 GiB | 2 × 24 GB GPU |

| Q5\_K\_M | ~23 GiB | 1 × 24 GB GPU or 2 × 16 GB |

| Q4\_K\_M | ~20 GiB | 1 × 24 GB GPU |

| Q3\_K\_M | ~15 GiB | 1 × 16 GB GPU |

| Q2\_K / IQ2\_M | ~10–11 GiB | 1 × 12 GB GPU |

> Memory caveat: "4B active" means 4B parameters are computed per token, not that

> the model fits in 4B-worth of memory. All 36B weights must be resident in RAM/VRAM.

---

Running with llama.cpp

> Prerequisite: build from the MBZUAI-IFM fork (model/K2Horizon branch) until

> upstream llama.cpp merges the PR.

git clone --depth 1 --branch model/K2Horizon \
    https://github.com/MBZUAI-IFM/llama.cpp.git llama.cpp-k2horizon
cd llama.cpp-k2horizon
cmake -B build -DGGML_NATIVE=OFF
cmake --build build -j4

Basic inference:

./build/bin/llama-cli \
  -m K2-Horizon-MoVA-36B-A4B-Q4_K_M.gguf \
  -p "Explain the difference between MoE and MoVA architectures." \
  -n 1024 \
  --temp 1.0 \
  --top-p 0.95 \
  -c 8192

---

Serving (Original BF16 Model)

The original BF16 safetensors checkpoint can be served with vLLM or SGLang.

vLLM:

vllm serve IFM/K2-Horizon-MoVA-36B-A4B \
  --revision main \
  --tensor-parallel-size 2 \
  --enable-expert-parallel \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --enable-auto-tool-choice

SGLang (validated on 2 × H200):

python3 -m sglang.launch_server \
  --model-path IFM/K2-Horizon-MoVA-36B-A4B \
  --revision main \
  --tp 2 \
  --ep 2 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --json-model-override-args '{"xllm_source_router_gemm_partitions":2}' \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --host 0.0.0.0 --port 30000

---

Best Practices

1. Always use reasoning_effort="high" — all reported benchmark scores were

obtained at high reasoning effort.

1. Recommended sampling: temperature=1.0, top_p=0.95,

max_new_tokens ≥ 32 768.

1. Tool use / agent tasks: enable the k2_horizon tool-call parser when using

the vLLM or SGLang API.

1. llama.cpp context: the model supports 512 K natively, but local inference is

memory-constrained — use -c 8192 or -c 32768 unless you have sufficient RAM.

---

Benchmark Results

Results from the IFM model card. All scores in %. Bold = best in row.

K2-Horizon-MoVA-36B-A4B uses high reasoning effort; other open models are in their

respective reasoning modes.

| Benchmark | K2-Horizon-MoVA-36B-A4B (4B active) | Nemotron 3 Ultra (55B active) | Nemotron 3 Super (12B active) | G9v3-39A5B (5B active) | Qwen3.6-35B-A3B (3B active) | Muse Glimmer-30B (Dense) | Gemma 4 31B-it (Dense) |

|---|---|---|---|---|---|---|---|

| Agents | | | | | | | |

| tau3-Banking (tool use) | 26.8 | 14.2 | 10.3 | 22.1 | 9.3 | 23.5 | 14.8 |

| Coding | | | | | | | |

| Terminal-Bench 2.1 (agentic) | 58.6 | 53.9 | 38.6 | 32.6 | 44.9 | 51.7 | 43.4 |

| SciCode (scientific coding) | 38.9 | 39.9 | 36.0 | 34.0 | 35.8 | 43.6 | 43.4 |

| Scientific Reasoning | | | | | | | |

| Humanity's Last Exam | 25.2 | 28.4 | 20.8 | 17.5 | 22.2 | 22.0 | 23.6 |

| GPQA Diamond | 80.8 | 86.7 | 80.0 | 80.5 | 84.1 | 83.5 | 85.7 |

| CritPt (frontier physics) | 2.1 | 3.1 | 3.1 | 0.3 | 0.3 | 2.6 | 1.4 |

| General | | | | | | | |

| AA-LCR (long-context) | 66.3 | 71.0 | 60.3 | 62.0 | 66.7 | 80.0 | 68.3 |

| AA-Omniscience Accuracy | 18.8 | 22.6 | 24.3 | 14.9 | 18.8 | 27.0 | 20.0 |

| AA-Omniscience Non-Hallucination | 69.2 | 70.3 | 13.0 | 87.0 | 49.5 | 18.1 | 15.0 |

---

Source & Links

• Original model: IFM/K2-Horizon-MoVA-36B-A4B
• Official BF16 GGUF: IFM/K2-Horizon-MoVA-36B-A4B-GGUF
• llama.cpp fork (K2-Horizon support): MBZUAI-IFM/llama.cpp @ model/K2Horizon
• IFM blog post: https://ifm.ai/blog/k2/
• vLLM recipe: https://recipes.vllm.ai/IFM/K2-Horizon-MoVA-36B-A4B
• SGLang cookbook: https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon

---

Quantization Pipeline

These GGUFs were generated with the following pipeline, mirroring the approach used for

other large MoE checkpoints in this workspace:

1. Fork checkout & build — the MBZUAI-IFM model/K2Horizon branch is cloned to a

dedicated directory and compiled with CMake (GGML_NATIVE=OFF for portability).

1. Source download — the IFM BF16 GGUF (~70 GiB) is downloaded from

IFM/K2-Horizon-MoVA-36B-A4B-GGUF instead of converting from safetensors

(a direct BF16 or Q8\_0 conversion would exceed the ~109 GiB free on /tmp).

1. Imatrix — computed on the Q2\_K quant (~10 GiB, fits in 15 GB RAM) with 8

calibration chunks of WikiText-2 to produce an importance matrix for the IQ quants.

1. Quantization — all standard and IQ quants are produced from the BF16 GGUF source;

each file is deleted immediately after a verified upload to keep disk usage under

BF16 (~70 GiB) + one quant at any given moment.

1. Resume support — a status.json file tracks which quants have been generated and

uploaded so interrupted runs resume from where they left off.

---

License

The original model and these GGUF quantizations are released under the

Apache 2.0 License. See LICENSE

on the upstream repository for full terms.

---

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}