---

license: apache-2.0

language:

• en

base_model:

• Hcompany/Holo-3.1-35B-A3B

base_model_relation: quantized

pipeline_tag: image-text-to-text

library_name: llama.cpp

tags:

• gguf
• moe
• multimodal
• computer-use
• gui-agents
• mtp
• speculative-decoding
• imatrix
• vision

---

Holo-3.1-35B-A3B — GGUF with restored MTP head, vision projector and native imatrix

Quantizations of Hcompany/Holo-3.1-35B-A3B

prepared with three properties not otherwise available in combination:

1. the model's native multi-token-prediction head restored, enabling speculative decoding;
2. the vision projector published alongside, preserving computer-use capability;
3. an importance matrix calibrated on the model's own agent-loop output.

All quantized files derive from a single bf16 master, which is also published.

---

1. Quantized models

| File | Size | Base type | Tensor map |

|---|---|---|---|

| …-MTP-UD-IQ4_XS.gguf | 18.21 GB | IQ4_XS | Unsloth Dynamic 2.0 |

| …-MTP-UD-Q4_K_XL.gguf | 22.85 GB | Q4_K_M | Unsloth Dynamic 2.0 |

| …-MTP-UD-Q5_K_S.gguf | 25.54 GB | Q5_K_S | Unsloth Dynamic 2.0 |

| …-MTP-UD-Q6_K.gguf | 30.01 GB | Q6_K | Unsloth Dynamic 2.0 |

| …-MTP-APEX-I-Mini.gguf | 14.27 GB | Q3_K_M | mudler APEX |

| …-MTP-APEX-I-Compact.gguf | 17.33 GB | Q4_K_M | mudler APEX |

| …-MTP-APEX-I-Quality.gguf | 23.49 GB | Q6_K | mudler APEX |

| …-MTP-APEX-I-Balanced.gguf | 26.06 GB | Q6_K | mudler APEX |

| …-MTP-APEX-I-Compact-v2D-lite.gguf | 17.34 GB | Q4_K_M | mudler APEX + v2D-lite |

Every file carries the MTP head at blk.40, and all are imatrix-guided.

2. Supporting artifacts

| File | Size | Purpose |

|---|---|---|

| BF16/…-BF16-MTP-0000N-of-00002.gguf | 45.94 + 25.13 GB | bf16 master with MTP embedded; source of every tier above |

| …-mmproj-F16.gguf | 0.90 GB | vision projector — required for image input |

| …-MTP-imatrix.gguf | 0.18 GB | importance matrix used for all tiers |

| original-MTP-head.safetensors | 1.69 GB | the MTP head in isolation, for re-grafting |

| sha256sums.txt | — | checksums for published artifacts |

---

3. Preparation

3.1 MTP head restoration

Holo-3.1-35B-A3B ships without multi-token-prediction tensors. The parent model was

established by direct weight comparison rather than metadata, the upstream card being

internally inconsistent on the point (prose states "Qwen 3.5 family"; YAML frontmatter

states Qwen/Qwen3.6-35B-A3B):

| Tensor | vs Qwen3.6-35B-A3B | vs Qwen3.5-35B-A3B |

|---|---|---|

| model.language_model.norm.weight | L1rel 1.23e-05, cos 1.0000000000 | L1rel 3.18e-02 |

| …layers.0.input_layernorm.weight | L1rel 3.43e-03 | L1rel 7.72e-02 |

Tensor cardinality corroborates it: Holo-3.1 contains 1026 tensors, Qwen3.6-35B-A3B

contains 1045, and the difference is exactly the 19 mtp.* tensors. Holo-3.1-35B-A3B is

therefore Qwen3.6-35B-A3B with the MTP head removed and then fine-tuned; the head here is

restored from the parent model, not adapted from an unrelated one.

The 19 mtp.* tensors (844.6 M parameters) were grafted at safetensors level before

conversion and asserted byte-identical to the donor as a precondition of proceeding:

donor head sha256  faac91f15cbe54475faa2578bedc46a7c29a947b8a3e7ef3ecd376ae079826ab

The head is deliberately not fine-tuned. Prior work on KAT-Coder-V2.5-Dev fine-tuned

the equivalent head twice on model-native rollouts; both attempts reduced draft acceptance

from 73% to 45–47%.

Measured draft acceptance: 85.0% (577/679 drafts) at 204 tokens/s, with the vision

projector loaded, reproduced on two machines. The same head on KAT-Coder-V2.5-Dev reaches

73–76%; the difference is attributable to lineage proximity. Measurement used a mixed

code/edit/structured prompt set rather than a GUI-agent benchmark.

3.2 Vision projector

Holo-3.1 contains 333 model.visual.* tensors. convert_hf_to_gguf.py emits the text

model only, so the projector was exported separately via --mmproj. A deployment without

it is text-only and cannot ground on screenshots.

3.3 Importance matrix

Calibrated on a tripartite corpus, evenly interleaved:

| Share | Source |

|---|---|

| 40% | Holo-native rollouts — the model's own agent-loop output: <tool_call> action traces, UI element descriptions, accessibility-tree reasoning, coordinate outputs, multi-step <tool_response> loops, recovery and refusal cases |

| 40% | bartowski calibration_datav5 |

| 20% | agentic-coding rollouts (KAT-Coder-V2.5) |

Rollouts were generated by the model itself at temperature 0.7 with no presence penalty,

then rendered through Holo-3.1's own chat_template.jinja, whose default strips <think>

blocks from assistant turns preceding the final user query — matching served context rather

than an idealised transcript.

The corpus is interleaved rather than concatenated, so any prefix preserves the 40/40/20

ratio, and is sized to the calibration budget: 980 chunks × 512 tokens = 501,760 tokens

against a corpus of 2,006,278 characters, so the corpus is consumed in full. The Holo share

is selected round-robin across task goals, so every distinct goal is represented before any

goal repeats.

Matrix verification: 510 tensors, no non-finite values, no all-zero tensors, no

zero-activation tensors, 430 of 442 quantizable tensors covered.

3.4 Tensor maps

Each tier's map was read from that tier's own published GGUF header by ranged HTTP request,

from Qwen3.6-35B-A3B sources — unsloth/Qwen3.6-35B-A3B-MTP-GGUF and

mudler/Qwen3.6-35B-A3B-APEX-MTP-GGUF — Holo-3.1 sharing that architecture exactly. No map

was inferred, and none was shared between tiers. Each was validated against the model's real

tensor list for double-matched patterns and uncovered quantizable tensors; all nine returned

zero of each.

Role-level precision allocation is the design of Unsloth and mudler respectively. The four

UD tiers reproduce Unsloth's published file sizes exactly (18.21 / 22.85 / 25.54 / 30.01 GB),

confirming faithful replay.

v2D-lite is applied to APEX-I-Compact only: it raises attn_k and attn_v on the ten

full-attention layers, a net-zero-byte reallocation with effect only where the parent map

leaves attention at Q4_K.

---

4. Deployment

llama-server \
  -m Hcompany_Holo-3.1-35B-A3B-MTP-UD-Q4_K_XL.gguf \
  --mmproj Hcompany_Holo-3.1-35B-A3B-mmproj-F16.gguf \
  -c 65536 -fa on --jinja \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 1 --spec-draft-n-min 0 --spec-draft-p-min 0.75 \
  --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 24 --spec-ngram-mod-n-match 48

Speculative decoding and the vision projector operate together; llama.cpp rejects that

combination only when draft and target differ in n_embd_inp, and here both are the same

trunk.

--spec-draft-p-min 0.75 is the highest-impact setting: restricting drafting to

high-confidence positions is what makes the head profitable.

Constraints imposed by --mmproj, per tools/server/server-context.cpp:

• ctx_shift is disabled
• cache_reuse is disabled
• prompt checkpoints are not created following image chunks

5. Building additional tiers

hf download gbuzhf/Holo-3.1-35B-A3B-MTP-GGUF --include "BF16/*" --local-dir .
llama-gguf-split --merge BF16/Hcompany_Holo-3.1-35B-A3B-BF16-MTP-00001-of-00002.gguf master.gguf
llama-quantize --imatrix Hcompany_Holo-3.1-35B-A3B-MTP-imatrix.gguf \
               --tensor-type-file your_map.txt master.gguf out.gguf Q4_K_M

6. Limitations

1. The importance matrix is text-only. --mmproj is registered in llama.cpp's

common/arg.cpp for LLAMA_EXAMPLE_{MTMD,SERVER,CLI,TTS} and not for IMATRIX, so no

projector output can enter calibration in any current build. The rollouts capture the

tool-call schema, ChatML framing and observation loop verbatim; they do not capture

visual grounding.

1. blk.40 is not covered by any importance matrix. llama-imatrix does not execute

the MTP head during a forward pass, so that block is quantized unguided in all builds.

1. The blend ratio was selected analytically, not empirically. A 50/50 two-way control

and a KL comparison against the bf16 reference were planned but not executed.

7. Credits

H Company — Holo-3.1 ·

Qwen — Qwen3.6-35B-A3B and the MTP head ·

Unsloth — Dynamic 2.0 tensor maps ·

mudler — APEX tensor maps ·

bartowski — calibration corpus ·

llama.cpp

Licensed Apache-2.0, inherited from the base model.