---

base_model: ManniX-ITA/Qwen3.6-27B-Omnimerge-v4

base_model_relation: quantized

license: apache-2.0

language:

• en

tags:

- gguf

- imatrix

- quantized

- merge

- mergekit

- qwen3_5

- reasoning

- code

pipeline_tag: image-text-to-text

library_name: gguf

---

Qwen3.6-27B-Omnimerge-v4-GGUF

GGUF quantizations of ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 — the MLP-passthrough variant that defends against the Qwen3.6 think-policy fragility we discovered. Source dtype is BF16; this repo provides the standard bartowski quant ladder (F16 → IQ2_XXS) for llama.cpp.

> Source model: ManniX-ITA/Qwen3.6-27B-Omnimerge-v4 (BF16 weights, model card with full benchmarks and methodology).

> NOT a quant of clean Qwen/Qwen3.6-27B — these GGUFs contain the v4 merge.

>

> MTP companion (2× decode speedup): weight-identical GGUFs with the MTP head retained for llama.cpp --spec-type draft-mtp self-speculative decoding are at ManniX-ITA/Qwen3.6-27B-Omnimerge-v4-MTP-GGUF. Quality is statistically indistinguishable from this repo (HE 137/164 ↔ 137/164, GPQA 155/198 ↔ 154/198); aggregate decode is 2.0-2.3 × faster on a single 24 GB GPU. Use that repo for interactive / single-request workloads where latency matters.

All quants made using imatrix with calibration data v5, the same calibration set bartowski uses for the Qwen3.6 base release — so quality fingerprints are directly comparable to bartowski's Qwen_Qwen3.6-27B-GGUF repo.

Why this merge exists

Same-base DARE-TIES (Omnimerge_v2 method) merge of Qwen/Qwen3.6-27B + 3 Qwen3.6 fine-tunes. Direct successor to ManniX-ITA/Qwen3.5-27B-Omnimerge-v2 on the newer Qwen3.6 base, with mlp.{gate,up,down}_proj copied verbatim from clean Qwen3.6 (the "MLP-passthrough" surgery) to defend against a Qwen3.6-specific reasoning-tag fragility we found during forensic delta inspection. See the v4 model card for the full story, scripts, and benchmark methodology.

Benchmark headline (Q6_K, head-to-head vs Qwen3.6 base + Omnimerge-v2)

All scored under identical llama.cpp + lm_eval conditions (--reasoning-format deepseek --reasoning-budget 8192 --parallel 2, raw /v1/completions, no chat template).

| Benchmark | Qwen3.6 base Q6_K (bartowski) | Omnimerge-v2 (Qwen3.5 base) | Omnimerge-v4-MLP (this) | Δ vs base | Δ vs v2 |

|---|---|---|---|---|---|

| HumanEval pass@1 (164q) | 84.76% | 79.27% | 83.54% (137/164) | −1.22 pp | +4.27 pp |

| MBPP pass@1 (500q) — corrected\* | 57.60% | 74.60% | 73.00% (365/500) | +15.40 pp | −1.60 pp |

| GPQA Diamond pass@1 (flex) — full greedy§ | not measured | 69.19% (full 198q) | 78.28% (155/198) | — | +9.09 pp |

\* MBPP scores are post-<think>-stripping (lm_eval's raw scorer SyntaxErrors on literal < in exec(prompt+completion+tests)). See the v4 model card for the per-model recovery breakdown.

§ Canonical full-198q greedy GPQA result measured 2026-05-22 on pod 37268930 (Vast.ai 3090) with the patched eval chain (lm-eval 0.4.11 + max_length=32768 override + the api_models.py:545 UnboundLocalError patch + aiohttp lifecycle workaround). Sampler: do_sample=False, temperature=0.0, max_gen_toks=8192. Wall time 4 h 55 min. Companion strict-match (rigid Answer: X template) is 7.58 % — the model emits CoT verbosely rather than the strict template, so flex is the real quality signal. Earlier card revisions reported an ≈ 84.75 % partial result (177/198 sampled at T=0.6, budget=16384); that number is superseded by this canonical greedy measurement on the full bench — the 6.5 pp difference is driven by the methodology change (sampler / budget / completeness), not by a model change.

Available Quantizations

All 27 files (F16 + 26 imatrix-quantized tiers, ~417 GB total) are uploaded and ready. imatrix.dat (used for every quant) is in the repo root for audit and reproduction.

| Quantization | File size | Use case |

|---|---|---|

| F16 (full precision) | 50.11 GB | Conversion source / lossless reference |

| Q8_0 | 26.63 GB | Highest fidelity, large |

| Q6_K_L | 21.14 GB | Q6_K with embed/output at Q8_0 |

| Q6_K | 20.57 GB | Recommended high tier — eval methodology used this |

| Q5_K_L | 18.64 GB | Q5_K_M with embed/output at Q8_0 |

| Q5_K_M | 17.91 GB | Strong fidelity, balanced |

| Q5_K_S | 17.40 GB | Slightly smaller K-mix |

| Q4_K_L | 16.29 GB | Q4_K_M with embed/output at Q8_0 |

| Q4_1 | 15.91 GB | Legacy 4-bit, dense |

| Q4_K_M | 15.41 GB | Recommended balanced tier for most users |

| IQ4_NL | 14.72 GB | Importance-aware 4-bit non-linear |

| Q4_K_S | 14.52 GB | K-mix small variant |

| Q4_0 | 14.41 GB | Legacy 4-bit |

| IQ4_XS | 14.05 GB | IQ4 extra-small |

| Q3_K_XL | 13.42 GB | Q3_K_L with embed/output at Q8_0 |

| Q3_K_L | 13.36 GB | 3-bit K-mix large |

| Q3_K_M | 12.39 GB | 3-bit K-mix medium |

| IQ3_M | 11.72 GB | Importance-aware 3-bit medium |

| Q3_K_S | 11.24 GB | 3-bit K-mix small |

| IQ3_XS | 11.15 GB | IQ3 extra-small |

| Q2_K_L | 11.13 GB | Q2_K with embed/output at Q8_0 |

| IQ3_XXS | 10.42 GB | IQ3 extra-extra-small |

| Q2_K | 9.98 GB | 2-bit K-mix |

| IQ2_M | 9.32 GB | Importance-aware 2-bit medium |

| IQ2_S | 8.72 GB | IQ2 small |

| IQ2_XS | 8.47 GB | IQ2 extra-small |

| IQ2_XXS | 7.85 GB | IQ2 extra-extra-small (smallest) |

How to Use

With llama.cpp:

# Recommended args for reasoning-tag-emitting models:
llama-server \
    -m Qwen3.6-27B-Omnimerge-v4-Q4_K_M.gguf \
    -c 32768 -ngl 99 -t 12 --no-warmup \
    --reasoning-budget 8192

See Reasoning budget and thinking stop phrase below for the budget, the wrap-up phrase that stops reasoning

leaking into the answer, and why --reasoning-format plays no part in it.

The published evals additionally pin --reasoning-format deepseek so that

lm_eval sees only the answer; it is not needed for ordinary serving.

Swap Q4_K_M for any tier from the table above. Q6_K matches the methodology used in our published evals; Q4_K_M is the typical "balanced" choice for most users.

For multimodal (vision) inference: the mmproj projector is in bartowski/Qwen_Qwen3.6-27B-GGUF and works with this model unchanged (vision tower is preserved verbatim from the base).

With ollama: use a Modelfile pointing to one of the GGUFs above, or HF direct load.

Reasoning budget and thinking stop phrase (llama.cpp)

Qwen 3.6 reasons at length by design, and on a hard prompt it can consume the

whole context window before it answers. llama.cpp can bound the thinking block

with a sampler, and — the part that actually matters — tell the model why the

block is being closed.

Needs llama.cpp b8508 or newer for the flags, b10091 or newer for the

per-request overrides.

Serve with a bounded thinking block

llama-server -m Qwen3.6-27B-Omnimerge-v4-Q4_K_M.gguf -c 32768 -ngl 99 \
    --jinja \
    --reasoning-budget 8192 \
    --reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n' \
    --temp 0.6 --top-k 20 --top-p 0.95

| flag | meaning |

|---|---|

| --reasoning-budget N | -1 unrestricted (default), 0 close the block immediately, N > 0 cap it at N tokens |

| --reasoning-budget-message | text written into the block just before the closing tag is forced |

| --jinja | required — the delimiters come from the chat template (<think> … </think>). Without it llama.cpp has no tags to count and the budget silently does nothing |

Both flags also read from the environment: LLAMA_ARG_THINK_BUDGET and

LLAMA_ARG_THINK_BUDGET_MESSAGE.

--reasoning-format is not part of this. It only decides how the thinking

is handed back — message.reasoning_content versus left inline in

message.content — and never whether the budget is enforced: the delimiters the

sampler counts are set by the chat template regardless, so the cap binds under

auto, deepseek and none alike. The default auto already extracts

reasoning and is behaviourally identical to deepseek (they differ only in

name; the sole branch in the parser is != none). Leave it at the default so

the model's own tool-call and channel handling stays in play, and pin

deepseek only when a harness needs the thinking kept out of content.

--reasoning-budget on its own forces the closing tag the moment the budget

runs out, wherever the model happens to be. When that lands mid-thought the

model frequently does not register that it was interrupted: it carries on

reasoning, now inside the visible answer. The stop phrase is what prevents

that — it gives the model a reason to be finishing.

Two wordings that work

# "qwen" — the string Qwen's own service uses, from their docs
--reasoning-budget-message $'\n\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n'

# "voice" — shorter, in the model's own reasoning voice
--reasoning-budget-message $'\n\nOK, I have enough to answer now.\n'

Wording is model-specific: Qwen note that the ability to act on such a message

"is not explicitly trained but emerges naturally", so it is worth trying both

on your own workload. Leading and trailing newlines matter — they keep the

phrase off whatever half-finished line the cut landed on.

What it measures out to

Measured on the Qwen3.6-35B-A3B base this model is pruned from. Three hard

questions, temperature 0.6, fixed seed, answer characters with wall time in

brackets. Every run answered all three correctly, and thinking length is

unchanged by the message in every row:

| budget | no message | qwen | voice |

|---|---|---|---|

| 2048 | 1907 (69 s) | 1838 (42 s) | 1615 (41 s) |

| 4096 | 18015 (170 s) | 2642 (78 s) | 1441 (104 s) |

| 8192 | 3642 (158 s) | 1848 (129 s) | 2023 (175 s)|

The 4096 row is the failure this exists for: the cap lands mid-thought and the

reasoning simply continues in the answer, ten times longer and 2.2x the wall

time, for the same three correct answers. Both phrases remove it.

Per request, instead of per server

The server accepts both as request fields, overriding the command line:

{
  "messages": [ ... ],
  "thinking_budget_tokens": 8192,
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On the raw /completion endpoint the delimiters are not inferred, so they have

to be supplied with the budget:

{
  "prompt": "...",
  "reasoning_budget_tokens": 8192,
  "reasoning_budget_start_tag": "<think>",
  "reasoning_budget_end_tag": "</think>",
  "reasoning_budget_message": "\n\nOK, I have enough to answer now.\n"
}

On b10091 the message field must be present on /completion requests even

when empty: llama.cpp builds the sequence it forces from message + end_tag

inside that field's handler, so omitting it leaves the budget with nothing to

force — the sampler logs as though the cap fired while the thinking block stays

open.

Rules of thumb

• Keep -c several times larger than the budget. A budget equal to the context

lets the thinking phase fill the window on its own.

• A quarter of the context is a sensible starting point: 8192 at -c 32768.
• Qwen recommend keeping a thinking budget above 1024 tokens; below that

the cap tends to land before the model has committed to an approach.

• The budget is per thinking block, not per response — the sampler re-arms

when it sees a new opening tag, so a multi-turn agent gets a fresh window each

time.

imatrix.dat

The imatrix.dat (~14 MB) used to generate every quant in this repo is uploaded alongside the GGUFs at the repo root. Reproducible, auditable.

Reproducing

See scripts/ on the source v4 model repo:

• dare_ties_merge.py — main merger (auto-detects Qwen3.6 base via output_gate_type and applies MLP-skip)
• v4_mlp_passthrough.py — post-process: rebuild merged dir with MLP layers from base
• quantize_gguf.py — the script that built this repo

For dense (non-Gemma-4-MoE) models, pass --exclude CD-Q6_K,CD-Q5_K_M,CD-Q4_K_M,CD-Q3_K_M,CD-Q2_K to skip ContribDynamic tiers (those require Gemma 4 expert-contribution maps).

License

Apache-2.0 (inherited from Qwen/Qwen3.6-27B and the fine-tune sources).

Acknowledgements

• Qwen team for the Qwen3.6 base
• rico03, ValiantLabs, kai-os for the fine-tunes
• bartowski for the calibration_datav5.txt set used here
• DARE / TIES / DARE-TIES authors and the arcee-ai/mergekit community