---

license: other

library_name: llama.cpp

pipeline_tag: text-generation

tags:

- gguf

- lynnstyle

- moe

- quantization

- glm52

- gpt55

---

Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-GGUF

Llama.cpp MUST READ BEFORE LAUNCH

Official llama.cpp prebuilt binaries without this compatibility patch will fail to load this GGUF.

Reason: unpatched official llama.cpp builds cannot recognize the pruned Step35 / Step 3.7 MoE layout where the routed expert count differs by layer. They still read step35.expert_count as a single u32, while this pruned MoE stores expert counts as a per-layer array.

For example, llama-b9892-bin-win-vulkan-x64 fails during hyperparameter loading with:

key step35.expert_count has wrong type arr but expected type u32

This is not a corrupted-shard issue and is not caused by the MTP sidecar. Use a patched Step 3.7 / LynnStyle-compatible build, or apply the patch and rebuild.

If you are not comfortable rebuilding llama.cpp, give the Patch and Usage notes links below to _Codex or another coding agent_, and ask it to apply the patch to your target llama.cpp source tree and rebuild.

| Required file | Repository link |

|---|---|

| Patch | patches/llama.cpp-step37-expert-count-array.patch |

| Usage notes | runtime/llama.cpp-step37-compat.md |

Reference runtime: llama.cpp 8c146a8366304c871efc26057cc90370ccf58dad; the release tests used CUDA llama-server built from the patched llama.cpp tree.

For newer official source trees or binaries such as b9892, apply the same patch logic and rebuild; re-downloading GGUF shards will not fix this error.

> Important GGUF parser note: Hugging Face / ModelScope GGUF auto-detection may parse a standalone MTP sidecar or one shard and show misleading metadata such as 3B params, 5-bit, or 2.49GB. That is not the main model. This repository uses multi-shard mixed-precision LynnStyle GGUF. Download the complete Q4/, Q5/, Q8/, or Q3/ shard set plus the matching MTP sidecar, and follow the README launch template. The platform hardware-compatibility panel is not authoritative for this mixed-precision release.

!model card header

> _Naming note: GLM5.2-SFT / GPT5.5-RL describes post-training data provenance, not weight provenance. Most SFT Agent/ReAct trajectories were generated, rewritten, or cleaned with GLM 5.2 API assistance; most RL preference pairs and judge signals came from a GPT-5.5/Codex-style review policy. No GLM or GPT weights are mixed into this model._

Repository Scope

This is the GGUF repository for Step-3.7-Flash-180B-LynnStyle-GLM5.2-SFT-GPT5.5-RL.

The 180B label comes from the pruned effective MoE scale: after expert-workload analysis and task-scope protection, about 8%, roughly 1000 low-contribution experts, were removed from the original MoE.

LynnStyle is the full local-inference method chain: localization-guided protection of key layers, experts, and tensor families; dynamic layered pruning of low-contribution paths; targeted SFT/RL to reduce overthinking, empty answers, non-delivery, and parse failures; and calibrated mixed-precision quantization that keeps core layers, hot experts, router paths, attention, lm_head, and MTP protected.

This repository is for GGUF quantized files, runtime smoke results, MTP single/concurrent checks, and quantized gate metrics. BF16 weights belong in the main repository.

Text Scope and MTP Sidecars

This release currently claims text reasoning, code, and Agent/ReAct text capability. It does not claim image input. The original dyn263 BF16 source retained multimodal traces such as vision_config and vision_encoder.py, but this final RL2 merge, export, quantization, and evaluation path did not validate the complete vision stack: vision-tower weights, projector / mmproj, image processor, image-token alignment, and llama.cpp image-input smoke tests.

MTP sidecars are optional speculative-decoding draft files. They accelerate inference together with a main GGUF model; they are not vision files and cannot be used as standalone main models.

• LynnStyle Q8 uses Q8/Step-3.7-Flash-MTP-Q8_0.gguf.
• LynnStyle Q5 uses Q5/Step-3.7-Flash-MTP-Q8_0.gguf.
• LynnStyle Q4 uses Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf.

LynnStyle GGUF Strategy

Q8 is the quality baseline. The most important endpoint tier in this release is LynnStyle Q4: around 90GB, aimed at fitting a 180B-class MoE into a 96GB/R6000 single-card setup.

The previous way to fit this model class into R6000 was closer to a quality-collapsed Q3. LynnStyle Q4 is tested early because it decides whether the 96GB endpoint route is real.

LynnStyle Q4 Profile

LynnStyle Q4 Profile F6 is not a uniform IQ4_XS file. It is explicit core protection plus interleaved imatrix calibration plus lower precision for cold MoE experts. The dry-run target is 93216.74 MiB / 91.03 GiB, leaving a more realistic margin for a 96GB/R6000 setup.

| Protection Bucket | Tensor Family | Precision | Matched Tensors |

|---|---|---:|---:|

| L0-L2 first 3 Dense blocks | all blk.0-2.*.weight | Q8_0 | 36 |

| Embedding / output / rope / output_norm anchors | token_embd.weight, output.weight, output_norm.weight, rope_freqs.weight | Q8_0 | 4 |

| Norm / gate / router small tensors | attn_norm, ffn_norm, attn_q_norm, attn_k_norm, attn_gate, ffn_gate_inp, exp_probs_b.bias | Q8_0 | 294 |

| Shared experts | ffn_down_shexp.weight, ffn_gate_shexp.weight, ffn_up_shexp.weight | Q8_0 | 126 |

| Late L43-L44 MoE experts | ffn_down_exps.weight, ffn_gate_exps.weight, ffn_up_exps.weight | Q5_K | 6 |

| L39-L42 shoulder and L4/L16/L30 hot-zone MoE experts | ffn_down_exps.weight, ffn_gate_exps.weight, ffn_up_exps.weight | Q4_K | 21 |

| Attention q/k/v/o family | attn_q.weight, attn_k.weight, attn_v.weight, attn_output.weight | Q4_K | 168 |

| Cold MoE experts L5/L6/L8-L11 | ffn_down_exps.weight, ffn_gate_exps.weight, ffn_up_exps.weight | Q3_K | 18 |

| Remaining MoE experts | other ffn_down/gate/up_exps.weight | default IQ4_XS | 81 |

The protection table is combined with a 2026-07-06 interleaved imatrix calibration set that rotates through LBC, Coding100, ReAct, GPQA, MMLU, native SFT, and RL-preference prompts. It does not use benchmark answer keys as supervised training labels.

!MoE expert concentration

!LBC/Coding expert heatmap

These figures are the evidence behind the pruning and quantization policy: MMLU/GPQA and LBC/Coding100 do not activate exactly the same experts; L39-L44 are high-risk in both scopes; code tasks expose specialized experts that reasoning-only traces do not fully cover. LynnStyle is therefore not blind pruning and not uniform low-bit quantization.

Available Tiers

| Tier | Target Hardware | Role | Status |

|---|---:|---|---|

| LynnStyle Q8 | 190GB+ / multi-GPU | quality baseline | accepted |

| LynnStyle Q4 | 96GB / R6000 | core recommendation | accepted Profile F6, imatrix, MTP matrix complete |

| LynnStyle Q5 | 128GB | high-quality tier | accepted, MTP matrix complete |

| LynnStyle Q3 | 64G | lower-memory tier | with Q4 MTP |

| LynnStyle Q2 Experimental | 48GB | experimental entry tier | not published |

Download guidance:

• LynnStyle Q4: download every shard under Q4/, from 00001-of-00005 to 00005-of-00005.
• LynnStyle Q5: download every shard under Q5/, from 00001-of-00005 to 00005-of-00005.
• LynnStyle Q8: download every Q8 shard under Q8/.
• MTP sidecar: for Q8, download Q8/Step-3.7-Flash-MTP-Q8_0.gguf; for LynnStyle Q5, download Q5/Step-3.7-Flash-MTP-Q8_0.gguf; for LynnStyle Q4, download Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf.

Metrics

Final RL2 Q8+MTP:

| Metric | Result |

|---|---:|

| MMLU500 | 465/500 = 93.0% |

| GPQA198 | 149/198 = 75.25% |

| Coding100 | 76/100 |

| LBC100 | 77/100 |

LynnStyle Q4 imatrix Profile F6:

| Metric | Result |

|---|---:|

| MMLU500 | 463/500 = 92.6% |

| GPQA198 | 142/198 = 71.72% |

| Coding100 | 76/100 |

| LBC100 | 73/100 |

LynnStyle Q5 imatrix:

| Metric | LynnStyle Q5 Result |

|---|---:|

| MMLU500 | 460/500 = 92.0% |

| GPQA198 | 143/198 = 72.22% |

| Coding100 | 77/100 |

| LBC100 | 74/100 |

!GPQA reasoning comparison

RL2 is the final release because it preserves the main public gates while reducing heavy-thinking and non-delivery failure modes. An RL1 comparison run was stopped at 81/117 = 69.23% on GPQA, making RL2 the stronger release for this objective.

MTP Recommendation

LynnStyle Q4 has completed the C1/C2/C4 by draft N=1..4 MTP matrix on the 2026-07-06 R6000 setup. TPS is llama.cpp predicted_tokens_seconds.

| Concurrency | draft N=1 | draft N=2 | draft N=3 | draft N=4 | Recommendation |

|---|---:|---:|---:|---:|---|

| C1 | 4/4, 124.0 tok/s, 4.816s | 4/4, 135.7 tok/s, 4.249s | 4/4, 142.3 tok/s, 4.041s | 4/4, 141.4 tok/s, 4.066s | N=3 |

| C2 | 4/4, 76.7 tok/s, 8.015s | 4/4, 79.0 tok/s, 7.733s | 4/4, 86.4 tok/s, 6.533s | 4/4, 84.5 tok/s, 6.417s | N=3 |

| C4 | 4/4, 56.2 tok/s, 10.164s | 4/4, 54.6 tok/s, 10.159s | 4/4, 51.1 tok/s, 10.115s | 4/4, 51.6 tok/s, 11.220s | N=3 |

For 96GB/R6000 LynnStyle Q4, use Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf as the draft model. LynnStyle Q5 and Q8 use the Step-3.7-Flash-MTP-Q8_0.gguf sidecar in their own tier directories. The Q8_0 MTP sidecar is about 3.71GB and is not the default Q4 pairing.

LynnStyle Q5 MTP:

| Concurrency | Recommended draft N | TPS | Avg latency |

|---|---:|---:|---:|

| C1 | N=3 | 124.1 tok/s | 4.372s |

| C2 | N=3 | 72.0 tok/s | 7.579s |

| C4 | N=1 | 52.2 tok/s | 10.976s |

File-size fit notes:

• LynnStyle Q4: about 93.16 GiB including its Q5_K_M MTP sidecar; recommended for 96GB/R6000 with conservative context first.
• LynnStyle Q5: about 123.23 GiB including its Q8_0 MTP sidecar; recommended for 128GB-class memory.
• LynnStyle Q8: about 182.55 GiB including its Q8_0 MTP sidecar; use multi-GPU / 190GB+ class memory.

llama.cpp Templates

LynnStyle Q8 + MTP

MAIN_GGUF="Q8/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q8_0-00001-of-00005.gguf"
MTP_GGUF="Q8/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 32768 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q5 imatrix + MTP

MAIN_GGUF="Q5/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q5-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q5/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q5 imatrix + MTP

MAIN_GGUF="Q5/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q5-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q5/Step-3.7-Flash-MTP-Q8_0.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --split-mode layer \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

LynnStyle Q4 imatrix + MTP

MAIN_GGUF="Q4/Step-3.7-Flash-180B-LynnStyle-GLM52-SFT-GPT55-RL-Q4-imatrix-MTP-00001-of-00005.gguf"
MTP_GGUF="Q4/Step-3.7-Flash-MTP-Q5_K_M.gguf"

./llama-server \
  -m "$MAIN_GGUF" \
  -ngl 999 \
  --ctx-size 8192 \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --spec-type draft-mtp \
  --model-draft "$MTP_GGUF" \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0.6 \
  --host 0.0.0.0 \
  --port 8000

---