---

license: other

base_model: Qwen/Qwen3.6-27B

library_name: gguf

tags:

- qwen

- layerdose

- gguf

- llama.cpp

- quantized

---

Qwen3.6-27B Layerdose GGUF

GGUF exports for the LayerDose 14-layer linear_attn replacement candidate derived from Qwen/Qwen3.6-27B.

Current Status

This GGUF model is not runnable yet in stock llama.cpp and should be treated as an archival/export artifact for the current LayerDose experiment. It requires additional runtime integration for the custom LayerDose replacement tensors/operators before it can be used as a normal GGUF model.

Files

• Qwen3.6-27B-LayerDose-14LinearAttnRank64-BF16.gguf
• Qwen3.6-27B-LayerDose-14LinearAttnRank64-Q4_K_M.gguf
• llama.cpp-layerdose-qwen36-source.zip

The source zip contains the patched llama.cpp source used during this experiment. The model is still not a turnkey runnable GGUF release.

Size

• BF16 GGUF: 50,590,118,944 bytes
• Q4_K_M GGUF: 15,534,421,024 bytes
• Normal Q4_K_M baseline used locally: 16,547,398,976 bytes
• LayerDose Q4_K_M saving vs normal Q4_K_M: 1,012,977,952 bytes

Local Decode Speed Smoke

CPU-only llama-simple, prompt Hello, -n 128.

| Model | Decode t/s |

|---|---:|

| Normal Q4_K_M | 5.16, 5.24 |

| LayerDose Q4_K_M | 5.70, 5.76 |

Reverse-order pass speedup: 1.099x by decode tokens/sec. Load time is not used for this comparison because it is strongly affected by filesystem cache/order.

Quality Smoke

Compared LayerDose BF16 GGUF against LayerDose Q4_K_M GGUF:

• ARC64-95: BF16 0.9375, Q4 0.9375, delta 0.0000
• HellaSwag64-95: BF16 0.5625, Q4 0.5625, delta 0.0000
• Combined option KL BF16->Q4: 0.008657
• LM KL Q4-vs-BF16 on 16 held-out prompts: 0.017597 +/- 0.001729
• Same-top-token rate: 96.429%
• Greedy generation smoke passed for both BF16 and Q4

This is a smoke-level validation, not a full benchmark suite.