---

license: apache-2.0

language:

• en

base_model: empero-ai/Qwythos-9B-Claude-Mythos-5-1M

pipeline_tag: text-generation

library_name: gguf

tags:

• qwen
• qwen3.5
• 9b
• quantized
• quantization
• gguf
• llama-cpp
• q5_k_m
• q8_0
• q6_k
• imatrix
• hybrid-quantization
• selective-quantization
• lwq
• lwq8
• lightweight
• reasoning
• llm
• open-source
• empero
• claude-mythos-5
• deltanet
• gated-attention
• vision

---

> ⚠️ Upload in progress — quantized models are being added. Check back soon for download links.

Qwythos-9B-Claude-Mythos-5-1M — LWQ8 (Selective Hybrid Quants)

> Note: File names contain Q5_K_M for HF parser compatibility only. These are not pure Q5_K_M — they're selective hybrid quants using Q8_0, Q6_K, IQ4_XS, Q5_K, and F16 across different tensor types. See each section for the exact per-tensor map.

> Selective hybrid quantizations for Empero's Qwythos-9B-Claude-Mythos-5-1M — a full-parameter reasoning fine-tune of Qwen3.5-9B with 1M context and vision support.

>

> Uses the exact same LWQ8 method and formulas as Qwable-9B-Claude-Fable-5-LWQ8-GGUF — same architecture, same imatrix, same quantization strategies. This is not a new LWQ9 series; it's the proven LWQ8 approach applied to a different fine-tune.

Status: Complete

| Quant | Size | PPL (ctx=1024) | vs Q6_K | vs OptA |

|---|---|---|---|---|

| Q6_K (baseline) | 7,008 MiB | 7.5876 ± 0.04948 | — | — |

| LWQ8-OptA ★ quality | 6,303 MiB | 7.4831 ± 0.04827 | −0.105, −705 MiB | — |

| LWQ8-v2 ★ compact | 5,726 MiB | 7.6542 ± 0.05003 | +0.067, −1,282 MiB | +0.171, −577 MiB |

Key finding: OptA formula transfers perfectly from Qwable (beats Q6_K PPL at 10% smaller). v2 shows a larger gap on Qwythos (+0.067) than on Qwable (+0.017) — IQ4_XS tensors impose a slightly higher cost on this fine-tune.

> LWQ8-v2 vs stock quants: Despite the Q5_K_M name tag, LWQ8-v2 is actually better than a pure Q5_K_M — Q8_0 on critical attention paths + Q6_K on mid layers more than compensates for IQ4_XS on low-importance tensors. A stock Q5_K_M would be ~5.3 GB with PPL ~7.9 on Qwythos; v2 is 7.65 at nearly the same size. And vs Q3_K_M (~4.1 GB, PPL ~8.5+): v2 is only 1.6 GB larger but dramatically better quality. If you need the smallest possible file, consider v2 before dropping to low-bit quants.

Architecture

Identical to Qwable-9B-Claude-Fable-5 — same Qwen3.5-9B backbone:

| Property | Value |

|---|---|

| Layers | 32 (24 Gated DeltaNet + 8 Full Attention) |

| Hidden dim | 4096 |

| FFN intermediate | 12288 |

| Vocabulary | 248,320 |

| Full Attention | blk.3, 7, 11, 15, 19, 23, 27, 31 |

| DeltaNet | all others |

| Context | 1,048,576 (YaRN factor 4.0) |

Tensors, shapes, and importance distribution are identical to Qwable. All LWQ8 techniques transfer directly.

Imatrix

Reused Qwable-9B-Claude-Fable-5.imatrix.gguf — same architecture, same tensor layout, same importance distribution. Confirmed working — OptA formula produces identical size (6,303 MiB) on Qwythos.

LWQ8-OptA (Quality Champion)

Q5_K_M base + Q8_0 on attn_gate/qkv/ssm_alpha/beta all layers + blk.31 ffn_down Q8_0 + F16 norms + imatrix per-row.

~/llm-tools/llama.cpp/build/bin/llama-quantize \
  --imatrix /mnt/everything/qwen/source/Qwable-9B-Claude-Fable-5.imatrix.gguf \
  --output-tensor-type Q5_K \
  --token-embedding-type Q4_K \
  --tensor-type "output_norm.*=Q8_0" \
  --tensor-type "blk\.\d+\.attn_gate=Q8_0" \
  --tensor-type "blk\.\d+\.attn_qkv=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_alpha=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_beta=Q8_0" \
  --tensor-type "blk\.31\.ffn_down=Q8_0" \
  --tensor-type ".*attn_q_norm.*=Q8_0" \
  --tensor-type ".*attn_k_norm.*=Q8_0" \
  --tensor-type ".*ssm_conv1d.*=Q8_0" \
  --tensor-type ".*ssm_dt.*=F16" \
  --tensor-type ".*ssm_a=F16" \
  --tensor-type ".*attn_norm.*=F16" \
  --tensor-type ".*post_attention_norm.*=F16" \
  --tensor-type ".*ssm_norm.*=F16" \
  /mnt/everything/qwen/source/Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf \
  /mnt/everything/qwen/output/Qwythos-9B-Claude-Mythos-5-1M-LWQ8-OptA.gguf \
  Q5_K_M

• PPL 7.4831 — beats Q6_K (7.5876) by −0.105 PPL at 10% smaller size
• Size 6,303 MiB vs Q6_K 7,008 MiB (10% smaller, 705 MiB saved)
• Same formula, same size, same quality transfer from Qwable

LWQ8-v2 (Compact Champion)

Q5_K_M base + tiered precision:

• Q8_0 — blk.0 (L0) + blk.26-31 attn_gate/qkv/ssm_alpha/beta, blk.31 ffn_down/attn_output, norms
• Q6_K — blk.1-25 attn_gate/qkv/ssm_alpha/beta
• IQ4_XS — ssm_out, attn_output, ffn_down (low importance)
• F16 — ssm_dt, ssm_a, all norms

Config: configs/lwq8_v2.sh (same as Qwable LWQ8-v2, adapted for Qwythos paths)

• PPL 7.6542 vs Q6_K 7.5876 (+0.067, within 2× SE)
• Size 5,726 MiB vs Q6_K 7,008 MiB (18% smaller, 1,282 MiB saved)
• Larger gap on Qwythos than Qwable (+0.067 vs +0.017) — IQ4_XS slightly more costly here

Speed

| Quant | Tokens/sec (GTX 1070) |

|---|---|

| LWQ8-OptA | ~26 t/s |

| LWQ8-v2 | ~28 t/s |

v2 is ~8% faster thanks to fewer Q8_0 tensors reducing VRAM bandwidth pressure on Pascal's limited PCIe bus.

Coding Examples

Both quantizations generate full, working HTML/CSS/JS websites in a single pass at temperature 0.6 with the prompt:

> "I'm a dev, my audience is youth. I like a creative/tech style. Write the full website code. This HTML will be our foundation."

LWQ8-OptA — mythos-LWQ8_temp-0.6.html

A complete dev portfolio in 1072 lines:

• Full hero, skills, projects, about, contact, and footer sections
• Animated grid background, floating elements, glow buttons
• Mobile menu, smooth scroll, active nav highlight, form with validation
• Stagger entrance animations for cards
• External deps: Google Fonts, Font Awesome

LWQ8-v2 — mythos-LWQ8-v2_temp-0.6.html

Same prompt, more streamlined output in 580 lines:

• Hero, projects, skills, and footer — focused layout
• Cleaner CSS, fewer sections, parallax effect
• No Font Awesome — lightweight approach
• External deps: Google Fonts only

> At temp 0.6, both quants produce professional code. OptA goes for completeness (more sections, more features), v2 for conciseness. The reasoning fine-tune shows its quality — no repetition, no broken markup at this temperature.

Usage

> Recommended sampling (wepiqx): Start with temperature 0.6, top_k 20, top_p 0.95, min_p 0. If the model starts repeating/looping, raise temperature to 1.0. If it over-thinks (excessive reasoning tokens) and temperature doesn't help, raise repeat_penalty to 1.05. Be cautious with high temperatures — this is a reasoning fine-tune and can get unstable above 1.2.

llama.cpp

llama-cli \
  -m M-LWQ8-OptA-Q5_K_M.gguf \
  -p "Your prompt here" \
  -ngl 99 \
  --flash-attn on \
  -c 4096 \
  --temp 0.6 \
  --top-k 20 \
  --top-p 0.95 \
  --min-p 0

For server mode (recommended):

llama-server \
  -m M-LWQ8-OptA-Q5_K_M.gguf \
  -c 65536 \
  -fit off \
  -ngl 99 \
  --jinja --chat-template-file /path/to/chat_template.jinja \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --port 8080 \
  --mmap \
  --temp 0.6 \
  --top-p 0.95 \
  --min-p 0 \
  --top-k 20 \
  --seed -1 \
  --parallel 1

> VRAM usage: ~7.4 GiB / 8 GiB at -c 65536 with flash-attn + q8_0 KV cache. The Gated DeltaNet architecture is memory-efficient even at long context.

>

> From wepiqx: This fine-tune can over-engineer at times, but overall it's a solid model. I'd love to see an MTP variant released — with MTP I could make an even better quantization with specialized draft model support for faster inference.

Ollama

1. Create a Modelfile:

FROM ./Qwythos-9B-Claude-Mythos-5-1M-LWQ8-OptA.gguf

TEMPLATE """{{ .System }}
{{ .Prompt }}"""

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
PARAMETER min_p 0

1. Build and run:

ollama create qwythos-lwq8 -f Modelfile
ollama run qwythos-lwq8

> If the model loops, increase temperature to 1.0. If it over-thinks, try repeat_penalty 1.05.

LM Studio

1. Open LM Studio
2. Drag Qwythos-9B-Claude-Mythos-5-1M-LWQ8-OptA.gguf into the app
3. Set GPU Offload to 99 layers
4. Enable flash-attention
5. Set context length to 8192+
6. Set sampling: temperature 0.6, top_k 20, top_p 0.95, min_p 0, repeat_penalty 1.0 (off)
7. If the model loops, raise temperature to 1.0. If it over-thinks, set repeat_penalty to 1.05.
8. Start chatting

Files

| File | Size | Description |

|---|---|---|

| Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf | 17 GB | BF16 source (from upstream) |

| Qwythos-9B-Claude-Mythos-5-1M-Q6_K.gguf | 6.9 GB | Q6_K baseline |

| M-LWQ8-OptA-Q5_K_M.gguf | 6.2 GB | Quality champion |

| M-LWQ8-v2-Q5_K_M.gguf | 5.7 GB | Compact champion |

Key Takeaways

1. OptA formula transfers perfectly — same architecture + same imatrix = same optimal quantization. PPL improved over Q6_K as expected.
2. Qwythos baseline is higher than Qwable (7.5876 vs 7.4394 at Q6_K) — 500M reasoning tokens and different eval domain (Wikitext-2).
3. v2 has a wider gap on Qwythos (+0.067 vs +0.017 on Qwable) — IQ4_XS tensors impose ~0.05 PPL extra cost on this fine-tune.
4. OptA remains the recommendation — best PPL at comfortable 6.3 GB. v2 only if 5.7 GB is required.

References

• Qwythos-9B-Claude-Mythos-5-1M
• Qwable-9B-Claude-Fable-5-LWQ8-GGUF — full methodology, importance analysis, PPL results (applies to Qwythos 1:1)