Koshkasa/TheDrummer_Skyfall-31B-v4.2_16G-checkerboard-trellis-GGUF overview
quantized for use with ik llama.cpp and its derivatives incompatible with mainline llama.cpp as of commit 34af94c What's that? Skyfall is a heavy hitter for ma…
Runs locally from ~14.51 GB disk (16 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| TheDrummer_Skyfall-31B-v4.2_16G-checkerboard-trellis.gguf | GGUF | GGUF | 14.51 GB | Download |
Model Details
| Model ID | Koshkasa/TheDrummer_Skyfall-31B-v4.2_16G-checkerboard-trellis-GGUF |
|---|---|
| Author | Koshkasa |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | TheDrummer/Skyfall-31B-v4.2 |
| Last modified | 2026-08-17T17:15:09.000Z |
Model README
---
license: apache-2.0
base_model:
- TheDrummer/Skyfall-31B-v4.2
library_name: ik_llama.cpp
pipeline_tag: text-generation
tags:
- gguf
- quantized
- ik_llama.cpp
- roleplay
- trellis
- imatrix
- mixed precision
quantized_by: Koshkasa
base_model_relation: quantized
datasets:
- Squish42/bluemoon-fandom-1-1-rp-cleaned
---
!!! quantized for use with ik_llama.cpp and its derivatives !!!
!!! incompatible with mainline llama.cpp as of commit #34af94c !!!
What's that?
Skyfall is a heavy hitter for mainline consumer GPUs. I have an abusive relationship with it - the goal of fitting the most of it into 16 GBs of VRAM, even without KVO, eluded me twice. I made two publicly avaliable ik_llama.cpp-exclusive quants of Skyfall before, each with their own issues. Lessons were learned. Most were forgotten.
It's time for Round 3.
The goal: shove Skyfall into 16 GB VRAM with system overhead, AGAIN, using ik_llama, hopefully make it all smart enough for a mixed quant.
The result: another hybrid quantization of TheDrummer/Skyfall-31B-v4.2, hopefully the final iteration
Rationale
Having a very limited budget for a 31b model (I run the system off the same GPU I use for ik_llama.cpp inference, so I am only safe within a 14.5 GB margin), I decided to play chess - literally. The recipe involves two layer precision schemae. The higher precision schema was used in 8 edge layers, and as every third middle layer as a drift prevention checkpoint.
The idea is simple. Keeping edge layers in higher precision should stabilize input comprehension/output composition (keeping them as unharmed as we can afford, given the size limits), and having checkpoints in an otherwise aggressive mixed 3/4bit ffn flow should let the model compensate for SOME quantization-borne drift. That, together with ik's advancements with trellis quantization and a bulkier imatrix dataset with an expanded RP corpus should keep the model sane. In theory.
Will it have a measurable mechanistic effect? Will such a ratio of layers be enough? Idk. That's an experiment and a love letter to Skyfall at the same time.
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Quant Recipe</title>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
background: #0d1117;
color: #e6edf3;
font-family: 'Segoe UI', -apple-system, sans-serif;
display: flex;
justify-content: center;
padding: 2rem 1rem;
}
.container {
background: #161b22;
border-radius: 12px;
padding: 2rem 1.5rem;
border: 1px solid #30363d;
max-width: 1100px;
width: 100%;
display: flex;
flex-direction: column;
align-items: center;
}
h1 {
font-size: 1.6rem;
font-weight: 600;
margin-bottom: 0.25rem;
color: #f0f6fc;
text-align: center;
}
.subtitle {
color: #8b949e;
font-size: 0.9rem;
margin-bottom: 1.5rem;
text-align: center;
}
.track {
display: flex;
flex-wrap: wrap;
align-items: flex-end;
justify-content: center;
gap: 3px 2px;
padding: 1rem 0.5rem;
background: #0d1117;
border-radius: 8px;
border: 1px solid #30363d;
margin-bottom: 1.5rem;
width: 100%;
min-height: 100px;
}
.block {
display: flex;
flex-direction: column;
align-items: center;
flex-shrink: 0;
}
.bar {
border-radius: 3px;
width: 22px;
transition: all 0.1s;
min-height: 6px;
}
.bar.io {
background: #2da44e;
height: 52px;
width: 32px;
}
.bar.high {
background: #1f6feb;
height: 40px;
}
.bar.low {
background: #da3633;
height: 20px;
}
.bar-label {
font-size: 0.5rem;
color: #8b949e;
margin-top: 4px;
font-family: 'JetBrains Mono', monospace;
letter-spacing: 0.2px;
}
.ellipsis {
display: flex;
flex-direction: column;
align-items: center;
color: #8b949e;
padding: 0 6px;
font-size: 1.2rem;
font-weight: 300;
letter-spacing: 2px;
user-select: none;
}
.ellipsis .dots {
font-size: 1.4rem;
line-height: 1.2;
}
.ellipsis .label {
font-size: 0.5rem;
color: #6e7681;
margin-top: 2px;
}
.legend {
display: flex;
flex-wrap: wrap;
justify-content: center;
gap: 1.5rem 2.5rem;
padding: 0.6rem 1rem;
background: #0d1117;
border-radius: 8px;
border: 1px solid #30363d;
margin-bottom: 1.5rem;
width: 100%;
}
.legend-item {
display: flex;
align-items: center;
gap: 0.6rem;
font-size: 0.8rem;
color: #c9d1d9;
}
.legend-swatch {
width: 28px;
height: 18px;
border-radius: 3px;
flex-shrink: 0;
}
.legend-swatch.green { background: #2da44e; height: 24px; width: 20px; }
.legend-swatch.blue { background: #1f6feb; height: 20px; width: 20px; }
.legend-swatch.red { background: #da3633; height: 12px; width: 20px; }
.panels {
display: flex;
flex-wrap: wrap;
justify-content: center;
gap: 1rem;
width: 100%;
margin-top: 0.5rem;
}
.panel {
flex: 1 1 200px;
min-width: 180px;
background: #0d1117;
border-radius: 8px;
padding: 0.8rem 0.6rem;
border: 2px solid transparent;
}
.panel.green { border-color: #2da44e; }
.panel.blue { border-color: #1f6feb; }
.panel.red { border-color: #da3633; }
.panel h4 {
font-size: 0.8rem;
font-weight: 600;
margin-bottom: 0.5rem;
text-align: center;
border-bottom: 1px solid #21262d;
padding-bottom: 0.3rem;
color: #f0f6fc;
}
.panel table {
width: 100%;
font-size: 0.7rem;
border-collapse: collapse;
}
.panel td {
padding: 0.2rem 0.1rem;
color: #e6edf3;
border-bottom: 1px solid #1c2128;
}
.panel td:first-child {
color: #8b949e;
font-weight: 400;
}
.panel td:last-child {
text-align: right;
font-weight: 600;
font-family: 'JetBrains Mono', monospace;
}
.panel .f32 { color: #58a6ff; }
.panel .iq6_k { color: #f0883e; }
.panel .iq5_k { color: #d29922; }
.panel .iq4_kt { color: #2da44e; }
.panel .iq3_kt { color: #da3633; }
.note {
margin-top: 1.2rem;
font-size: 0.75rem;
color: #6e7681;
background: #0d1117;
padding: 0.6rem 1.2rem;
border-radius: 6px;
border-left: 3px solid #1f6feb;
text-align: center;
width: 100%;
}
.note strong { color: #f0f6fc; }
@media (max-width: 700px) {
.container { padding: 1rem; }
.bar { width: 16px; }
.bar.io { width: 24px; height: 40px; }
.bar.high { height: 32px; }
.bar.low { height: 16px; }
.panel { flex: 1 1 100%; }
.legend { gap: 0.8rem 1.2rem; }
}
</style>
</head>
<body>
<div class="container">
<h1>Quant Recipe</h1>
<div class="track">
<div class="block"><div class="bar io"></div><span class="bar-label">Input</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">0</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">1</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">2</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">3</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">4</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">5</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">6</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">7</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">8</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">9</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">10</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">11</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">12</span></div>
<div class="ellipsis"><span class="dots">⋯</span><span class="label">13–48</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">49</span></div>
<div class="block"><div class="bar low"></div><span class="bar-label">50</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">51</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">52</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">53</span></div>
<div class="block"><div class="bar high"></div><span class="bar-label">54</span></div>
<div class="block"><div class="bar io"></div><span class="bar-label">Output</span></div>
</div>
<div class="legend">
<span class="legend-item">
<span class="legend-swatch green"></span> Input / Output
</span>
<span class="legend-item">
<span class="legend-swatch blue"></span> Higher Precision
</span>
<span class="legend-item">
<span class="legend-swatch red"></span> Lower Precision
</span>
</div>
<div class="panels">
<div class="panel green">
<h4>Embeddings</h4>
<table>
<tr><td>token_embd</td><td class="iq6_k">iq6_k</td></tr>
</table>
</div>
<div class="panel blue">
<h4>High-Precision Blocks</h4>
<table>
<tr><td>attn_k</td><td class="iq5_k">iq5_k</td></tr>
<tr><td>attn_q</td><td class="iq5_k">iq5_k</td></tr>
<tr><td>attn_v</td><td class="iq6_k">iq6_k</td></tr>
<tr><td>attn_output</td><td class="iq6_k">iq6_k</td></tr>
<tr><td>ffn_up</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>ffn_gate</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>ffn_down</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>attn_norm / ffn_norm</td><td class="f32">f32</td></tr>
</table>
</div>
<div class="panel red">
<h4>Low-Precision Blocks</h4>
<table>
<tr><td>attn_k</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>attn_q</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>attn_v</td><td class="iq6_k">iq6_k</td></tr>
<tr><td>attn_output</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>ffn_up</td><td class="iq3_kt">iq3_kt</td></tr>
<tr><td>ffn_gate</td><td class="iq3_kt">iq3_kt</td></tr>
<tr><td>ffn_down</td><td class="iq4_kt">iq4_kt</td></tr>
<tr><td>attn_norm / ffn_norm</td><td class="f32">f32</td></tr>
</table>
</div>
<div class="panel green">
<h4>lm_head</h4>
<table>
<tr><td>output</td><td class="iq6_k">iq6_k</td></tr>
</table>
</div>
</div>
<div class="note">
<strong>Pattern:</strong> Blocks <strong>0–3</strong> high · Middle: <strong>2 low + 1 high</strong> repeating ·
Blocks <strong>51–54</strong> high · Total: <strong>54 layers</strong>
</div>
</div>
</body>
imatrix generated using bartowski's combined dataset coupled with a random conversation 200k-token prune of Squish42/bluemoon-fandom-1-1-rp-cleaned.
quantized with ik_llama.cpp as of commit 43afea46c25a12aae6db1e3105643267164898b4 (Fri Aug 14 07:55:11 2026 +0200)
Cheers
MistralAI - the beloved base model(s).
ikawrakow and contributors of ik_llama.cpp - I probably misused your creation.
TheDrummer - you cooked a legendary one.
bartowski - for the combined dataset + the myriad of quants we all benefit from.
EmanuelOverride - for sparking my interest in custom quantization as a concept.
Run Koshkasa/TheDrummer_Skyfall-31B-v4.2_16G-checkerboard-trellis-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models