---

license: gemma

base_model: Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA-GGUF

library_name: peft

pipeline_tag: text-generation

language:

• en
• ru

tags:

• lora
• roleplay
• sillytavern
• english
• russian
• gemma4
• moe
• conversational

---

Dark-Goetia-26B-A4B-LoRA-v2

Atmospheric genre LoRA adapter primarily for English-language roleplay (also works in

Russian). Adds a darker, more literary tone to the narrative. Designed exclusively to

adjust style and response structure in Dark Fantasy RP. The adapter contains no plots or

characters from the training data.

This is the general-purpose release. Two experimental variants exist that split this

adapter's target set in half — see Related adapters at the bottom.

Adapter versions

• goetia-book-lora-v2-main — 2 epochs → use the recommended scales below
• goetia-book-lora-v2-chk177 — 1 epoch → use 1.5× the scale from the table

Example: if you run main at 0.55, run chk177 at 1.1.

Recommended scale settings (main ver.)

Test setup: SillyTavern + Marinara's preset + 2000+ token character card.

The right ceiling depends on what your card asks for. Free-form prose RP tolerates

much more than a card that demands structured output blocks (status trackers, World State

sections, inner-thought blocks). Both regimes below are measured, not guessed.

Free-form prose RP — no strict formatting requirements

| Scale | Effect |

| --- | --- |

| 0.1–0.2 | Effect barely noticeable |

| 0.2–0.3 | Minimal / partial influence |

| 0.3–0.55 | Recommended — stable, controlled style |

| 0.55+ | Very strong influence, can start overpowering the base model |

Performs reliably in all RP scenarios at scales above 0.5. The darker literary tone

becomes most pronounced in Dark Fantasy settings. Best used in full roleplay sessions

(character card + system prompt); isolated short prompts show a weaker effect.

Cards with strict structured output

If your card requires the model to emit formatted blocks every turn, the ceiling is

considerably lower, and it differs by language:

| Language | Safe | First observed failure |

| --- | --- | --- |

| English | up to 0.40 | above 0.43 (estimated) |

| Russian | up to 0.37 | 0.40 |

Above these, the model keeps the style but starts silently dropping the formatted blocks.

Russian is measurably more fragile than English at identical adapter strength — if you

run bilingual sessions, tune to the Russian number.

Note on scaling: alpha/r = 2 is stored in the GGUF metadata, and llama.cpp multiplies

by it on top of your scale. All numbers above assume a loader that respects that. If

yours ignores the metadata, halve everything.

What's actually in the adapter

For anyone merging this or building on it — the trained delta isn't distributed evenly

across attention. Frobenius norms of the effective delta ΔW = (B·A)·(alpha/r), aggregated

over layers:

| Projection | ‖ΔW‖ |

| --- | --- |

| o_proj | 1.563 |

| q_proj | 1.314 |

| v_proj | 1.049 |

| k_proj | 0.828 |

| OV total | 1.882 |

| QK total | 1.553 |

| Full delta | 2.440 |

So roughly 61% OV / 39% QK by squared norm. That split turns out to matter: the OV

half carries almost all of the style, while both halves independently contribute to

formatting degradation — OV about 3× more readily per unit of weight change. The

structured-output ceilings in the table above are set by the OV component alone.

Training

• Base: Naphula/Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA (Gemma 4 MoE, abliterated)
• Method: QLoRA 4-bit, attention-only (q/k/v/o_proj), all 30 layers of the text tower,

r=32, alpha=64, 2 epochs, completion-only loss

• Scope: 115 attention projections, not 120 — layers 5, 11, 17, 23 and 29 are

global-attention layers with no v_proj at all. 22,978,560 trainable params (0.089%).

• Data: Novel broken down into scenes → scenes rewritten by a neural network into a

synthetic RP dataset (actions and internal thoughts preserved, original author's prose

style removed). Contains both English and Russian samples, English-primary.

Limitations

• 18+. Base model is abliterated, training data includes adult and dark content.
• Primarily optimized for English. Works in Russian, but the style effect is generally

weaker and instruction-following is more fragile there — see the scale tables.

• Structured-output thresholds were measured with one generation per setting at a single

context depth. Treat them as calibrated starting points, not hard constants.

• Inherits Gemma license terms.

Related adapters

v3 splits this adapter's 115 targets into two non-overlapping halves, trained separately

on identical data with identical hyperparameters and seed:

• v3-A — v_proj + o_proj

(55 modules). The style channel on its own, without the QK component. Experimental;

useful if your card demands strict formatting and v2 keeps dropping it.

• v3-B — q_proj + k_proj

(60 modules). Research artifact — weak style, published for reproducibility.

A + B = exactly this adapter's target set and parameter budget.