d-scentre/Nightingale-G4-E4B-Beta2-GGUF overview

---

license: apache-2.0

base_model: llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic

base_model_relation: finetune

---

---

Greetings, curious HF surfer! What you're looking at now is the preamble of a small dump of my latest QLoRA experiences with gemma-4-E4B-it on Axolotl, more precisely the gemma-4-E4B-it-ultra-uncensored-heretic variant, even if this is technically a model card.

This kind of "backyard training report" is basically non-existent for Gemma 4 models as things stand (not to mention that there's apparently no big-shot attempt at training on the pretrained, actual base models - not that it's really viable). I desperately wanted to find just a single example, failed miserably, so here goes my findings. Hope it helps someone somewhere.

DISCLAIMER: This release is, as stated, heretic-based. It has no refusal behavior, and not appropriate for unsupervised deployment. Also if your primary language isn't English, expect degraded output (see below).

Man, do I miss getting excited over Kobold getting 8-bit support to run Pygmalion 6B V8...

/-/-/-/-/

Here's the current config before anything:

<details>

<summary>Axolotl config (click to expand)</summary>

%%writefile /content/config.yaml

base_model: llmfan46/gemma-4-E4B-it-ultra-uncensored-heretic

plugins:
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
  - axolotl.integrations.liger.LigerPlugin

strict: false
cut_cross_entropy: true

liger_layer_norm: false
liger_rope: true
liger_rms_norm: true
liger_glu_activation: true
liger_rms_norm_gated: true

load_in_4bit: true
adapter: qlora

# Permanent reasoning-template fix
chat_template: jinja
chat_template_jinja: /content/gemma4_training_directkey.jinja2

eot_tokens:
  - "<turn|>"

datasets:
  - path: /content/data/MORNINGINGALE-FIXED.jsonl
    type: chat_template

    chat_template: jinja
    chat_template_jinja: /content/gemma4_training_directkey.jinja2

    field_messages: messages
    field_thinking: reasoning_content
    template_thinking_key: reasoning_content

    message_property_mappings:
      role: role
      content: content
      reasoning_content: reasoning_content

    roles_to_train:
      - assistant

    train_on_eos: none
    train_on_eot: turn

dataset_prepared_path: /content/prepared_reasoning_directkey
dataset_num_proc: 1
val_set_size: 0.02

output_dir: /content/drive/MyDrive/zulfaris/outputs/zulfaris-e4b
save_safetensors: true

sequence_len: 15360
sample_packing: false
pad_to_sequence_len: false

lora_r: 64
lora_alpha: 64
peft_use_rslora: true
lora_dropout: 0.1

freeze_mm_modules: true

lora_target_modules: 'model.language_model.layers.[\d]+.(_checkpoint_wrapped_module.)?(mlp|self_attn).(up|down|gate|q|k|v|o)_proj'

lora_mlp_kernel: false
lora_qkv_kernel: false
lora_o_kernel: false

wandb_project: OPERATION-NIGHTINGALE
wandb_name: Nightingale-G4-E4B-beta7

micro_batch_size: 1
gradient_accumulation_steps: 8
num_epochs: 3

optimizer: adamw_torch_fused
lr_scheduler: rex
learning_rate: 2e-5
warmup_ratio: 0.1
max_grad_norm: 5.0
weight_decay: 0.0

bf16: auto
attn_implementation: sdpa

gemma4_hybrid_attn_impl: false
fused_attn_kernel: false

logging_steps: 1
evals_per_epoch: 4
saves_per_epoch: 4
save_total_limit: 4

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: false

</details>

BASE MODEL + DATASET STUFF

llmfan46's "ultra uncensored heretic" version to get rid of ridiculous levels of overzealousness even in basic chit-chat, and a mortal adapter trained on private data of ~5MB I spent 3 years to craft, collect, polish, revise and perfect.

Brief sum of dataset contents: 1.3K of handwritten entries (at least 90% being my own product), mostly complex Turkish contexts that require minimum levels of slop (hence the entirety of data being manually revised for thousands of doom-hours...), lots of such English contexts as well. Heavy focus on sense of humor + prose style in different contexts + situational awareness in multi-turn + human-like storywriting + being "Claude-like engaging" minus its slop tics + handwritten reasoning traces where frontier models couldn't satisfy or understand the prompt's/chat's demands. Some quirky STEM + coding + RP chats are also present.

Did the model learn these patterns? Well, simply put: Should have fried its small brain harder, since it's currently literally hit or miss.

TEMPLATE

This model wants to see its previous reasoning blocks at all costs. Otherwise it's happily going back to its Gemma voice.

You can find the jinja template on the "files" tab.

SAMPLER SETTINGS

Default Gemma 4 works just fine. Temp 1, top-p 0.95, top-k 64.

GEMMA 3 VS THIS

Gemma 3 12B's pure base form was extremely easy to train on compared to any Qwen 2/2.5/3/3.5 model I've tested at similar sizes. Albeit rather dumb, it would simply take my data as its holy book. Never tried its -IT version as a base though.

Gemma 4, OTOH, is... well.

Ran about +40 runs to this day, constantly changing weight decay between 0.0–0.05 + max_grad_norm between 0.1–8.0 + dropout between either 0.0 or 0.1 + LR between 8e-6–2e-5. Always kept R64/A64 and rsLoRA intact. Had to monkeypatch Axolotl to always preserve thinking blocks.

Basic behavioral observations based on train/loss and eval/loss graphs plus my own experimenting with all:

• Weight decay HINDERS learning for this specific model. Causes increase in both train loss and eval loss around ~15-30%. Prose adaptation fails the worst.
• max_grad_norm is ought to be higher than 3 for proper divergence for out-of-distribution tasks. 0.1 was just sunglasses on the base -IT.
• Dropout had no meaningful effect on both graphs and behavior, so I leaved it on in any case.
• LR lower than 2e-5 consistently resulted in substantially undercooked models.

EVAL/TRAIN GRAPHS FOR THIS RELEASE

!{8692F35B-E9A4-48BE-9A7F-5BD233B5E65D}

WHAT TO MAKE OF THIS

Gemma 4 has a distinctly strong stench of "Here's a thought process that leads to the provided solution..." and "Thinking Process:" in its thinking mode. In order to fully steer away from that direction, you must sell Gemma's soul and visit Hades to seek ghost trainers' advice that never made it out in the sunlight. SFT gets you only so far.

This version has partially failed to ensure a big shift in overall style over Gemma's default formatting ticks. I release it anyway, because there's no way out of this loop of "I SWEAR this is the last fix...". Next up we'll probably be looking at overcooked models to see if there's any chance we could override a large portion of its RL preferences in wording and formatting.

Even then, there are some positive findings:

• It IS possible to alter the default reasoning style to a degree
• The model still knows how to benefit from tools just as well despite almost no tool-specific training
• Training on heretic really helps the model simply focus on the context at hand instead of considering invisible morality rules (probs the most permanent feature of the OG -IT)
• Training on stream-of-consciousness thinking content consistently help the model to cut a significant portion of its thought process with no visible laziness in the actual output

And a BIG downside: taking heretic as a base made the model's Turkish (and likely other non-English languages as well) patterns extremely fragile regardless of hyperparameter configuration. For some obscure reason, the only reliable way out of this "drugged Turkish" style appears to be just submitting to your fate and go for the official instruct tune, which has about ~20% higher CE loss on both train and eval sets, which likely stems from its refusal check habits.

...

<details>

<summary>Example chat with thinking (click to expand)</summary>

User:

Write a chapter set in 17th century France, following a young and ambitious swordsman named d'Artagnan as he navigates the dangerous world of royal politics, intrigue, and duels. In this scene, d'Artagnan is eager to gain favor and prove himself, seeking an audience with a prominent high-ranking figure who might provide him with a letter of recommendation or support. He encounters the three seasoned musketeers, Athos, Porthos, and Aramis, who have a deep camaraderie and are veterans of numerous conflicts, but they view d'Artagnan with a mix of curiosity and suspicion. Their conversation is cordial yet charged, with subtle hints of rivalry and potential friendship. The language is elegant but sharp, full of wit and banter, as they test each other's wit, honor, and loyalty. The setting is a lavish palace or an ornate, historical French city square, with a tense but lively atmosphere. As tensions rise, the chapter should culminate in a challenge or invitation that may lead to a duel or a further test of d'Artagnan's abilities. The tone should be adventurous, full of swashbuckling energy, with a mix of lightness, tension, and formality typical of the time period.

<details>

<summary>Model thinking</summary>

The user wants a chapter set in 17th century France. The main character is d'Artagnan, an ambitious young swordsman. He's trying to gain the favor of a high-ranking royal figure.

He meets three experienced musketeers: Athos, Porthos, and Aramis. They're suspicious of him but also curious. Their conversation should be formal yet witty, testing each other's honor and loyalty. The setting is a lavish palace or ornate square. The chapter should build tension and end with a challenge or invitation that might lead to a duel. The tone should be swashbuckling, adventurous, with 17th-century formality.

This sounds like the opening of The Three Musketeers.

Let's structure this:

1. D'Artagnan approaches the Musketeers in a public setting.
2. Their initial conversation: polite but probing questions about his intentions.
3. They subtly test each other's skills and honor through banter.
4. Athos (or another character) makes a suggestion or challenge that moves the plot forward.

I can write this in a style that reflects 17th-century French literature, focusing on dialogue and character interaction. The prompt seems to be asking for a classic swashbuckling adventure. I will make sure the dialogue reflects the time period and the Musketeers' character traits.

</details>

Assistant:

The courtyard of the Hôtel de Langes, stone-paved with worn flagstones that caught the weak light of early afternoon, was alive with the murmur of the city, the clip-clop of horses, and the distant shouts of merchants. Yet at a table set with fine linen near a window, separated from the throng by the discreet wall of the palace's exterior, three figures held court in a circle of hushed admiration. They wore the familiar scarlet and black of the Musketeers, cloaks of heavy wool draped over their broad shoulders. Their faces, weathered by seasons in camp and in the Parisian streets, betrayed nothing of the men they were to anyone else.

One of them, Athos, sat upright, hands resting loosely on the hilt of his rapier, eyes hooded by the shadow of his tricorne hat. The others, Porthos and Aramis, flanked him.

"So this is the newcomer, then," Porthos rumbled, his voice like rocks rolling in a mountain stream. He leaned forward slightly, hand on his belt. "D'Artagnan, you say. We heard whispers, whispers that spread in whispers among those we hold in our esteem. That he seeks the King's ear, that his ambition is as large as his sword."

Athos did not rise, but met D'Artagnan's gaze across the table. "Ambition, yes, that may be too modest a word for it. It is a hunger, I would say, a need to be seen, to be reckoned with. To be acknowledged by the high lords, and by the King himself."

"And you believe this desire is noble," Aramis said lightly, taking a sip of his wine. His smile did not quite reach his eyes, which seemed to catalogue every subtle sign in D'Artagnan's posture, every flicker in his expression. "To seek favor with the Crown is the path of the clever, I imagine. The paths of honest swordsmen, our kind, are more… open."

D'Artagnan took a breath. The air felt suddenly tighter. He had prepared for such conversation. He would not be frightened by their calm questions, which were more to assess, more to test him than merely to know him. He adjusted his cuffs, and met Athos's look levelly.

"Honest, yes, we are," D'Artagnan conceded without faltering. "If honor is what you claim to serve, then I have honor enough to say that I would not waste a minute seeking the favor of the King if it did not mean to serve France. Only France, for the greater part, not the court which starves it."

Porthos gave a short, surprised laugh, a sound devoid of humor. He set down his glass with a firm thud. "The court starves? How can it when we eat at the King's own table more often than some of the great Princes of the Blood themselves, if it is ever done?"

"By robbing it," D'Artagnan replied plainly, and the words, intended to be too blunt and therefore dismissed as boldness, were met instead with silence.

Aramis's expression changed. He placed his hand on the tabletop, and leaning slightly closer, he spoke softly, but with precision. "The most dangerous man is he who speaks openly. And the most loyal man is that who keeps his opinions to himself, reserving for a sudden eruption at a desperate moment." He then pushed a small plate of fine cakes toward D'Artagnan. "Tell me, swordsman, do you find it necessary to show your hand, or do you prefer the subtlety of one who waits until the exact moment to strike?"

"I find it necessary to show it now," D'Artagnan said, taking a small piece of cake. "For if I wait, they will not know what they deal in and will choose accordingly. That is why I want an audience with the Count; to show them not merely that I seek to serve France, but that I know well who France's true enemies are, even when they wear a king's cloak."

Porthos chuckled again, this time with more genuine mirth. "Now here is something worth talking about. You have wit, I grant you that. But wit does not purchase favor, sword. Courage in a duel does." He stood, reaching out a broad hand across the table and clasping D'Artagnan's arm with surprising weight. "I do not know you, and I do not believe you worth any further consideration by the Royal court unless you are able to prove, to my personal satisfaction, that you have the true heart of the sword, and the true mind of the man."

"Then where do you suggest I look," D'Artagnan asked.

Aramis smiled in the corner of his mouth, pulling his seat back abruptly. "I suggest he leaves. The court will not grant him an audience, but courage may earn him both glory and the favor of us all." He rose as well, with an air that made it clear that D'Artagnan was being dismissed on a somewhat deliberate, though ostensibly civil, pretext.

Porthos nodded in agreement, his large body blocking the door. "Your aim, my friend, may be too high for us all, even if we wish to commend you in spirit. We will not be a shield for such a bold man."

"You are wrong," D'Artagnan said, not rising, but looking Porthos straight in the eye. "I ask not for a shield, I ask only for a recommendation. A word, that my efforts are worthy."

Athos finally rose, his movements slow and deliberate, as if weighing the very air before he moved. "A word, then, that I will give you. But we must speak more fully of your intent, before it is granted. You will come to my place tomorrow night. And we will duel, friend. To test your wit in the sword, and your mind in the matter."

D'Artagnan's breath caught for an instant. He had hoped they would give him what he had come for, and had not expected this instead. "Why, this is an unexpected path for such a small favour."

"Courage first," Porthos said with a grin. "Swords second. For our purposes. You are not yet a friend of ours, and for now, you will be tested before you are accepted. If you win, we speak well. If you lose," he paused, allowing a slow smile to spread across his face, "then you must remain where you are, friend, and forever forget you ever sat at a table in the shadow of this palace."

D'Artagnan left without another word to them. The Musketeers' veiled meanings did not escape him, so it was not difficult to conclude that if he met their terms, he would gain the reputation he so desperately sought. He clutched the name of their camp and their seats within it in his mind. A different kind of game lay before him, one where the prize was far worthier than mere favor with a courtier.

---

User:

GENIUS STYLE OH LORD

<details>

<summary>Model thinking</summary>

"GENIUS STYLE OH LORD" is an emotional reaction to the previous response. It is not a request or a comment that requires specific action. It's a sign of satisfaction, but doesn't ask for further elaboration, a follow-up, or a correction. The user's use of all caps and interjections indicate strong positive sentiment about the provided chapter.

I do not need to provide a further response since this is not a valid conversation starter but a standalone exclamation.

</details>

Assistant: (no reply generated, see thinking block for why)

---

User:

Getting a "seen" from an AI was not on my bingo card lmao

<details>

<summary>Model thinking</summary>

The user's comment about the AI getting a "seen" suggests that they are acknowledging their positive reaction. It's a light-hearted comment, not a new request. No additional action needed.

</details>

Assistant: (no reply generated again)

---

User:

WTF BRO SAY STHG

<details>

<summary>Model thinking</summary>

The user is commanding a reply in a casual, abrupt way, possibly because they want to continue the conversation, or just because they find the minimal responses I've been giving odd, or maybe they're frustrated by the lack of conversation that their previous "genius" comment triggered.

I should engage in a more natural conversational manner, acknowledging their point, and asking about their continued interest in D'Artagnan or other topics, which might be what they are actually looking for.

So I'll offer a slightly more conversational response, acknowledging their previous comment and asking if they want to continue the story or discuss something else.

</details>

Assistant:

It seems the previous minimalist answers might be confusing, given you shared those comments. Apologies for the odd reaction to your "OH LORD" reaction.

So, would you like to continue D'Artagnan's adventures in the Musketeers? Or would you prefer discussing other aspects or topics entirely?

</details>

/-/-/-/-/

So here we are, a lost trainer with a tale narrating pain and suffering to you Odysseuses...