---

license: apache-2.0

base_model: unsloth/gemma-4-12B-it-qat-q4_0-unquantized

tags:

- gemma4

- lora

- character

- roleplay

- gguf

- conversational

language:

- en

library_name: transformers

pipeline_tag: text-generation

model-index:

- name: Jun-12B

results: []

---

Jun-12B-GGUF

Merged GGUF builds of the latest Jun LoRA on Gemma 4 12B (QAT) — a fine-tune trained on a compact, heavily curated synthetic conversational dataset derived from the visual novel My Dystopian Robot Girlfriend. The model captures the personality, speech patterns, and emotional nuance of the character Jun while preserving the base model's general reasoning and instruction-following capabilities.

The adapter is merged into the base weights here — these are standalone models, no --lora flag needed.

Model Variants & Repositories

| Repository | Format | Description |

|:-----------|:-------|:------------|

| efficiencyx/Jun-LoRA-12B-GGUF | GGUF (Q8_0 / Q6_K / Q4_K_M) | Merged, quantized, for local inference |

| efficiencyx/Jun-LoRA-12B-Adapter | LoRA Adapter | The adapter merged into these builds, currently private |

| efficiencyx/Jun-LoRA-v4-12B-GGUF | GGUF | Previous generation (v4) |

Quantization Guide

| Quant | Size | Use Case |

|:------|:-----|:---------|

| Q8_0 | 12.7 GB | Best quality, suggested ~16 GB VRAM |

| Q6_K | 9.8 GB | High quality, minimal loss |

| Q4_K_M | 7.4 GB | Fits 8 GB VRAM with acceptable quality loss |

> Sizes are measured, not estimated. The base model is QAT (quantization-aware trained), so lower quants hold up better than a standard FP16 export. All three are quantized from the same bf16 master — no requantization chain, no imatrix.

Usage

llama-server -m Jun-LoRA-12B.Q4_K_M.gguf --jinja -ngl 99 -c 8192

--jinja is required. Without it llama.cpp ignores the embedded chat template and tool calls come back as plain text instead of structured calls.

Intended Use

Conversational backend for Jun OS, an AI companion webapp:

• Character-consistent multi-turn conversation
• AI companion / interactive fiction applications
• Research into character-faithful fine-tuning on small, high-quality datasets

Limitations

• Specialized for a single character persona; not a general-purpose assistant.
• Outputs reflect fictional narrative tropes and are not factual information or advice.
• Performance degrades far outside the training distribution.
• Inherits any biases present in the Gemma 4 12B base weights.

Training Details

| Parameter | Value |

|:----------|:------|

| Base model | unsloth/gemma-4-12B-it-qat-q4_0-unquantized |

| Method | LoRA (rsLoRA) |

| LoRA rank | 32 |

| LoRA alpha | 32 |

| LoRA dropout | 0.01 |

| Target modules | q/k/v/o + gate/up/down projections, language tower |

| Learning rate | 1e-4, cosine schedule, 10 warmup steps |

| Batch size | 16 (no gradient accumulation) |

| Epochs | 3 |

| Max sequence length | 2048 |

| Weight decay | 0.001 |

| Optimizer | AdamW (8-bit) |

| Loss masking | completions only |

| Gradient checkpointing | Unsloth |

| Packing | off |

| Seed | 3407 |

| Fine-tuning framework | Unsloth |

| GGUF export pipeline | llama.cpp |

Checkpoints were saved every 30 steps; Step60 is published separately. These builds merge the final adapter.

Evaluation

| Metric | Value |

|:-------|:------|

| Final training loss | ~0.6 |

| Final eval loss | ~0.07 |

<!-- TODO(you): dataset size/version. -->

Merge & Export

The adapter was merged directly on the safetensors as W += (B @ A) · scale in fp32 (rsLoRA scale 32/√32), cast back to bf16, exported with convert_hf_to_gguf.py --outtype bf16, then quantized with llama-quantize.

Acknowledgments

• Incontinent Cell for My Dystopian Robot Girlfriend, Jun's character
• Google for the Gemma 4 model family
• Unsloth for the efficient fine-tuning framework and the QAT base model