---

base_model: ambrosfitz/gemma4-ushistory-apush-v2

tags:

• gguf
• gemma

---

gemma4-ushistory-apush-v2-GGUF

GGUF quantizations of ambrosfitz/gemma4-ushistory-apush-v2

(epoch 3, merged 16-bit), for local inference via llama.cpp / LM Studio.

| File | Size | Notes |

|---|---|---|

| gemma4-ushistory-apush-v2-e3-Q4_K_M.gguf | 7.38 GB | Structural format adherence weaker than Q5 on this smoke test (see below) |

| gemma4-ushistory-apush-v2-e3-Q5_K_M.gguf | 8.55 GB | Recommended — cleaner structural adherence, verified 7-8/8 on smoke test |

Why epoch 3, not epoch 2

Per-epoch intra-set duplicate rate during training (HF Transformers, from the

source repo's TRAIN_REPORT.md):

| Epoch | greedy | temp 0.7 |

|---|---|---|

| 1 | 0.133 | 0.100 |

| 2 | 0.100 | 0.067 |

| 3 | 0.150 | 0.067 |

Epoch 2 scored better on greedy duplicate rate (0.100 vs 0.150); temp 0.7 was

tied (0.067). Epoch 3 was the chosen conversion target per this project's

handoff, not because it measured better on this specific metric. The epoch-2

adapter remains available in the source repo (adapter/epoch2/, with full

optimizer/scheduler state) if a re-convert from that checkpoint is ever wanted.

Required generation recipe

This model was trained and evaluated with a specific chat template and stop-token

setup, inherited from the same lineage as

gemma4-ushistory-sft-parityfix-GGUF.

Getting either wrong produces malformed or garbage output.

• Chat template: gemma-4, non-thinking mode. The correct jinja template

(verified to contain the strip_thinking macro and emit <turn|> correctly) is

embedded in this GGUF's own metadata (tokenizer.chat_template) — confirmed

present and intact via gguf.GGUFReader directly on the conversion pod. **Do

not override it** with a runtime's built-in "gemma" preset — llama.cpp ships a

built-in template named gemma for the classic Gemma 1/2/3 format

(<start_of_turn>/<end_of_turn>), incompatible with this checkpoint's

Gemma-4 format. Leave the chat template on default (LM Studio: don't override

it; llama.cpp: don't pass --chat-template).

• Stop token: <turn|> (vocab id 106, confirmed present in the GGUF's vocab

at that id). This is not encoded as an EOS/EOG token in GGUF metadata —

tokenizer.ggml.eos_token_id is [1] (<eos> only) — confirmed directly on

the conversion pod. **You must set <turn|> as an explicit stop string in your

runtime** (LM Studio: add it under the model's stop-string/sequence setting).

This was verified end-to-end at generation time (see below): once given an

adequate token budget, both quants stopped cleanly at <turn|> on 100%/88% of

smoke-test items — the mechanism works, but only because the stop string is set

explicitly. Without it, generation would run past the intended turn boundary.

Verification

Both quants were served on-pod (RTX 3090) via llama-server (no --chat-template

override) and smoke-tested against 8 real held-out stimuli — the identical

seed=42 validation split used for this model's own epoch-by-epoch training-time

sanity generations — at both greedy and temperature 0.7/top_p 0.95, with

stop=["<turn|>"] explicit in every request.

| Quant | Setting | Parse | Structural | Stop-clean | Dup rate |

|---|---|---|---|---|---|

| Q4_K_M | greedy | 8/8 | 3/8 | 8/8 | 0.150 |

| Q4_K_M | temp0.7 | 8/8 | 4/8 | 8/8 | 0.013 |

| Q5_K_M | greedy | 8/8 | 7/8 | 7/8* | 0.171 |

| Q5_K_M | temp0.7 | 8/8 | 7/8 | 7/8* | 0.073 |

\* Both Q5_K_M "unclean" cases are the same single stimulus whose natural

response (5-6 fully-explained questions) exceeds even a 2048-token budget —

confirmed well-formed up to the cutoff, i.e. a budget limit, not a stop-token

failure.

**The <turn|> stop-token risk this conversion was built around did not

materialize**: once given an adequate token budget, Q4_K_M reached 100%

stop-clean and Q5_K_M 88% (with the sole "failure" being budget exhaustion on an

unusually long response, not a metadata problem). Duplicate rates on both quants

(0.013–0.171) are in the same range as this model's own HF Transformers

training-time numbers (0.067–0.150), with Q4_K_M-greedy matching almost exactly

(0.150) — no evidence quantization introduces or worsens repetition.

**Q4_K_M does show real, reproducible structural degradation that Q5_K_M mostly

avoids**: on specific stimuli, Q4_K_M consistently produces a well-formed

numbered question set with 4 choices each, but omits Correct:/Explanation:

entirely for every question, identically at both greedy and temp 0.7 (ruling out

decoding-randomness noise). Q5_K_M shows almost none of this. This was not

cross-checked against the unquantized fp16 model on these same stimuli, so

whether this is introduced by Q4_K_M's quantization specifically or is a

pre-existing model weak point on this stimulus type is an open question — flagged

rather than overclaimed. Full per-item results and raw malformed outputs in this

repo's gguf_verification.md.

Decision: start local use from Q5_K_M — Q4_K_M showed malformed output on

this smoke test that Q5_K_M did not.

Recommended local config

• Full GPU offload (-ngl 99 or LM Studio's "GPU offload: max") — both quants fit

comfortably on a 12GB card

• Context length: -c 4096
• Chat template: leave on default / embedded (see above — do not override)
• Stop string: add <turn|> explicitly (see above — not in metadata)
• Given the structural-adherence gap above, prefer Q5_K_M for anything beyond

casual testing

Task

Source excerpt (primary or secondary source) in → a set of AP US History-style

multiple-choice questions out (2-8 questions per set observed in training data,

each with a stem, 4 choices A-D, a Correct: key, and an Explanation:).

Training provenance

• Base: Gemma 4 (12B), continued from ambrosfitz/gemma4-ushistory-sft-parityfix
• SFT: LoRA (r=32, alpha=64) on ambrosfitz/apush-synthetic-mcq (895 synthetic

train / 47 held-out val, seed=42; a separate 286-set authentic eval split

exists in the dataset for answering-accuracy evaluation, not used in this

generation-focused fine-tune)

• Masking: Unsloth's train_on_responses_only (marker-string based, independent

of chat-template {% generation %} tags), with a blocking step-0 mask gate

that verified the mask structurally correct before training started —

assistant_only_loss was deliberately left unset, unlike the prior

gemma4-ushistory-apush-continued run whose masking status could not be

determined post-hoc

• Result: intra-set duplicate rate dropped roughly an order of magnitude versus

the prior (mask-unverified) run, across all 3 epochs

• Merged epoch-3 model → this GGUF conversion

Full training report: TRAIN_REPORT.md

in the source repo.