---

license: apache-2.0

base_model: google/gemma-4-E4B-it

language:

• cy
• en

pipeline_tag: text-generation

tags:

• welsh
• cymraeg
• gemma4
• gguf
• llama.cpp
• quantized

datasets:

• HuggingFaceFW/fineweb-2
• techiaith/cardiff-university-tm-en-cy
• techiaith/llyw-cymru-cy-ogl
• techiaith/cofnodycynulliad_en-cy
• techiaith/banc-trawsgrifiadau-bangor

---

<p align="center">

<img src="cymraeg-ar-yr-ymyl.jpg" alt="Cymraeg ar yr Ymyl" width="360">

</p>

Gemma 4 E4B Cymraeg v4 (GGUF)

Welsh on the edge

Quantised builds of a conversational Welsh model small enough to run offline on a laptop, a phone or a Raspberry Pi class board with llama.cpp or LM Studio. Q6_K needs about 7 GB of memory, Q4_K_M about 5 GB. No internet, no data centre, just Welsh.

GGUF quantisations of

EryriLabs/gemma-4-e4b-cymraeg-v4,

a conversational Welsh fine-tune of google/gemma-4-E4B-it that retains

English.

| quant | file | size | sha256 |

|---|---|---:|---|

| Q8_0 | gemma4-e4b-cymraeg-v4-Q8_0.gguf | see MANIFEST.json | see MANIFEST.json |

| Q6_K | gemma4-e4b-cymraeg-v4-Q6_K.gguf | see MANIFEST.json | see MANIFEST.json |

| Q4_K_M | gemma4-e4b-cymraeg-v4-Q4_K_M.gguf | see MANIFEST.json | see MANIFEST.json |

Q6_K or better is the recommendation. Each quantisation was validated at

temperature 0 against a corruption-prone prompt subset; sha256 sums and the

corruption counts for every quant are in MANIFEST.json. Read that evidence

before trusting the Q4_K_M build for anything where Welsh accuracy matters ,

it is provided for tight memory budgets, not as the default choice.

Deployment detail is in README-DEPLOY.md.

How to run

llama-server -m gemma4-e4b-cymraeg-v4-Q6_K.gguf --jinja -c 4096 -ngl 99

--jinja is required. Without it llama.cpp substitutes a generic chat

template, the Gemma 4 thought channel is never closed, and replies come back with

empty content after the whole token budget is spent. LM Studio has shown the

same failure; if answers arrive empty, this is the cause.

Keep thinking off (the shipped chat template's default).

Recommended sampling:

| setting | value |

|---|---|

| temperature | 0.3 |

| top_p | 0.9 |

| min_p | 0.05 |

| repeat_penalty | 1.05 |

Fraud-related facts (contact numbers, organisation names, reporting routes)

should be supplied through the system prompt or retrieved context rather than

relied on from the model's memory: see README-stopthinkfraud.md.

Evaluation

Blind LLM-as-judge scoring (Opus), identical prompts and settings for every arm

(nf4, thinking off, greedy, 400 max tokens, no system prompt). The judge is not

a native speaker; structural findings (language choice, loops, checkable facts)

are reliable, fine idiom judgements less so: native review welcome.

Single-turn, 40 Welsh prompts

| model | replies in the requested language | fluency /5 | grammar /5 | task /5 |

|---|---|---|---|---|

| stock gemma-4-E4B-it | 47.5 % | 2.46 | 2.38 | 1.45 |

| v3 (SFT only) | 97.5 % | 3.87 | 3.59 | 2.33 |

| v4 (CPT 43.8M tokens + SFT) | 100 % | 3.68 | 3.76 | 2.70 |

Pairwise v4 vs v3 (blind): 23 wins, 13 ties, 4 losses. English is fully

retained (fluency 5.0, task 4.95: best in the family). Invented Welsh words

roughly halved versus v3; the corpus-installed mutations (i Gaerdydd,

i Fangor, dy fore di) are v3's exact failures now correct.

Multi-turn (14 fixed turns, history kept, sampling as recommended above)

| model | conversation score /5 |

|---|---|

| stock | 1.97 |

| v3 | 3.35 |

| v4 | 3.99 |

Quantisation: Q6_K scored highest of the three quants under llama.cpp

(fluency 4.42, grammar 4.36); Q4_K_M showed no measurable damage; all three

quants: 0 corrupt tokens on the corruption-prone subset.

Known limitations: occasional repetition loops on long open-ended answers

(mitigated by repeat_penalty); factual recall (dates, statistics) is weak ,

supply facts via the system prompt / retrieval; when given long retrieved

context it can over-quote it. Fraud-safety facts must not be relied on from

memory. Full judgement report and per-response scores:

eval/judgement_e4b_v4.md, eval/scores_e4b_v4.jsonl in the base repository.

Comparison with other open Welsh models: see the note on the base

repository: techiaith (Bangor University) and BangorAI publish the only other

open Welsh LLMs to compare against, and their open datasets are part of this

model's training data, with thanks. A head-to-head against

BangorAI/Mistral-7B-Cymraeg-Welsh-v2 on this exact protocol is in the section

below.

Head-to-head with BangorAI Mistral-7B-Cymraeg-Welsh-v2

BangorAI/Mistral-7B-Cymraeg-Welsh-v2

is the strongest open Welsh model we know of and the only open comparison in the

7B class, so it is the bar this model is measured against. It is a genuinely good

Welsh model, and everything below should be read as a measurement against a high

bar rather than as a criticism of it. Diolch to BangorAI for publishing it.

It is not a like-for-like comparison, and it is not meant to be:

| | this model (v4) | BangorAI 7B v2 |

|---|---|---|

| parameters | 6.9B total, ~4B effective per token | 7B dense |

| Welsh continued pretraining | 43.8M tokens, 1 pass | MADLAD Welsh, 2 epochs, on the order of 412M token-passes |

| ratio | 1x | ~9.4x the Welsh tokens, ~1.75x the active parameters |

BangorAI was run with its own documented Welsh system prompt, which is its

intended interface, in the single-turn arm and in the multi-turn sys arm. The

multi-turn nosys arm is the properly matched comparison. Sampling,

quantisation, prompts and token limits were identical throughout.

Single-turn, 40 Welsh prompts plus 20 English prompts

| arm | replies in the requested language | Welsh fluency /5 | Welsh grammar /5 | task /5 | collapses | English task /5 |

|---|---:|---:|---:|---:|---:|---:|

| v4 | 40/40 | 3.68 | 3.76 | 2.70 | 5 | 4.95 |

| BangorAI 7B v2 | 37/40 | 4.85 | 4.65 | 3.35 | 0 | 4.20 |

Pairwise on the same 40 prompts: BangorAI 23 wins, 13 ties, v4 4 wins. That is

the mirror image of v4 against v3. BangorAI's Welsh essentially never breaks:

zero degeneration loops and zero corrupt word-forms anywhere. v4's four wins are

the three prompts that ask for English output, which BangorAI answers in Welsh,

plus one conversational prompt. v4 leads on task in translation (3.50 against

2.00) and assistant chat (3.43 against 2.86); BangorAI leads everywhere else.

Multi-turn, 14 fixed turns with history kept

| arm | fluency | grammar | relevance | conversational | memory | mean |

|---|---:|---:|---:|---:|---:|---:|

| v4 / sys | 4.07 | 4.14 | 3.21 | 3.36 | 3.69 | 3.70 |

| v4 / nosys | 4.29 | 4.14 | 3.79 | 3.86 | 3.85 | 3.99 |

| BangorAI / sys | 4.79 | 4.93 | 3.36 | 3.00 | 2.92 | 3.80 |

| BangorAI / nosys | 4.64 | 4.93 | 3.21 | 2.93 | 3.00 | 3.74 |

Per-turn wins, ties and losses:

| condition | v4 | tie | BangorAI |

|---|---:|---:|---:|

| sys vs sys (each with its own intended prompt) | 5 | 2 | 7 |

| nosys vs nosys (matched) | 5 | 4 | 5 |

| combined, 28 turns | 10 | 6 | 12 |

v4 wins the turns that hold a conversation together: the opening greeting, the

summary, the close (both v4 conditions produce an eleven-turn callback,

"Mwynha dy daith i Aberystwyth", which no BangorAI condition does), and above

all turn 11, the request to switch to English. v4 switches cleanly and

translates faithfully in both conditions; BangorAI stays in Welsh in both, and

under its own system prompt it states in Welsh that it can only help in English.

BangorAI wins the turns that reward content and polish: the tips, the facts, the

joke, the fraud advice and the poem.

Verdict in three lines. BangorAI wins overall and wins Welsh polish

decisively: it is more fluent, more grammatical and never degenerates, and if you

want the best open Welsh output and can run a 7B, use it. The conversation is far

closer, and in the matched condition v4 takes it on the arm mean (3.99 against

3.74) and draws on turns, so roughly 4B effective parameters trained on a tenth of

the Welsh fights a 7B to a draw over fourteen turns. What the extra scale and data

buy is surface quality and degeneration resistance; what they do not buy is

memory, register mirroring or switching language on request, which is why a small

bilingual model still has a place on edge devices.

The 12B flagship gemma-4-12b-cymraeg-v6

has since won a pre registered 100 prompt conversational benchmark against BangorAI's

7B, 66 to 33; see that repo for method and results.

Caveats. The judge is an LLM (Opus) and not a native Welsh speaker;

structural findings are reliable, fine idiom judgements are not, and a non-native

judge is least reliable exactly where BangorAI wins most, so the fluency gap is

more likely understated than overstated. Sizes and Welsh token counts are not

matched. BangorAI ran through its own system-prompt interface, so only the

multi-turn nosys comparison is matched on prompt text. The rubric was written

for a friendly conversational model, which favours v4 on register and warmth.

Sample sizes are small. Full report and per-item scores:

eval/judgement_v4_vs_bangorai.md and eval/scores_v4_vs_bangorai.jsonl in the

base repository.

Limitations

• Quantisation below Q6_K measurably increases degradation inside Welsh text;

read the corruption counts in MANIFEST.json before choosing Q4_K_M.

• Welsh costs more tokens per word than English for this tokeniser, which

reduces the effective context length available for Welsh text relative to

English.

• Fraud-related facts (numbers, organisation names, reporting routes) can be

misremembered or go stale; supply them through the system prompt or retrieved

context rather than relying on the model's memory.

• No tool-calling and no guarantee of factual accuracy.

Licence and attributions

Apache-2.0, inherited from google/gemma-4-E4B-it (trained from the

unsloth/gemma-4-e4b-it mirror). Commercial use is permitted. Training sources

and their individual licences are listed in DATA-CARD-v4.md and

README-stopthinkfraud.md in the

base repository;

where a source licence requires attribution (CC-BY-4.0, ODC-By 1.0, OGL v3),

that requirement passes through to users of these files.

---

Built 2026-08-18.