---

license: apache-2.0

license_link: https://ai.google.dev/gemma/docs/gemma_4_license

base_model: google/gemma-4-E2B-it

base_model_relation: quantized

pipeline_tag: text-generation

language:

- en

tags:

- on-device

- edge

- iphone

- offline

- tool-calling

- gguf

- q4_k_m

- llama.cpp

- imatrix

- calibration-aware-quantization

- gemma-4

- gemma-4-e2b

- on-device

- quantized

- fraqtl

inference: false

---

Gemma-4-E2B Q4_K_M GGUF — fraQtl Hi-Fi (calibration-aware quantization)

> 36% closer to the original model than the community Q4_K_M — at the same file size. The most faithful Gemma-4-E2B quant we've measured, on every arm we tested. Every number reproducible. Receipts inside.

Gemma-4-E2B Q4_K_M GGUF built with calibration-aware quantization: −36.0% code/math KLD and −27.2% general KLD to the original bf16 model versus the pinned community Q4_K_M baseline, at the same file size (−0.03%). A second 2.86 GB phone artifact is 17.5% smaller and still −31.8%/−26.7% versus that same-size baseline. Every number on this page is measured, pinned, and reproducible from the provenance table below.

!Fidelity receipts chart

Model Overview

Description

This repository contains two GGUF quantizations of google/gemma-4-E2B-it produced with fraQtl's calibration-aware per-tensor quantization: the same llama.cpp Q4_K machinery everyone uses, with per-tensor precision decisions driven by a calibration analysis instead of fixed defaults. The claim is smarter bits, not more bits.

| Artifact | Bytes | True bpw (all tensors) | KLD code/math (vs original bf16) | KLD general | Verdict vs baseline |

|---|---:|---:|---:|---:|---|

| Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf (iso-size) | 3,461,675,328 | 5.93 | 0.05846 (3-run, drift 0.0) | 0.09646 (3-run, drift 0.0) | −36.0% / −27.2% |

| Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf | 2,856,122,688 | 4.89 | 0.06231 | 0.09716 | −31.8% / −26.7% (and 17.5% smaller) |

| bartowski Q4_K_M (pinned baseline) | 3,462,678,272 | 5.93 | 0.09139 (3-run) | 0.13249 (3-run) | — |

Fidelity metric: symmetric top-20 KL divergence on Q8 teacher support, mean over positions; teacher is a Q8_0 quantization of the ORIGINAL bf16 checkpoint. Lower is better; this measures how faithfully the quantized model reproduces the original model's token distributions.

On-device: what it does on an iPhone (airplane mode)

These are unedited screenshots from our demo assistant running this model family entirely on an iPhone — on-device speech recognition, this model for understanding, deterministic Swift code executing the actions. The airplane icon in every status bar is the point: zero network at runtime.

The proof pair: the app says it booked — the Calendar shows it did

| | |

|---|---|

| !Voice-in booking with receipt chip | !The real event in Apple Calendar |

| Spoken request (note the waveform icons): "Find some time on Thursday for VC meeting." The model finds the free slot, the code layer writes the event, a receipt chip states exactly what was written — and the reply is read aloud by on-device TTS ("Speaking — tap to stop"). | Apple Calendar, same phone, still in airplane mode: "VC meeting", Thursday Jul 30, 11:30–12:30 — the exact event the chip claimed. Note the "◀ fraQtl" breadcrumb: navigated straight from the app. The app cannot claim a write that did not happen. |

Honesty, receipts, and speed

| | |

|---|---|

| !States its own limits | !Books from a typed request |

| Asked about email, it answers "I do not have access to your emails" with the context tag no connectors. Then a two-step instruction (move + alert) executes as two real calendar writes. | Typed request, same behavior: finds a free 90-minute block and writes the event, receipt chip naming calendar and time. |

| !Live receipts panel | !Benchmark gate pass |

| The receipts panel: model sha256-verified on load, live decode rate, resident memory, thermal state, network status. | In-app benchmark, 256 tokens greedy: 21.6 tok/s sustained against a stated gate of ≥15 tok/s — the number ships with its pass condition. |

!Offline code generation

Dijkstra's algorithm in C++, generated offline at ~11 tok/s interactive.

!Personality

It also has opinions. (Every answer still carries its context tag.)

Measured on an iPhone-16-class device: ≈516 MB resident · ~12 tok/s interactive (last turn, includes prefill) · 21.6 tok/s sustained on the 256-token greedy benchmark (gate ≥15: PASS) · sha256-verified artifact · llama.cpp b10075 · Metal · mmap · airplane mode throughout.

What it can do (all offline): voice input and spoken replies, calendar questions, find-a-slot-and-book, move/retitle events, chained reminders, contacts-aware event titles, reading documents and photos, general knowledge, code generation.

What it cannot do — by design and honestly labeled: no email, no messages, no network calls of any kind (there are no connectors, and it says so when asked); no action without a receipt chip — if a write fails, no chip appears; and the model never announces an action itself — a deterministic code layer executes and reports.

Artifact note: the demo runs Gemma-4-E2B-it-fraQtl-SmartEdge-IQ3XXS.gguf — in this repository now, 2.45 GB, sha256 761c7002…816776, byte-identical to the sha shown in the receipts panel screenshot above. What you download is exactly what the phone is running. Fidelity receipts (KLD ladder) for this edge variant are the next lane item; until they land, its published evidence is the on-device behavior and benchmark shown here. The Q4_K_M and Phone artifacts carry full fidelity receipts today.

License/Terms of Use

Apache 2.0 for the quantization artifacts and receipts in this repository. The model weights derive from google/gemma-4-E2B-it and remain subject to Google's Gemma terms (license link). Nothing here modifies the base model's own usage terms.

Use Case

On-device and edge inference of Gemma-4-E2B-it via llama.cpp and llama-cpp-python where fidelity to the original model matters at 4-bit-class sizes: assistants, code/math workloads, and phone/laptop deployments (the Phone artifact targets memory-constrained devices at 2.86 GB).

Release Date

Hugging Face 07/2026 via this repository.

References

• Serving lane (KV-cache compression for vLLM, same receipt discipline): Qwen3-4B kit · Mistral-7B kit
• All fraQtl artifacts: huggingface.co/fraQtl

Model Architecture

Base model: google/gemma-4-E2B-it (instruction-tuned E2B). The GGUFs were converted from the pinned bf16 checkpoint with llama.cpp's convert_hf_to_gguf.py --outtype bf16, then quantized with llama-quantize using per-tensor --tensor-type overrides selected by fraQtl's calibration-aware analysis on a code+math calibration packet. How the per-tensor decisions are made is not published; the recipe hash, the imatrix, and every artifact hash are, so the result is verifiable end to end.

Phone artifact: identical transformer quantization; the per-layer-embedding (PLE) table is additionally demoted to Q4_K. Speed-neutral by construction: bandwidth-relevant bytes (everything except the two lookup tables) are byte-identical between the two artifacts (1,188,795,532 B).

True-bpw honesty: "Q4_K_M" for E2B is really 5.3–5.9 true bits/weight because the 2.35B-parameter per-layer-embedding table dominates bytes. We publish true average bits/weight over all tensors AND transformer-resident ("bandwidth-relevant") bytes — E2B decode reads ≈1.15 GB of transformer weights per token; the PLE table is a ≈7 KB/token lookup. Size comparisons on this page are total bytes; speed-relevant comparisons are bandwidth-relevant bytes.

Input / Output

• Input: text (Gemma chat format)
• Output: text
• Context length: as the base model; evaluation on this page used ctx 2048 (calibration) and ctx 512 (perplexity sanity metric).

Calibration Dataset

• Dataset: fraQtl Apex code+math packet (sha256 9bec84a28dcb0c940047e6084561a00857fd610b8a1e148cc38e27929e0a7e02), decontaminated against the eval slices.
• Data collection method: Automated (curated public code/math text).
• Properties: full packet, 212 chunks / 434,176 tokens; importance matrix computed with llama-imatrix at ctx 2048 on the BF16 substrate. The exact imatrix.dat ships in this repository (sha256 892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c) so the quantization is reproducible.
• Recorded deviation from the 26B build pattern: single full-packet imatrix; no imatrix-budget sweep (E1 lean scope).

Usage

Receipt-validated stack: llama.cpp @ 78d2f524682d9fee790a6460c93d018dafeb5229 (2026-07-05) / llama-cpp-python==0.3.33 (vendors that commit). Newer releases should work; the pin is what every number on this page was measured with.

pip install llama-cpp-python==0.3.33
hf download fraQtl/Gemma-4-E2B-it-Hi-Fi-GGUF Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf --local-dir .

from llama_cpp import Llama

llm = Llama(model_path="Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf", n_ctx=4096, seed=0)
out = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain in two sentences why the sky is blue."}],
    temperature=0.0,
)
print(out["choices"][0]["message"]["content"])

Or with llama.cpp directly:

llama-cli -m Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf -cnv -p "You are a helpful assistant."

Chat template / turn-token note: the Gemma chat template is embedded in the GGUF — use create_chat_completion (or llama-cli -cnv) and let the runtime apply it. Gemma turns are delimited by <start_of_turn>user / <start_of_turn>model and terminated by <end_of_turn>, and the assistant role is named model, not assistant. If you hand-roll prompts, make sure <end_of_turn> is in your stop set: stacks that only stop on <eos> will run past the turn boundary and generate spurious extra turns.

Evaluation

All evaluation is fidelity to the ORIGINAL gemma-4-E2B-it (the axis a user migrating from the bf16 model experiences), on an A100-80GB with seed 0, deterministic settings, 3 independent runs where stated (run-to-run drift 0.0). Eval slices: code/math 400-record held-out (sha256 cce6860274ab715e3ab8493c9f817b1c29ff99c148f83a25de8944e027db1665); general chat/tool/wikitext 400-record (sha256 b10a79caf2c17cc10cd1edcae44d4655278baf5b09b6a867b4d3ade2f996b276).

Fidelity to the original model (KLD, lower is better)

| Arm | bytes | true bpw | KLD code/math ↓ | KLD general ↓ | top-1 agree (gen) |

|---|---:|---:|---:|---:|---:|

| bartowski Q4_K_M (community) | 3,462,678,272 | 5.93 | 0.09139 (3-run) | 0.13249 (3-run) | 0.888 |

| unsloth Q4_K_M (community) | 3,106,738,272 | 5.32 | 0.10348 | 0.14767 | 0.882 |

| google QAT q4_0 | 3,349,516,256 | 5.76 | 0.24730 | 0.35353 | 0.831 |

| unsloth qat UD-Q4_K_XL | 2,620,370,976 | 4.50 | 0.20666 (3-run) | 0.30275 (3-run) | 0.849 |

| fraQtl Hi-Fi (iso-size) | 3,461,675,328 (−0.03% vs baseline) | 5.93 | 0.05846 (3-run, −36.0%) | 0.09646 (3-run, −27.2%) | 0.905 |

| fraQtl Hi-Fi Phone (PLE-Q4_K) | 2,856,122,688 (−17.5%) | 4.89 | 0.06231 (−31.8%) | 0.09716 (−26.7%) | — |

QAT context (read before comparing QAT rows): the QAT GGUFs descend from google/gemma-4-E2B-it-qat-q4_0-unquantized — a retrained checkpoint. Unsloth's published E2B QAT mean KLD 0.00173 is measured against that retrained checkpoint, not against the original gemma-4-E2B-it. All rows above measure fidelity to the ORIGINAL model, which is the axis a user migrating from the bf16 model experiences. Both statements are true simultaneously; they answer different questions.

GSM8K

| Metric | fraQtl Hi-Fi (iso-size) | bartowski Q4_K_M | Verdict |

|---|---:|---:|---|

| GSM8K, 0-shot instructed chain-of-thought, n=200, seed 0, greedy | 86.0% (172/200) | 79.0% (158/200) | diff +7.0pp, 95% tie band ±7.42pp → statistical tie |

The +7.0pp difference is inside the ±7.42pp tie band (1.96·sqrt((p₁(1−p₁)+p₂(1−p₂))/n), recomputed for this model at n=200). We make no win claim on GSM8K. The protocol label is exact: 0-shot with a chain-of-thought instruction; no few-shot exemplars; not the full 1319-item set. Phone artifact: not run (lean scope).

Wikitext-2 perplexity (sanity flag — disclosed loss)

| Metric | fraQtl Hi-Fi (iso-size) | bartowski Q4_K_M | Verdict |

|---|---:|---:|---|

| Wikitext-2 PPL, ctx 512 | 20009.7 | 18005.3 | LOSS (flag) |

Disclosed honestly: our artifact is worse on raw-web perplexity. Two caveats, neither of which removes the flag: (1) both absolute values are harness-inflated — ctx-512 chunked raw text on an instruction-tuned model is a sanity metric, not a capability measure; (2) the code/math-domain imatrix likely explains the relative gap (domain shift). Note the general-slice KLD — which includes a wikitext-anchor domain — still favors this artifact. PPL here is a sanity flag, not a gate.

Reproducibility

Deterministic eval rig: 3 independent KLD runs are byte-identical (drift 0.0). Verdicts are by mean; no confidence-interval claim is made on KLD. GSM8K uses the tie-band discipline above.

Provenance

| Field | Value |

|---|---|

| Base model | google/gemma-4-E2B-it @ 179516f0c449474fdc46f08f30ead5b11e178497 |

| Base checkpoint sha256 | model.safetensors = 2db5482b20d746879bb3ef79b5203e9075a2e2b98f54ec7c2f281c1477ddc550 (10,246,621,918 B) |

| Conversion + quantization | llama.cpp @ 78d2f524682d9fee790a6460c93d018dafeb5229 (2026-07-05); convert_hf_to_gguf.py --outtype bf16 then llama-quantize with per-tensor --tensor-type overrides |

| Calibration | fraQtl Apex code+math packet (sha256 9bec84a28dcb0c940047e6084561a00857fd610b8a1e148cc38e27929e0a7e02), llama-imatrix ctx 2048, BF16 substrate, full packet (212 chunks / 434,176 tokens) |

| imatrix (ships in this repo) | imatrix.dat sha256 892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c |

| Recipe sha256 | 69f2151874412bcda4df1028c5b76743b1a60afa9f0a0abb2ff87c26a982bd37 (phone artifact differs only in the PLE embedding override) |

| Iso-size GGUF | Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf — 3,461,675,328 B, sha256 d008ba9d736b1d51e55d1d1944196441c1dbfe4fe9eb4ebd9bfbda4c2f36e013, true bpw 5.93, bandwidth-relevant bytes 1,188,795,532 |

| Phone GGUF | Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf — 2,856,122,688 B, sha256 d535787b323f79d8faf3867501c028dc05be4e0a3413a6ccdf54cb88bb737866, true bpw 4.89, bandwidth-relevant bytes 1,188,795,532 (identical by construction — speed-neutral) |

| Eval runtime | llama-cpp-python==0.3.33 (vendored llama.cpp 78d2f5246), A100-80GB, seed 0, deterministic (3-run drift 0.0) |

| Fidelity metric | symmetric top-20 KLD on Q8 teacher support, mean over positions; teacher = unsloth/gemma-4-E2B-it-GGUF Q8_0 @ 0314792d7f1f7e229411f620751375812bb9faf2 (sha256 605d3c2647d7c58c1e4b5375ccb5702acf94c2611b4c8d4877812f8fdd32d053), quantized from the ORIGINAL bf16 |

| Eval slices | code/math 400-record held-out (sha256 cce6860274ab715e3ab8493c9f817b1c29ff99c148f83a25de8944e027db1665); general chat/tool/wikitext 400-record (sha256 b10a79caf2c17cc10cd1edcae44d4655278baf5b09b6a867b4d3ade2f996b276) |

| Baseline (iso-size) | bartowski/google_gemma-4-E2B-it-GGUF Q4_K_M @ b5e99bd964eaacc27ba484bb2eb3e9f6160b9143 — 3,462,678,272 B, sha256 b5310340b3a23d31655d7119d100d5df1b2d8ee17b3ca8b0a23ad7e9eb5fa705, true 5.93 bpw (all tensors) |

| Probe receipt | gemma4_e2b_probe_2026-07-19.partial.json sha256 8511c8d6f1530a94e66f1f344e1e0bed57cc692819e36ca801946a71452f036a |

| Drift | 3-run KLD drift 0.0 (deterministic rig) |

Model Limitations

• Raw-web perplexity flag: wikitext-2 PPL (ctx 512) is worse than the community baseline (20009.7 vs 18005.3) — see the Evaluation section for the domain-shift note. If your workload is raw web-text continuation rather than chat/code/math, weigh this flag accordingly.
• Calibration domain: the imatrix is code+math. Both eval slices (including the general chat/tool/wikitext slice) favor this artifact on KLD, but calibration is not domain-neutral by construction.
• Lean eval scope (E1): GSM8K n=200 only; no MMLU, MATH-500, or long-context retrieval benchmarks were run for this model. The GSM8K result is a statistical tie, not a win.
• Phone artifact: KLD-evaluated (single run) and smoke-tested; GSM8K/PPL were not run on it.
• Quantization is lossy by construction; the evidence standard here is measured distributional fidelity (KLD) with pinned receipts, not bit-identity.
• The base model's own limitations and biases are inherited unchanged.

Files

| File | What | sha256 |

|---|---|---|

| Gemma-4-E2B-it-fraQtl-HiFi-Q4_K_M.gguf | Iso-size artifact (3.46 GB, true bpw 5.93) | d008ba9d736b1d51e55d1d1944196441c1dbfe4fe9eb4ebd9bfbda4c2f36e013 |

| Gemma-4-E2B-it-fraQtl-HiFi-Phone.gguf | Phone artifact (2.86 GB, true bpw 4.89, PLE at Q4_K) | d535787b323f79d8faf3867501c028dc05be4e0a3413a6ccdf54cb88bb737866 |

| imatrix.dat | Calibration importance matrix (reproducibility) | 892e5f908dc43cef60be5bf02b3246bf5da8651d88d107c9fd00c20771d83e3c |

More from fraQtl

Weight quantization is the on-device lane. The serving lane — KV-cache compression sidecars for vLLM with per-user retrieval verification — lives at fraQtl/qwen3-4b-instruct-2507-kv-sidecars and fraQtl/mistral-7b-instruct-v0.3-kv-sidecars. Same honesty rules per artifact: pinned provenance, measured numbers, disclosed flags. Org page: huggingface.co/fraQtl.