jamiefutch/gemma-4-26B-A4B-it-MXFP4_MOE-GGUF overview
These are MXFP4 quantizations of the model gemma 4 26B A4B it https://huggingface.co/google/gemma 4 26B A4B it Quick Start 1. Download the latest release of ll…
Runs locally from ~1.11 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| gemma-4-26B-A4B-it-MXFP4_MOE.gguf | GGUF | GGUF | 14.36 GB | Download |
| gemma-4-26B-A4B-it-MXFP4_MOE_BF16.gguf | GGUF | BF16 | 15.80 GB | Download |
| gemma-4-26B-A4B-it-MXFP4_MOE_F16.gguf | GGUF | F16 | 15.80 GB | Download |
| gemma-4-26B-A4B-it-mmproj-BF16.gguf | GGUF | BF16 | 1.11 GB | Download |
| gemma-4-26B-A4B-it-mmproj-F32.gguf | GGUF | F32 | 2.13 GB | Download |
Model Details
| Model ID | jamiefutch/gemma-4-26B-A4B-it-MXFP4_MOE-GGUF |
|---|---|
| Author | jamiefutch |
| Pipeline | image-text-to-text |
| License | — |
| Base model | google/gemma-4-26B-A4B-it |
| Last modified | 2026-07-06T10:48:16.000Z |
Model README
---
pipeline_tag: image-text-to-text
base_model:
- google/gemma-4-26B-A4B-it
---
These are MXFP4 quantizations of the model gemma-4-26B-A4B-it
Quick Start
- Download the latest release of llama.cpp.
- Download your preferred model variant from below.
- For the
mmprojfile, it is recommended to use the F32 version for the best visual processing results. F32 > BF16 > F16
Which version should I choose?
All variants use MXFP4 for the MoE (Mixture of Experts) weights to keep the model efficient. The difference lies in how the remaining tensors are handled:
| Variant | Quality | Performance | Size | Recommendation |
| :--- | :--- | :--- | ---: | :--- |
| BF16 | ⭐⭐⭐ | Variable* | 15.80GiB | Best for maximum accuracy; original unquantized weights. |
| F16 | ⭐⭐ | Fast | 15.80GiB | Great alternative if BF16 is slow on your hardware. |
| Q8 | ⭐ | Fastest | 14.36GiB | Balanced performance and memory usage. |
Note: On some older architectures, BF16 may be slower than F16.
Check that your GPU supports native BF16
Read the guide from unsloth in order to set up the model's recommended settings:
The official chat template has been updated from Google. If you do not want to download the model again, you can just tell llama.cpp to use the new chat template.
Run jamiefutch/gemma-4-26B-A4B-it-MXFP4_MOE-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models