---

license: apache-2.0

license_link: https://ai.google.dev/gemma/docs/gemma_4_license

base_model: google/gemma-4-31B-it

tags:

- gguf

- gemma4

- gemma

- oxidize

- al-quant

- text-generation

library_name: gguf

pipeline_tag: text-generation

---

Gemma 4 31B — Oxidize AL Quants

Oxidize AL-family GGUF quants of

google/gemma-4-31B-it.

Quantized with oxidize-quantize

from Unsloth BF16 source weights (unsloth/gemma-4-31B-it-GGUF) — lossless

requant at the weight level (no quality loss from a prior quant).

Disclaimer

This quant type is experiemental

Files

| File | Type | ~Size | Bits | When to use |

|------|------|-------|------|-------------|

| gemma-4-31B-it-AL5.gguf | AL5 | 17 GB | ~4.5 | Default — same block size as Q4_0, lower RMSE |

What is AL?

Oxidize AL quants are custom ggml types (IDs 240–243) that use **multi-seed

MSE-optimal per-block scales** at the same block layouts as Q4_0 / Q5_0 / Q8_0.

AL5 decodes identically to Q4_0 bitstream layout but encodes with lower

reconstruction error. Implementation:

al_family.rs.

Quick start — llama.cpp

huggingface-cli download freakyskittle/gemma-4-31B-it-AL-GGUF gemma-4-31B-it-AL5.gguf --local-dir .

./llama-cli -m gemma-4-31B-it-AL5.gguf \
  -p "<|turn>user\nHello<turn|>\n<|turn>model\n<|channel>final<|message|>" \
  -n 256 --temp 0.7

Quick start — oxidize-cpp (CPU)

git clone https://github.com/Zapdev-labs/oxidize.git && cd oxidize/oxidize-cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j

./build/oxidize-cpp --model gemma-4-31B-it-AL5.gguf \
  --prompt "Hello" --max-tokens 128 --auto

Quick start — oxidize-c (CUDA)

git clone https://github.com/Zapdev-labs/oxidize.git && cd oxidize/oxidize-c
make cuda

./oxidize-c-cuda --model gemma-4-31B-it-AL5.gguf \
  --prompt "<|turn>user\nHello<turn|>\n<|turn>model\n" --max-tokens 128

Reproduce these quants

cargo build -p oxidize-quantize --release
./target/release/oxidize-quantize \
  --input gemma-4-31B-it-BF16-00001-of-00002.gguf \
  --output gemma-4-31B-it-AL5.gguf \
  --target AL5 --threads 96

Remote pipeline:

scripts/gemma4_31b_al_remote.sh

Links

• Oxidize: https://github.com/Zapdev-labs/oxidize
• Base model: https://huggingface.co/google/gemma-4-31B-it
• Source BF16 GGUF: https://huggingface.co/unsloth/gemma-4-31B-it-GGUF

License

Apache 2.0 (per Gemma 4 license).