---

license: apache-2.0

language:

• en

library_name: gguf

tags:

• gguf
• quantized
• nvfp4
• gemma-4
• multimodal
• vision
• audio

extra_gated_fields:

Allowlisting: text

Name: text

Organization: text

Country: text

Expected Use: text

base_model: google/gemma-4-31B-it

---

Google Gemma 4 31B IT — NVFP4 GGUF

This repository contains a NVFP4 (4-bit NVIDIA FP4) quantized GGUF of google/gemma-4-31B-it, optimized for efficient inference on NVIDIA GPUs with FP4 tensor core support.

> Note: Access to the original model requires accepting Google's terms. By using this quantized version, you agree to the original license (Apache 2.0).

Model Details

| Property | Value |

|---|---|

| Base Model | google/gemma-4-31B-it |

| Architecture | gemma4 |

| Quantization | NVFP4 (4-bit NVIDIA FP4, ftype=39) |

| Parameters | ~31B |

| Context Length | 262K tokens |

| Vocabulary | 262K tokens |

| Modalities | Text + Vision + Audio (via mmproj) |

| License | Apache 2.0 |

Files

| File | Size | Description |

|---|---|---|

| gemma-4-31B-it-NVFP4.gguf | ~16.5 GB | Main model weights (NVFP4 quantized) |

| mmproj-gemma-4-31B-it-f16.gguf | ~1.2 GB | Multimodal projector for vision/audio inputs |

Usage

With llama.cpp

# Text-only inference
./llama-cli -m gemma-4-31B-it-NVFP4.gguf -p "Your prompt here" -n 512

# Multimodal (image input)
./llama-cli -m gemma-4-31B-it-NVFP4.gguf --mmproj mmproj-gemma-4-31B-it-f16.gguf \
    --image path/to/image.jpg -p "Describe this image" -n 256

With LM Studio

1. Download all .gguf files from this repository
2. Place them in a folder (e.g., models/Gemma-4-31B-it-NVFP4/)
3. Launch LM Studio — it should auto-detect the model
4. The mmproj file will be automatically picked up for multimodal support

With other GGUF backends

Any GGUF-compatible backend that supports NVFP4 (ftype 39) and the gemma4 architecture can load this model.

Quantization Details

This model uses NVFP4 (NVIDIA FP4) quantization, which stores weights in 4-bit floating-point format (2:1:1 sign-exponent-mantissa). This is the same format supported natively by NVIDIA Blackwell GPUs with FP4 tensor cores.

• ftype: 39 (LLAMA_FTYPE_MOSTLY_NVFP4)
• Per-tensor dtype: 40 (GGML_TYPE_NVFP4)
• Compression ratio: ~4× vs FP16
• Requires: CUDA-capable GPU, or any backend supporting NVFP4

Multimodal Support

Gemma 4 is a natively multimodal model supporting:

• Text — standard text generation
• Vision — image understanding (via mmproj projector)
• Audio — audio understanding (via mmproj projector)

The mmproj-gemma-4-31B-it-f16.gguf file contains the multimodal projector weights needed for vision and audio inputs.

Original Model

For full details on capabilities, training, and evaluation, see the original model card.

License

This quantized model is distributed under Apache 2.0, consistent with the original Gemma 4 license.