---

library_name: node-llama-cpp

license: apache-2.0

license_link: https://ai.google.dev/gemma/docs/gemma_4_license

pipeline_tag: image-text-to-text

base_model: google/gemma-4-31B-it

quantized_by: giladgd

tags:

• node-llama-cpp
• llama.cpp
• conversational

---

gemma-4-31B-it-GGUF

> [!NOTE]

> Read our blog post to learn more about using Gemma 4 with node-llama-cpp

Static quants of google/gemma-4-31B-it.

Quants

| Link | URI | Quant | Size |

|:-----|:--------------------------------------------------|:------|-----:|

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q2_K | Q2_K | 11.9GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q3_K_S | Q3_K_S | 13.8GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q3_K_M | Q3_K_M | 15.3GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q3_K_L | Q3_K_L | 16.6GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q4_0 | Q4_0 | 17.7GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q4_K_S | Q4_K_S | 17.8GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q4_K_M | Q4_K_M | 18.7GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q5_0 | Q5_0 | 21.3GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q5_K_S | Q5_K_S | 21.3GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q5_K_M | Q5_K_M | 21.8GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q6_K | Q6_K | 25.2GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:Q8_0 | Q8_0 | 32.6GB |

| GGUF | hf:giladgd/gemma-4-31B-it-GGUF:BF16 | BF16 | 61.4GB |

> [!TIP]

> Download a quant using node-llama-cpp (more info):

> ```bash

> npx -y node-llama-cpp pull <URI>

> ```

Usage

Use with node-llama-cpp (recommended)

Ensure you have node.js installed:

brew install nodejs

CLI

Chat with the model:

npx -y node-llama-cpp chat hf:giladgd/gemma-4-31B-it-GGUF:Q4_K_M

Code

Use it in your project:

npm install node-llama-cpp

import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";

const modelUri = "hf:giladgd/gemma-4-31B-it-GGUF:Q4_K_M";


const llama = await getLlama();
const model = await llama.loadModel({
    modelPath: await resolveModelFile(modelUri)
});
const context = await model.createContext();
const session = new LlamaChatSession({
    contextSequence: context.getSequence()
});


const q1 = "Hi there, how are you?";
console.log("User: " + q1);

const a1 = await session.prompt(q1);
console.log("AI: " + a1);

> [!TIP]

> Read the getting started guide to quickly scaffold a new node-llama-cpp project

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

CLI

llama-cli -hf giladgd/gemma-4-31B-it-GGUF:Q4_K_M -p "The meaning to life and the universe is"

Server

llama-server -hf giladgd/gemma-4-31B-it-GGUF:Q4_K_M -c 2048