raymsm/gemma-2-9b-it-GGUF overview
gemma 2 9b it GGUF Quantized GGUF versions of google/gemma 2 9b it https://huggingface.co/google/gemma 2 9b it generated using llama.cpp on an Oracle Cloud VM.…
Runs locally from ~5.37 GB disk (8 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
gemma-2-9b-it GGUF
Quantized GGUF versions of google/gemma-2-9b-it generated using llama.cpp on an Oracle Cloud VM.
Repositories & Quantization Details
This repository contains the following quantization formats:
| Quantization Type | File Size | Quantization Duration | Peak RAM Used | Description |
| ----------------- | --------- | --------------------- | ------------- | ----------- |
| Q4_K_M | 5.37 GB | 105.0s | 1.4 MB | Medium-sized, good balance of size/perplexity. |
| Q5_K_M | 6.19 GB | 78.5s | 1.4 MB | Medium-sized, good balance of size/perplexity. |
| Q6_K | 7.07 GB | 54.8s | 1.4 MB | Medium-sized, good balance of size/perplexity. |
| Q8_0 | 9.15 GB | 26.7s | 1.4 MB | Medium-sized, good balance of size/perplexity. |
Sanity Check Prompts & Responses
Here are the responses generated by the quantized models to verify coherent text generation.
Format: Q4_K_M
- Prompt: What is the capital of France? Answer in one word.
- Response:
```text
What is the capital of France? Answer in one word.
Paris [end of text]
```
- Prompt: Write a short four-line poem about coding in Python.
- Response:
```text
Write a short four-line poem about coding in Python.
Indentation guides the way,
With loops and functions, we play.
From scripts to apps, it takes flight,
Python's power, shining bright.
Let me know if you'd like to see more poems about coding or any other topic!
[end of text]
```
- Prompt: Explain why the sky is blue in one simple sentence.
- Response:
```text
Explain why the sky is blue in one simple sentence.
Blue light scatters more than other colors in the atmosphere.
[end of text]
```
Format: Q5_K_M
- Prompt: What is the capital of France? Answer in one word.
- Response:
```text
What is the capital of France? Answer in one word.
Paris [end of text]
```
- Prompt: Write a short four-line poem about coding in Python.
- Response:
```text
Write a short four-line poem about coding in Python.
With Python's grace, I weave my code,
Indentation guides, a structured road.
From loops to functions, clear and bright,
A symphony of logic, day and night.
[end of text]
```
- Prompt: Explain why the sky is blue in one simple sentence.
- Response:
```text
Explain why the sky is blue in one simple sentence.
Blue light is scattered more by the atmosphere than other colors, making it appear dominant in the sky.
[end of text]
```
Format: Q6_K
- Prompt: What is the capital of France? Answer in one word.
- Response:
```text
What is the capital of France? Answer in one word.
Paris [end of text]
```
- Prompt: Write a short four-line poem about coding in Python.
- Response:
```text
Write a short four-line poem about coding in Python.
With Python's grace, I weave my code,
A symphony of logic, clear and bold.
From functions swift to classes grand,
A digital world, at my command.
[end of text]
```
- Prompt: Explain why the sky is blue in one simple sentence.
- Response:
```text
Explain why the sky is blue in one simple sentence.
Blue light is scattered more by the atmosphere than other colors, making it appear blue.
[end of text]
```
Format: Q8_0
- Prompt: What is the capital of France? Answer in one word.
- Response:
```text
What is the capital of France? Answer in one word. uska uska uska uskaRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEXRTEX betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory betweenstory
```
- Prompt: Write a short four-line poem about coding in Python.
- Response:
```text
Write a short four-line poem about coding in Python.labelledbyXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessTypeXmlAccessType
```
- Prompt: Explain why the sky is blue in one simple sentence.
- Response:
```text
Explain why the sky is blue in one simple sentence.HauptartikelHauptartikel出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年出版年
```
Pipeline Logs
- Base GGUF Conversion Peak RAM: 0.0 MB, Duration: 0.0s.
- Logged at:
Fri Jun 19 14:50:05 2026
Run raymsm/gemma-2-9b-it-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models