---

base_model: google/gemma-2-9b-it

language:

- tr

- en

library_name: unsloth

tags:

- gguf

- llama-cpp

- gemma-2

- turkish

- instruct

license: gemma

---

🇹🇷 Turkish Gemma-2-9B-Instruct (GGUF)

This repository contains the GGUF formatted version of the Gemma-2-9B-It model, which has been fine-tuned on a Turkish instruction dataset using Unsloth.

The model is optimized to act as a highly capable Turkish AI assistant running locally and 100% offline via llama.cpp, Ollama, and LM Studio.

💾 Available GGUF File and System Requirements

Currently, the most optimal quantization version is available in this repository:

| File Name | Size | Recommended RAM | Description |

| :--- | :--- | :--- | :--- |

| *q4_k_m.gguf | ~5.4 GB | 12 GB | 🔥 The Golden Standard.** Offers the best balance between inference speed and model intelligence. Ideal for running on standard consumer hardware. |

---

📊 Model Performance Benchmarks (LLM-as-a-Judge)

This model has been tested under identical conditions alongside other popular Turkish GGUF models and evaluated via an LLM-as-a-Judge benchmark to measure Turkish language proficiency, instruction-following capabilities, and coding performance.

🏆 Comparison Table

| Model | Parameters | Geography (10) | Email Formatting (10) | Coding (10) | Total | Performance Summary |

|---|---|---|---|---|---|---|

| Qwen-2.5-Instruct | 7B | 4.0 | 8.5 | 10.0 | 22.5 / 30 | Most Balanced: Flawless Python code, fluent Turkish, and high instruction adherence. Minor hallucination tendencies on local geographical data. |

| Llama-3.1-Instruct | 8B | 0.5 | 0.0 | 5.0 | 5.5 / 30 | Partial Success: Strong algorithmic background (generates working code), but suffers from severe token repetition and looping on text tasks. |

| Mistral-NeMo-Instruct | 12B | 2.0 | 2.0 | 1.0 | 5.0 / 30 | Weak Instruction Following: While grammar is readable, it lacks task orientation (generates a list instead of code, fails to formalize casual tone). |

| Gemma-2-IT | 9B | 0.0 | 0.0 | 0.0 | 0.0 / 30 | Format Incompatibility: Due to special token structures and quantization sensitivity, it fails to produce meaningful output and enters a repetition loop. |

💬 Prompt Template (Gemma-2 Format)

The Gemma-2 architecture does not use the standard ChatML. It uses its own specific <start_of_turn> tokens. To prevent hallucinations and get the best performance, you must use the following structure:

<start_of_turn>user
Sen yardımsever bir Türkçe asistansın.
[Write your prompt here]<end_of_turn>
<start_of_turn>model

🚀 How to Run the Model

You can run this model locally with complete privacy and zero internet connection required.

Option 1: Using Python (llama-cpp-python)

1. Install the library via pip:

pip install llama-cpp-python

1. Download model from Hugging Face:

wget -O Turkish-Gemma-2-9B-Instruct-q4_k_m.gguf [https://huggingface.co/mehmettozlu/Turkish-Gemma-2-9B-Instruct-GGUF/resolve/main/Turkish-Gemma-2-9B-Instruct.Q4_K_M.gguf](https://huggingface.co/mehmettozlu/Turkish-Gemma-2-9B-Instruct-GGUF/resolve/main/Turkish-Gemma-2-9B-Instruct.Q4_K_M.gguf)

1. Create a Python script (run.py):

from llama_cpp import Llama

llm = Llama(
    model_path="./Turkish-Gemma-2-9B-Instruct-q4_k_m.gguf",
    n_ctx=4096,          # Context window size
    n_gpu_layers=-1      # Offload all layers to GPU
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Sen yardımsever bir Türkçe asistansın.\n\nYapay zeka modellerinin nasıl eğitildiğini kısaca açıklar mısın?"}
    ]
)

print(response["choices"][0]["message"]["content"])

Option 2: Using LM Studio (Easiest - GUI)

• Download and install LM Studio.
• In the search bar, type mehmettozlu/Turkish-Gemma-2-9B-Instruct-GGUF.
• Download the q4_k_m.gguf file.
• Navigate to the "Chat" tab, ensure the prompt format is set to "Gemma", and start chatting in Turkish.

Option 3: Using Ollama (For Developers)

Create a text file named Modelfile (no extension) and paste the following content:

FROM hf.co/mehmettozlu/Turkish-Gemma-2-9B-Instruct-GGUF/Turkish-Gemma-2-9B-Instruct-q4_k_m.gguf

TEMPLATE """<start_of_turn>user
{{ if .System }}{{ .System }}

{{ end }}{{ .Prompt }}<end_of_turn>
<start_of_turn>model
"""
SYSTEM """Sen yardımsever bir Türkçe asistansın."""
PARAMETER stop "<start_of_turn>"
PARAMETER stop "<end_of_turn>"

• Build and run the model:

ollama create turkish-gemma -f Modelfile
ollama run turkish-gemma