---

base_model: google/gemma-4-E2B-it

tags:

• gemma4
• gemma
• gguf
• llama.cpp
• unsloth
• vision-language-model

---

Viorra-Gemma-4-E2B-GGUF (Non-QAT Editions)

This repository contains the fine-tuned Viorra reasoning models converted to GGUF format using Unsloth.

> Important Architecture Notice: These models are Non-QAT (Standard LoRA fine-tunes). Extensive 4-way shootout testing revealed that merging a continuous 16-bit LoRA into a discrete 4-bit QAT grid caused severe precision smearing during GGUF export. This double-quantization error destroyed the Chain of Thought (CoT) reasoning logic. The Non-QAT versions successfully preserved 100% of the Claude Sonnet/Opus CoT reasoning engine and are the mathematically sound, definitive choice for Viorra.

Available Model files & VRAM Requirements:

| Model File | Precision | File Size | Est. Peak VRAM (Context) | Recommended Use |

|---|---|---|---|---|

| viorra-gemma-4-E2B-it-reasoning-q4_k_m.gguf | 4-bit | ~3.19 GB | ~4.5 GB | Best balance of extreme speed and reasoning accuracy. Ideal for local deployment. |

| viorra-gemma-4-E2B-it-q8_0.gguf | 8-bit | ~4.61 GB | ~6.0 GB | Maximum precision fallback for server-grade accuracy. |

| viorra-gemma-4-E2B-it-BF16-mmproj.gguf | 16-bit | ~0.98 GB | N/A | Vision Encoder Projection (used for multimodal). |

Example usage:

• For text only LLMs: llama-cli -hf qsardor/Viorra-Gemma-4-E2B-GGUF --jinja
• For multimodal models: llama-mtmd-cli -hf qsardor/Viorra-Gemma-4-E2B-GGUF --jinja

⚠️ Critical Formatting Requirements (Reasoning Engine)

Viorra 1.3 is a specialized reasoning model. It no longer uses the standard Gemma 4 chat template under the hood. If you are manually formatting prompts (bypassing apply_chat_template), you must use the following exact structure, otherwise the model will hallucinate or enter an infinite loop:

1. You must prepend <bos> to the very beginning of the prompt.
2. Use <|turn>user (Not <|turn|>user): The tags are asymmetrical.
3. Trigger the Reasoning Block: To activate Viorra's internal thinking, include <|think|> in the system prompt. The model will respond with <|channel>thought\n before its final answer.

Correct Manual Prompt Structure:

<bos><|turn>system
<|think|>
You are Viorra, a brutal essay reviewer.<turn|>
<|turn>user
Review my essay: ...<turn|>
<|turn>model

⚠️ Ollama Note for Vision Models

Important: Ollama currently does not support separate mmproj files for vision models.

To create an Ollama model from this vision model:

1. Place the Modelfile in the same directory as the finetuned bf16 merged model
2. Run: ollama create model_name -f ./Modelfile

(Replace model_name with your desired name)

This will create a unified bf16 model that Ollama can use.

This was trained 2x faster with Unsloth

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>