NANI-Nithin/G9v3-3B-GGUF overview
G9v3 3B GGUF GGUF quantized releases of ai9stars/G9v3 3B https://huggingface.co/ai9stars/G9v3 3B for llama.cpp and compatible runtimes. Model Information Base …
Runs locally from ~1.05 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| G9v3-3B-F16.gguf | GGUF | F16 | 5.57 GB | Download |
| G9v3-3B-IQ2_M.gguf | GGUF | IQ2_M | 1.05 GB | Download |
| G9v3-3B-IQ3_M.gguf | GGUF | IQ3_M | 1.34 GB | Download |
| G9v3-3B-IQ3_XS.gguf | GGUF | IQ3_XS | 1.25 GB | Download |
| G9v3-3B-IQ3_XXS.gguf | GGUF | IQ3_XXS | 1.16 GB | Download |
| G9v3-3B-IQ4_NL.gguf | GGUF | IQ4_NL | 1.64 GB | Download |
| G9v3-3B-IQ4_XS.gguf | GGUF | IQ4_XS | 1.56 GB | Download |
| G9v3-3B-Q2_K.gguf | GGUF | Q2_K | 1.13 GB | Download |
| G9v3-3B-Q3_K_L.gguf | GGUF | Q3_K_L | 1.51 GB | Download |
| G9v3-3B-Q3_K_M.gguf | GGUF | Q3_K_M | 1.41 GB | Download |
| G9v3-3B-Q3_K_S.gguf | GGUF | Q3_K_S | 1.30 GB | Download |
| G9v3-3B-Q4_0.gguf | GGUF | Q4_0 | 1.64 GB | Download |
| G9v3-3B-Q4_1.gguf | GGUF | Q4_1 | 1.79 GB | Download |
| G9v3-3B-Q4_K_M.gguf | GGUF | Q4_K_M | 1.72 GB | Download |
| G9v3-3B-Q4_K_S.gguf | GGUF | Q4_K_S | 1.64 GB | Download |
| G9v3-3B-Q5_K_M.gguf | GGUF | Q5_K_M | 1.99 GB | Download |
| G9v3-3B-Q5_K_S.gguf | GGUF | Q5_K_S | 1.95 GB | Download |
| G9v3-3B-Q6_K.gguf | GGUF | Q6_K | 2.29 GB | Download |
| G9v3-3B-Q8_0.gguf | GGUF | Q8_0 | 2.96 GB | Download |
Model Details
| Model ID | NANI-Nithin/G9v3-3B-GGUF |
|---|---|
| Author | NANI-Nithin |
| Pipeline | text-generation |
| License | apache-2.0 |
| Base model | ai9stars/G9v3-3B |
| Last modified | 2026-08-12T13:03:17.000Z |
Model README
---
license: apache-2.0
base_model:
- ai9stars/G9v3-3B
library_name: llama.cpp
tags:
- gguf
- llama-cpp
- text-generation
- conversational
- quantized
- 3b
- ai9stars
language:
- en
pipeline_tag: text-generation
quantized_by: NANI-Nithin
---
G9v3-3B-GGUF
GGUF quantized releases of ai9stars/G9v3-3B for llama.cpp and compatible runtimes.
Model Information
- Base Model: ai9stars/G9v3-3B
- Parameter Size: 3B
- Format: GGUF
- Quantized By: NANI-Nithin
- Quantization Tool: llama.cpp
Available Files
2-bit
- Q2_K
- IQ2_M
- Q2_K_L
3-bit
- IQ3_XXS
- IQ3_XS
- Q3_K_S
- IQ3_M
- Q3_K_M
- Q3_K_L
- Q3_K_XL
4-bit
- IQ4_XS
- IQ4_NL
- Q4_0
- Q4_1
- Q4_K_S
- Q4_K_M
5-bit
- Q5_K_S
- Q5_K_M
6-bit
- Q6_K
- Q6_K_L
8-bit
- Q8_0
Full Precision
- F16/BF16 GGUF
Recommended Quantizations
Best Overall
Q4_K_M
Recommended for most users. Excellent balance of quality, memory usage, and speed.
Higher Quality
Q5_K_M or Q6_K
For users seeking maximum quality while still benefiting from quantization.
Best IQ Quant
IQ4_NL
Excellent quality-per-GB and one of the strongest modern importance-aware quantizations.
Low Memory Systems
IQ3_M or Q3_K_M
Good balance of usability and reduced memory requirements.
IQ Quantizations
The following importance-aware quantizations are included:
- IQ2_M
- IQ3_XXS
- IQ3_XS
- IQ3_M
- IQ4_XS
- IQ4_NL
These quantizations were generated using an importance matrix (imatrix) calibration pass and typically provide improved quality retention compared to traditional quantization methods at similar file sizes.
Usage
llama.cpp
./llama-cli \
-m G9v3-3B-Q4_K_M.gguf \
-p "Hello!"Ollama
Create a Modelfile:
FROM ./G9v3-3B-Q4_K_M.ggufThen:
ollama create g9v3-3b -f Modelfile
ollama run g9v3-3bLM Studio
Download the desired GGUF file and import it directly into LM Studio.
Credits
- Original Model: ai9stars/G9v3-3B
- GGUF Conversion & Quantization: NANI-Nithin
- Quantization Framework: llama.cpp
Disclaimer
This repository contains converted GGUF files only.
Please refer to the original model repository for licensing terms, training methodology, benchmark results, intended use, limitations, and safety information.
Original model:
https://huggingface.co/ai9stars/G9v3-3B
Run NANI-Nithin/G9v3-3B-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models