---

license: apache-2.0

base_model: Ttimms/Bible-Assistant-Qwen3.5-4B-v2

tags:

- gguf

- quantization

- qwen

language:

- en

pipeline_tag: text-generation

library_name: gguf

---

Bible Assistant Qwen3.5-4B-v2 GGUF

GGUF quantized versions of Ttimms/Bible-Assistant-Qwen3.5-4B-v2.

Quantization Details

| Quantization | Bits | Size | Use Case |

|---|---|---|---|

| Q8_0 | 8 | ~4.2 GB | High quality, minimal loss |

| Q6_K | 6 | ~3.2 GB | Balanced quality/size |

| Q5_K_M | 5 | ~2.7 GB | Good quality, reduced size |

| Q5_K_S | 5 | ~2.5 GB | Faster inference, slight quality loss |

| Q4_K_M | 4 | ~2.1 GB | Recommended default |

| Q4_K_S | 4 | ~1.9 GB | Faster, smaller footprint |

| Q3_K_L | 3 | ~1.6 GB | Mobile-friendly |

| Q3_K_M | 3 | ~1.5 GB | Aggressive quantization |

| Q3_K_S | 3 | ~1.4 GB | Mobile/edge devices |

| Q2_K | 2 | ~1.1 GB | Extreme compression |

VRAM Requirements

| Quantization | VRAM | Device |

|---|---|---|

| Q8_0 | ~4.5 GB | Desktop GPU |

| Q6_K | ~3.5 GB | Desktop GPU |

| Q5_K_M | ~3.0 GB | Desktop GPU |

| Q4_K_M | ~2.5 GB | Mid-range GPU / CPU |

| Q3_K_M | ~2.0 GB | Laptop GPU |

| Q2_K | ~1.5 GB | CPU-only |

Usage

llama.cpp

./main -m model_q4_k_m.gguf -n 512 -p "What does the Bible say about..."

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q4_k_m.gguf", n_gpu_layers=-1)
response = llm("What does the Bible say about...", max_tokens=512)
print(response["choices"][0]["text"])

LM Studio

1. Download any .gguf file from this repo
2. Open LM Studio → Load Model
3. Select and chat

Ollama

ollama pull hf.co/tinyopsec/bible-assistant-qwen35-4b-GGUF:q4_k_m
ollama run bible-assistant-qwen35-4b-GGUF:q4_k_m

Original Model

See Ttimms/Bible-Assistant-Qwen3.5-4B-v2 for model details, training data, and capabilities.

License

Licensed under Apache 2.0. See original model repo for details.