---

license: mit

base_model: Ttimms/Bible-Assistant-Qwen3.5-4B-v2

tags:

- gguf

- quantization

- qwen

- bible

- assistant

language: en

pipeline_tag: text-generation

library_name: gguf

---

Bible Assistant Qwen3.5-4B GGUF

GGUF quantized versions of Ttimms/Bible-Assistant-Qwen3.5-4B-v2.

Quantization

| Quant | Bits | Size | Use Case |

|-------|------|------|----------|

| Q8_0 | 8 | ~4.2 GB | Maximum quality, slower |

| Q6_K | 6 | ~3.1 GB | High quality |

| Q5_K_M | 5 | ~2.6 GB | Balanced quality/speed |

| Q5_K_S | 5 | ~2.4 GB | Balanced quality/speed |

| Q4_K_M | 4 | ~2.0 GB | Good quality/speed trade-off |

| Q4_K_S | 4 | ~1.9 GB | Good quality/speed trade-off |

| Q3_K_L | 3 | ~1.6 GB | Small, acceptable quality |

| Q3_K_M | 3 | ~1.5 GB | Small, acceptable quality |

| Q3_K_S | 3 | ~1.4 GB | Small, acceptable quality |

| Q2_K | 2 | ~1.1 GB | Minimal, fast |

VRAM Requirements

| Quantization | VRAM | Notes |

|---|---|---|

| F16 | ~8 GB | Full precision |

| Q8_0 | ~4.5 GB | Minimal overhead |

| Q6_K | ~3.5 GB | Minimal overhead |

| Q5_K_M | ~3.0 GB | Minimal overhead |

| Q4_K_M | ~2.5 GB | Minimal overhead |

| Q3_K_M | ~2.0 GB | Minimal overhead |

| Q2_K | ~1.5 GB | Minimal overhead |

Usage

llama.cpp

./main -m model_q5_k_m.gguf -p "Your prompt here" -n 256

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q5_k_m.gguf", n_gpu_layers=-1)
output = llm("Your prompt here", max_tokens=256)
print(output["choices"][0]["text"])

LM Studio

Load any .gguf file directly in LM Studio.

Ollama

ollama create bible-assistant-qwen35 -f Modelfile

Where Modelfile contains:

FROM model_q5_k_m.gguf
PARAMETER temperature 0.7

Original Model

Ttimms/Bible-Assistant-Qwen3.5-4B-v2 — A 4B parameter model fine-tuned for Bible-related assistance and knowledge.

License

Refer to the original model's license.