---

license: apache-2.0

base_model:

• Qwen/Qwen3.5-4B
• Ttimms/Bible-Assistant-Qwen3.5-4B-v3.2

base_model_relation: quantized

tags:

- bible

- rag

- gguf

- llama.cpp

- qwen3.5

- text-generation

language:

- en

pipeline_tag: text-generation

quantized_by: Ttimms

license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE

---

Bible AI Assistant v3.2 — GGUF

GGUF quants of Ttimms/Bible-Assistant-Qwen3.5-4B-v3.2

— a Qwen3.5-4B continued-fine-tune for retrieval-grounded Bible Q&A, and the

first checkpoint in the project's line to clear every acceptance gate: 80.3%

verbatim verse recall, 98.9% citation, 1.9% hallucination, and the strongest

result in its size class on a cross-encoder semantic metric against every

other ~4B bible-tuned model tested. See the base repo for the full model

card, version history, and the honest evaluation against larger models too.

Architecture

graph TD
    Base["Qwen/Qwen3.5-4B"]
    V31["v3.1 adapter (thematic-synthesis SFT)"]
    Cont["DMT-style continued fine-tune - RAFT-fixed thematic_qa + rehearsal, lr 5e-5"]
    Merge["merge adapter -> bf16"]
    Conv["convert_hf_to_gguf --no-mtp + llama-quantize"]
    ST["Bible-Assistant-Qwen3.5-4B-v3.2 (safetensors)"]
    GG["...-v3.2-GGUF (Q4_K_M / Q5_K_M / Q6_K / Q8_0 / F16)"]
    RAG["hybrid RAG: dense (nomic) + BM25 + RRF + bge-reranker-v2-m3"]
    LLM["llama.cpp / LM Studio / Ollama"]
    Base --> V31 --> Cont --> Merge --> ST
    Merge --> Conv --> GG
    ST --> RAG --> LLM
    GG --> LLM

Download

Grab one file, not the whole repo.

| File | Quant | Size | Notes |

|---|---|--:|---|

| bible-v3.2-4b-Q4_K_M.gguf | Q4_K_M | 2.7 GB | recommended |

| bible-v3.2-4b-Q5_K_M.gguf | Q5_K_M | 3.1 GB | |

| bible-v3.2-4b-Q6_K.gguf | Q6_K | 3.5 GB | |

| bible-v3.2-4b-Q8_0.gguf | Q8_0 | 4.5 GB | near-lossless |

| bible-v3.2-4b-f16.gguf | F16 | 8.4 GB | full precision |

Run it in

• llama.cpp — llama-server -m <file>.gguf -ngl 99
• LM Studio (bundles a recent llama.cpp)
• koboldcpp
• Jan
• text-generation-webui
• Ollama — once its bundled llama.cpp includes this arch (see Requirements)

Requirements

Qwen3.5 is a hybrid architecture (qwen35 / Gated-DeltaNet + attention). You

need a recent llama.cpp — a build that includes the qwen35 hybrid arch

(commit 3173a56 or newer). Verified working with llama-cli/llama-server

from a source build — coherent output, correctly quoted John 3:16, 175 tok/s

on an RTX 5070 Ti.

• ✅ llama.cpp (current): llama-server -m bible-v3.2-4b-Q4_K_M.gguf -ngl 99
• ✅ LM Studio (recent versions bundle a current llama.cpp)
• ⚠️ Ollama 0.33.x: the bundled llama.cpp is too old for the qwen35 arch

(check_tensor_dims: tensor 'blk.32.attn_norm.weight' not found). Use once

Ollama updates its runtime, or run llama.cpp directly.

Thinking mode

The Qwen3.5 chat template defaults to thinking on. This model was fine-tuned

without <think> traces, so for a grounded RAG assistant you want it off:

• llama.cpp /v1/chat/completions: pass

"chat_template_kwargs": {"enable_thinking": false}, or

• use a chat template that emits a closed empty <think>\n\n</think>\n\n after

<|im_start|>assistant\n (the project's deployment/pc/Modelfile does this).

Intended use

Retrieval-augmented Bible Q&A — the model expects retrieved verses in a Context:

block, then the question. It is not designed for context-free use, medical /

legal / financial advice, counselling (it redirects those to a pastor / crisis

line), or authoritative theological rulings.

License

Weights: Apache-2.0 (inherits from Qwen3.5-4B). Project code: MIT. Bible

translations: public domain. See the base repo for full attribution and the

version history that led to this checkpoint.