---

license: apache-2.0

base_model: mistralai/Mistral-7B-v0.3

language:

- fr

tags:

- mistral

- gguf

- french

- insurance

- healthcare

- lora

quantized_by: goldthinger

---

Mistral-7B-v0.3 — French Insurance (GGUF)

GGUF quantizations of a LoRA fine-tune of Mistral-7B-v0.3 on French health-insurance terminology for healthcare professionals.

Files

| File | Quant | Size | Notes |

|---|---|---|---|

| model-Q4_K_M.gguf | Q4_K_M | ~4.4 GB | Recommended default |

| model-Q5_K_M.gguf | Q5_K_M | ~5.1 GB | Better quality |

| model-Q8_0.gguf | Q8_0 | ~7.2 GB | Near-lossless |

Benchmark results

50-question French insurance Q&A benchmark across three models — the base mistral:7b, the FP16 fine-tune, and the Q8 quantization from this repo — judged by Claude against a frozen rubric (4 sub-dimensions + holistic overall, 0–5). Full report: model-comparison_2026-06-16.html.

| Rank | Model | Mean overall | factual_accuracy | completeness | Median latency |

| --- | --- | --- | --- | --- | --- |

| 1 | mistral-insurance-fr (FP16) | 3.66 | 3.64 | 3.74 | 5.94 s |

| 2 | mistral-insurance-fr-gguf-q8 (this repo) | 2.94 | 3.28 | 2.98 | 3.07 s |

| 3 | mistral:7b (base) | 1.40 | 1.44 | 2.00 | 8.72 s |

The unquantized FP16 fine-tune is the natural accuracy winner — that's where the industry knowledge is carried in full, leading every question category (verbatim 4.40, paraphrased 3.45, new 2.60) and every domain-relevant dimension. The Q8 quantization in this repo retains roughly 80% of the overall score and 90% of the factual accuracy at ~2× the inference speed (3.07 s vs 5.94 s median latency), making it the practical choice for local inference. The base mistral:7b ranks last on every domain dimension despite winning raw french_quality (4.76 vs 4.34) — fluent French paired with hallucinated insurance facts (factual_accuracy 1.44 / 5). Pick Q8 when local speed and footprint matter; pick the FP16 model when you want every accuracy point.

Benchmark harness, prompts, rubric, and per-question results: rlondner/ft-mistral7b-french-insurance-tests.

Usage

Ollama

ollama create insurance-mistral -f Modelfile
ollama run insurance-mistral

llama.cpp

llama-cli -m model-Q4_K_M.gguf -p "..."

Base model: mistralai/Mistral-7B-v0.3

Full-precision (safetensors) version: rlondner/mistral-7b-insurance