douganger/Dolphin-Mistral-24B-Venice-Edition-1.2-GGUF overview
Dolphin Mistral 24B Venice Edition 1.2 GGUF GGUF quantizations of dphn/Dolphin Mistral 24B Venice Edition https://huggingface.co/dphn/Dolphin Mistral 24B Venic…
Runs locally from ~9.6 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Dolphin-Mistral-24B-Venice-Edition-BF16.gguf | GGUF | BF16 | 43.92 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-IQ4_XS.gguf | GGUF | IQ4_XS | 11.88 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-Q3_K_M.gguf | GGUF | Q3_K_M | 10.69 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-Q4_K_M.gguf | GGUF | Q4_K_M | 13.35 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-Q5_K_M.gguf | GGUF | Q5_K_M | 15.61 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-Q6_K.gguf | GGUF | Q6_K | 18.02 GB | Download |
| Dolphin-Mistral-24B-Venice-Edition-Q8_0.gguf | GGUF | Q8_0 | 23.33 GB | Download |
| imatrix.gguf | GGUF | GGUF | 9.6 MB | Download |
| mmproj-Dolphin-Mistral-24B-Venice-Edition-BF16.gguf | GGUF | BF16 | 846.5 MB | Download |
| mmproj-Dolphin-Mistral-24B-Venice-Edition-F16.gguf | GGUF | F16 | 837.4 MB | Download |
Model Details
| Model ID | douganger/Dolphin-Mistral-24B-Venice-Edition-1.2-GGUF |
|---|---|
| Author | douganger |
| Pipeline | image-text-to-text |
| License | apache-2.0 |
| Base model | dphn/Dolphin-Mistral-24B-Venice-Edition |
| Last modified | 2026-09-01T21:05:58.000Z |
Model README
---
license: apache-2.0
base_model: dphn/Dolphin-Mistral-24B-Venice-Edition
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
quantized_by: douganger
tags:
- gguf
- llama.cpp
- quantized
- imatrix
- vision
- mistral3
---
Dolphin Mistral 24B Venice Edition 1.2 GGUF
GGUF quantizations of dphn/Dolphin-Mistral-24B-Venice-Edition, built from version 1.2, with vision adapters and an importance matrix. All the Dolphin-Mistral-24B-Venice-Edition GGUFs I could find on Hugging Face were converted from an earlier release, so I built these from version 1.2, introduced in 337ce042026e: "Updated to version 1.2 - vision + 131k context + improved tool calling".
Quality vs size
Every quant was measured by KL-divergence against the BF16 weights on held-out wikitext-2 (128 chunks).
| Quant | Size | bpw | Mean KLD vs BF16 | PPL ratio | Notes |
|---|---:|---:|---:|---:|------------------------|
| Q3_K_M | 10.69 GB | 3.89 | 0.049230 | 1.0585 | |
| IQ4_XS | 11.88 GB | 4.33 | 0.020138 | 1.0244 | Best pick under 12 GB. |
| Q4_K_M | 13.35 GB | 4.87 | 0.016559 | 1.0221 | Good default choice. |
| Q5_K_M | 15.61 GB | 5.69 | 0.003989 | 1.0049 | |
| Q6_K | 18.02 GB | 6.57 | 0.001576 | 1.0018 | |
| Q8_0 | 23.33 GB | 8.50 | 0.000188 | 1.0004 | Effectively lossless. |
BF16 (43.92 GB) is the reference and is included for anyone wanting to re-quantize without re-downloading the safetensors.
Run douganger/Dolphin-Mistral-24B-Venice-Edition-1.2-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models