---

tags:

• gguf
• deepseek
• deepseek-v4
• vision
• multimodal
• image-text-to-text
• solstice-ai
• anvil
• turboquant
• sovereign-ai
• llama.cpp
• ollama
• endpoints_compatible
• vision
• mmproj
• unsloth
• base_model:deepseek-ai/DeepSeek-V4-Flash-Vision-Exp

license: mit

library_name: gguf

pipeline_tag: image-text-to-text

region: us

---

DeepSeek-V4-Flash-Vision-Exp UltraOptimised GGUF

Curated Unsloth Dynamic 3.0 quantisations of deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — the experimental multimodal variant of DeepSeek-V4-Flash with vision support.

All quants use Unsloth's UD 3.0 methodology with a high-quality imatrix calibration dataset refined for agentic coding, chat, and multilingual performance. Each quant is model-specific and layer-optimized for maximum quality at its size.

Base model: 305B total parameters, 284B MoE (13B active per token), 1M context, vision encoder included.

Quantisation Tiers

| Quant | Size | RAM Required | Description |

|-------|------|-------------|-------------|

| UD-Q8_K_XL | ~151 GB | 160+ GB | Near-lossless — full precision equivalent |

| UD-Q4_K_XL | ~145 GB | 155+ GB | Primary workhorse — only 7 GB smaller than Q8 |

| UD-Q3_K_XL | ~119 GB | 128+ GB | Fits 128 GB RAM Macs — strong community pick |

| UD-Q2_K_XL | ~90 GB | 96+ GB | Fits 96 GB RAM — still usable for agentic tasks |

| UD-IQ4_XS | ~127 GB | 135+ GB | Compact 4-bit with importance matrix |

| UD-IQ3_S | ~106 GB | 115+ GB | Balanced 3-bit |

| UD-IQ3_XXS | ~96 GB | 105+ GB | Smaller 3-bit |

| UD-IQ2_XXS | ~85 GB | 95+ GB | Ultra-compact 2-bit |

| UD-IQ1_S | ~77 GB | 85+ GB | Extreme compression — general knowledge only |

| UD-IQ1_M | ~81 GB | 90+ GB | Extreme compression — general knowledge only |

Vision Support

Both vision projector files are included:

• mmproj-BF16.gguf — BF16 precision (recommended)
• mmproj-F16.gguf — FP16 precision

Quick Start

# Download with HF CLI
hf download Solstice-AI/DeepSeek-V4-Flash-Vision-Exp-UltraOptimised-GGUF \
  UD-Q4_K_XL/* mmproj-BF16.gguf

# Run with llama.cpp (needs build >= 2026-06-04 for Gemma4/Vision-Exp support)
./llama-server \
  -m UD-Q4_K_XL/DeepSeek-V4-Flash-Vision-Exp-UD-Q4_K_XL-00001-of-00005.gguf \
  --mmproj mmproj-BF16.gguf \
  --ctx-size 32768 \
  --n-gpu-layers 99

Recommended Tiers

For most users: UD-Q4_K_XL or UD-Q3_K_XL

• Q4 is only ~6 GB smaller than Q8 but significantly more practical to serve
• Q3 fits 128 GB unified memory Macs (M2/M3/M4 Ultra)

For maximum quality: UD-Q8_K_XL

• Near-lossless — the model is already ~4.25 bit in its experts, so Q8 only upscales the shared dense layers

For constrained hardware: UD-Q2_K_XL

• Last tier before the Divergence-300 @32 quality cliff — still functional for agentic use

Quantisation Details

• Method: Unsloth Dynamic v3.0 — model-specific layer optimization
• Calibration: High-quality imatrix dataset refined for agentic coding, chat, and multilingual performance
• No QAT/QAD: Pure post-training quantization — no overfitting risk
• MTP module: Included in Q3+ quants; removed in smaller quants to save ~500 MB

Hardware Requirements

| Config | Min RAM/VRAM | Example Hardware |

|--------|-------------|-----------------|

| UD-Q8_K_XL | 160 GB | 2× M2 Ultra, 4× A100 80GB |

| UD-Q4_K_XL | 155 GB | 2× M2 Ultra, 4× A100 80GB |

| UD-Q3_K_XL | 128 GB | 1× M2/M3/M4 Ultra 128GB |

| UD-Q2_K_XL | 96 GB | 1× M2/M3 Max 96GB |

Credits

Quantisation by Unsloth — Dynamic v3.0 methodology.

Curated and hosted by Solstice-AI.

License

MIT — inherited from deepseek-ai/DeepSeek-V4-Flash-Vision-Exp.