tarruda/DeepSeek-V4-Flash-GGUF overview
DeepSeek V4 Flash GGUF GGUF quantizations for deepseek ai/DeepSeek V4 Flash https://huggingface.co/deepseek ai/DeepSeek V4 Flash DeepSeek published the origina…
Runs locally from ~5.0 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00001-of-00004.gguf | GGUF | IQ3_XXS | 5.0 MB | Download |
| IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00002-of-00004.gguf | GGUF | IQ3_XXS | 45.80 GB | Download |
| IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00003-of-00004.gguf | GGUF | IQ3_XXS | 46.56 GB | Download |
| IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00004-of-00004.gguf | GGUF | IQ3_XXS | 12.04 GB | Download |
| MXFP4/DeepSeek-V4-Flash-MXFP4-00001-of-00005.gguf | GGUF | GGUF | 5.0 MB | Download |
| MXFP4/DeepSeek-V4-Flash-MXFP4-00002-of-00005.gguf | GGUF | GGUF | 45.70 GB | Download |
| MXFP4/DeepSeek-V4-Flash-MXFP4-00003-of-00005.gguf | GGUF | GGUF | 45.69 GB | Download |
| MXFP4/DeepSeek-V4-Flash-MXFP4-00004-of-00005.gguf | GGUF | GGUF | 45.69 GB | Download |
| MXFP4/DeepSeek-V4-Flash-MXFP4-00005-of-00005.gguf | GGUF | GGUF | 8.56 GB | Download |
| Q2_K/DeepSeek-V4-Flash-Q2_K-00001-of-00003.gguf | GGUF | Q2_K | 5.0 MB | Download |
| Q2_K/DeepSeek-V4-Flash-Q2_K-00002-of-00003.gguf | GGUF | Q2_K | 45.92 GB | Download |
| Q2_K/DeepSeek-V4-Flash-Q2_K-00003-of-00003.gguf | GGUF | Q2_K | 44.37 GB | Download |
| Q3_K/DeepSeek-V4-Flash-Q3_K-00001-of-00004.gguf | GGUF | Q3_K | 5.0 MB | Download |
| Q3_K/DeepSeek-V4-Flash-Q3_K-00002-of-00004.gguf | GGUF | Q3_K | 46.55 GB | Download |
| Q3_K/DeepSeek-V4-Flash-Q3_K-00003-of-00004.gguf | GGUF | Q3_K | 45.82 GB | Download |
| Q3_K/DeepSeek-V4-Flash-Q3_K-00004-of-00004.gguf | GGUF | Q3_K | 24.13 GB | Download |
| imatrix.gguf | GGUF | GGUF | 448.6 MB | Download |
Model Details
| Model ID | tarruda/DeepSeek-V4-Flash-GGUF |
|---|---|
| Author | tarruda |
| Pipeline | text-generation |
| License | mit |
| Base model | deepseek-ai/DeepSeek-V4-Flash |
| Last modified | 2026-07-02T05:07:00.000Z |
Model README
---
quantized_by: tarruda
pipeline_tag: text-generation
base_model: deepseek-ai/DeepSeek-V4-Flash
base_model_relation: quantized
license: mit
license_link: https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md
tags:
- gguf
- imatrix
- conversational
- deepseek
- llama.cpp
---
DeepSeek V4 Flash GGUF
GGUF quantizations for
DeepSeek published the original model weights in MXFP4, so the MXFP4 GGUFs in
this repo are direct conversions of those original safetensors.
Quant Recipes
| Recipe | Quant Size | Default type | Tensor-specific overrides |
| --- | --- | --- | --- |
| Q3_K | 119297.23 MiB (3.52 BPW) | Q6_K | ffn_down_exps=q3_k, ffn_gate_exps=q3_k, ffn_up_exps=q3_k |
| IQ3_XXS | 106912.23 MiB (3.15 BPW) | Q6_K | ffn_down_exps=iq3_xxs, ffn_gate_exps=iq3_xxs, ffn_up_exps=iq3_xxs |
| Q2_K | 92465.23 MiB (2.73 BPW) | Q6_K | ffn_down_exps=q2_k, ffn_gate_exps=q2_k, ffn_up_exps=q2_k |
Usage
This is the script I use to run:
#!/bin/sh -e
model="./IQ3_XXS/DeepSeek-V4-Flash-IQ3_XXS-00001-of-00004.gguf"
ctx=131072
parallel=1
ctx_size=$((ctx * parallel))
llama-server --no-mmap --no-warmup \
--model $model --ctx-size $ctx_size -np $parallel \
--temp 1.0 --top-p 1.0Run tarruda/DeepSeek-V4-Flash-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models