---

license: mit

base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

language:

- en

- multilingual

tags:

- gguf

- deepseek

- deepseek-r1

- qwen2

- llama.cpp

- iq2_xxs

- quantization

pipeline_tag: text-generation

---

DeepSeek-R1-Distill-Qwen-1.5B (IQ2_XXS GGUF)

A heavily compressed GGUF quantization of

DeepSeek-R1-Distill-Qwen-1.5B

— the smallest official DeepSeek reasoning model.

This card covers the single file:

• DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf

| Property | Value |

|---|---|

| File | DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf |

| Size | 560.37 MiB (587.59 MB) |

| Quantization | IQ2_XXS (≈ 2.06 bits/weight) |

| Parameters | 1.78B (total) / ~1.5B (active) |

| Architecture | Qwen2 |

| Context length | 32768 tokens |

| GGUF version | 3 (latest) |

Why this quantization

IQ2_XXS is the smallest IQ type that keeps the model usable on low-VRAM

hardware. At ~0.56 GB the weights load on any 4 GB-class GPU (e.g. GTX 1650)

while retaining the model's reasoning behaviour, including the

[Start thinking] / [End thinking] chain-of-thought format.

It was produced from the F16 source with llama.cpp using an **importance

matrix** computed with llama-imatrix over a wikitext-2 calibration set. The

advertised context was capped at 32768 tokens so the KV cache fits a 4 GB GPU

when fully offloaded (-ngl 999).

Usage

llama.cpp (CLI)

llama-cli \
  -m DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
  -p "What is the capital of France?" \
  -n 256 \
  -ngl 999 \
  -c 32768

llama-server (OpenAI-compatible API)

llama-server \
  -m DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
  -ngl 999 \
  -c 32768 \
  --port 8080
# curl http://localhost:8080/v1/chat/completions ...

Docker Model Runner (docker model)

docker model package \
  --gguf DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf \
  --context-size 32768 \
  deepseek-r1
docker model run deepseek-r1 "What is the capital of France?"

Ollama

ollama create deepseek-r1-iq2-xxs -f Modelfile   # FROM ./DeepSeek-R1-Distill-Qwen-1.5B-f16-iq2_xxs.gguf
ollama run deepseek-r1-iq2-xxs

Quality expectations

IQ2_XXS is an aggressive compression. The model answers factual questions

correctly and keeps its reasoning structure, but expect degraded fluency and

precision compared to F16 / Q4_K_M. For higher quality on the same hardware,

prefer Q4_K_M (~1.0 GB); for absolute minimum size, IQ1_S or Q1_0 exist

but degrade much faster.

Credits & license

• Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (MIT)
• Source GGUF: bartowski/DeepSeek-R1-Distill-Qwen-1.5B-GGUF (MIT)
• Quantized with llama.cpp + llama-imatrix
• This file is released under the MIT license.