---

license: apache-2.0

base_model: Qwen/Qwen3-0.6B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

• en

tags:

• gguf
• qwen3
• llama.cpp
• quantized
• config-generation

---

Qwen3-0.6B — GGUF (Q6_K, ~472 MB)

A tiny, Apache-2.0 Qwen3 instruction model in GGUF format, sized to be embedded directly

into an application (e.g. a local helper for configuration tasks) and run with

llama.cpp, Ollama, LM Studio, or any llama.cpp binding.

Single self-contained file, no Python runtime, runs on CPU or GPU, cross-platform.

> Note on "Qwen3.6": there is no small Qwen3.6 model — that family ships only as 27B and

> 35B-A3B, which cannot be quantized near this size. Qwen3-0.6B is the smallest current-gen Qwen,

> and is the right pick for a ~512 MB footprint.

Quantized weights redistributed unmodified from

unsloth/Qwen3-0.6B-GGUF, derived from

Qwen/Qwen3-0.6B. Credit to the Qwen team (model) and

Unsloth (GGUF conversion). Rehosted for convenience/pinning. Apache-2.0 — free to use, modify,

and redistribute, including bundled inside a product, provided the license and attribution are kept.

File

| File | Quant | Size | Notes |

|------|-------|------|-------|

| Qwen3-0.6B-Q6_K.gguf | Q6_K | ~472 MB | High quality, under 512 MB. |

SHA-256

1c15a88244c9e852516e5ba0e394fc97f2182f1f2757413fb99fe6a9214c033b  Qwen3-0.6B-Q6_K.gguf

Thinking mode

Qwen3 is a hybrid reasoning model. By default it may emit a <think>…</think> reasoning trace.

For terse config/tool output, disable thinking by appending /no_think to your message (or set

enable_thinking=false where your runtime exposes it):

<|im_start|>user
Write a minimal systemd unit that runs /usr/bin/myapp on boot as user app. /no_think<|im_end|>
<|im_start|>assistant

Platform compatibility

Standard GGUF v3, little-endian, qwen3-architecture, Q6_K k-quant

(quantization_version 2), tokenizer + chat template embedded — maximally portable.

| Axis | Support |

|------|---------|

| CPU architectures | x86-64, ARM64 / aarch64 (Apple Silicon, Raspberry Pi, Graviton, Ampere), plus anything else llama.cpp compiles for. CPU-only needs no GPU. |

| Operating systems | Linux, macOS, Windows (also iOS/Android via llama.cpp mobile builds). |

| GPU backends (optional) | Metal, CUDA, Vulkan, ROCm/HIP, SYCL. |

| Endianness | Little-endian (only big-endian s390x would need conversion). |

| Runtimes | llama.cpp, Ollama, LM Studio, llama-cpp-python, text-generation-webui, KoboldCpp, Jan, node-llama-cpp, and other GGUF loaders. |

Quick start

llama.cpp (server, OpenAI-compatible):

llama-server -m Qwen3-0.6B-Q6_K.gguf --port 8080 --jinja
# POST to http://localhost:8080/v1/chat/completions

Ollama:

printf 'FROM ./Qwen3-0.6B-Q6_K.gguf\n' > Modelfile
ollama create qwen3-0.6b -f Modelfile && ollama run qwen3-0.6b

Python (llama-cpp-python):

from llama_cpp import Llama
llm = Llama(model_path="Qwen3-0.6B-Q6_K.gguf", n_ctx=4096)
print(llm.create_chat_completion(messages=[
    {"role": "user", "content": "Write a minimal Dockerfile for a static site in ./public. /no_think"},
])["choices"][0]["message"]["content"])

License

Apache-2.0. Copyright the Qwen team; see the upstream model card for details.