---

license: apache-2.0

base_model: Qwen/Qwen2.5-1.5B-Instruct

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

language:

• en

tags:

• gguf
• qwen2.5
• llama.cpp
• quantized
• config-generation

---

Qwen2.5-1.5B-Instruct — GGUF

Small, fast, Apache-2.0 instruction model in GGUF format — a portable single-file build

meant to be embedded directly into an application (e.g. to help with configuration tasks) and run

locally with llama.cpp, Ollama, LM Studio, or any

llama.cpp binding (Python/Rust/Go/Node/C#). No Python runtime required; runs on CPU or GPU,

cross-platform (macOS / Linux / Windows / ARM).

These GGUF weights are redistributed unmodified from the official

Qwen/Qwen2.5-1.5B-Instruct-GGUF,

which is derived from Qwen/Qwen2.5-1.5B-Instruct.

All credit for the model belongs to the Qwen team. Rehosted here for convenience/pinning under a

single project namespace. Licensed Apache-2.0 — free to use, modify, and redistribute,

including bundled inside a product, provided the license and attribution are retained.

Files

| File | Quant | Size | Notes |

|------|-------|------|-------|

| qwen2.5-1.5b-instruct-q3_k_m.gguf | Q3_K_M | ~882 MB | Smallest; comfortably under 1 GB. |

| qwen2.5-1.5b-instruct-q4_k_m.gguf | Q4_K_M | ~1.07 GB | Recommended; best quality/size balance. |

SHA-256

58cb5c05ecef48e82961f1a2be6544145ea26136f69dddda4bbbd092f0e4b993  qwen2.5-1.5b-instruct-q3_k_m.gguf
6a1a2eb6d15622bf3c96857206351ba97e1af16c30d7a74ee38970e434e9407e  qwen2.5-1.5b-instruct-q4_k_m.gguf

Platform compatibility

These are standard GGUF v3, little-endian, qwen2-architecture files using universal

k-quants (Q3_K_M / Q4_K_M, quantization_version 2) with the tokenizer and chat template

embedded — i.e. maximally portable. They load on any build of llama.cpp from mid-2024 onward and

everything downstream of it.

| Axis | Support |

|------|---------|

| CPU architectures | x86-64, ARM64 / aarch64 (Apple Silicon, Raspberry Pi, AWS Graviton, Ampere), plus anything else llama.cpp compiles for (RISC-V, POWER). CPU-only inference needs no GPU. |

| Operating systems | Linux, macOS, Windows (also iOS/Android via llama.cpp mobile builds). |

| GPU backends (optional) | Metal (Apple), CUDA (NVIDIA), Vulkan (cross-vendor, incl. Android), ROCm/HIP (AMD), SYCL (Intel). |

| Endianness | Little-endian. Only big-endian hosts (e.g. s390x) would need a byte-swapped GGUF. |

| Runtimes | llama.cpp, Ollama, LM Studio, llama-cpp-python, text-generation-webui, KoboldCpp, Jan, GPT4All, node-llama-cpp, and other GGUF loaders. |

Because everything (weights, tokenizer, chat template) is inside the single .gguf, deployment is a

file copy — no external config, no network fetch, no Python.

Quick start

llama.cpp (server, OpenAI-compatible):

llama-server -m qwen2.5-1.5b-instruct-q4_k_m.gguf --port 8080 --jinja
# then POST to http://localhost:8080/v1/chat/completions

Ollama:

# Modelfile
FROM ./qwen2.5-1.5b-instruct-q4_k_m.gguf

ollama create qwen2.5-1.5b -f Modelfile && ollama run qwen2.5-1.5b

Python (llama-cpp-python):

from llama_cpp import Llama
llm = Llama(model_path="qwen2.5-1.5b-instruct-q4_k_m.gguf", n_ctx=4096, chat_format="qwen")
print(llm.create_chat_completion(messages=[
    {"role": "system", "content": "You generate configuration files. Output only the file."},
    {"role": "user", "content": "Write a minimal nginx server block that serves /var/www on port 8080 for example.com."},
])["choices"][0]["message"]["content"])

Embedding in another app (OpenAI-compatible)

Any host that speaks the OpenAI Chat Completions API can use this model by pointing at a local

llama-server:

• Base URL: http://localhost:8080/v1
• API key: any non-empty string (llama.cpp ignores it)
• Model id: the file name / --alias you set

This is the same pattern used to plug a local model into agent harnesses (Codex custom

model_providers, Ollama-compatible clients, etc.).

Prompt format

Standard Qwen2 ChatML (handled automatically by --jinja / the chat templates above):

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

License

Apache-2.0. Copyright the Qwen team; see the upstream model card for full details.