---

language:

- en

license: apache-2.0

base_model:

- unsloth/Qwen2.5-1.5B-Instruct

datasets:

- r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation

tags:

- qwen2.5

- qwen

- tool-calling

- function-calling

- agents

- lora

- sft

- gguf

library_name: gguf

---

Qwen2.5-1.5B Tools GGUF

GGUF quantizations of the Qwen2.5-1.5B-Instruct model fine-tuned

for tool calling and agent-style interactions.

Base model

This model was fine-tuned from:

unsloth/Qwen2.5-1.5B-Instruct

Quantizations

| Quantization | File |

|---|---|

| Q4_K_M | qwen25-1.5b-tools-Q4_K_M.gguf |

| Q5_K_M | qwen25-1.5b-tools-Q5_K_M.gguf |

| Q8_0 | qwen25-1.5b-tools-Q8_0.gguf |

| F16 | qwen25-1.5b-tools-f16.gguf |

Training

Training was performed using NVIDIA NeMo AutoModel with LoRA/PEFT.

LoRA configuration

• LoRA rank (dim): 32
• LoRA alpha: 32
• LoRA dropout: 0.05
• Target modules: *.proj

Training configuration

• Sequence length: 4096
• Micro batch size: 2
• Gradient accumulation: 32
• Effective batch size: 64
• Epochs: 3
• Learning rate: 1e-5
• Weight decay: 0.01
• Optimizer: AdamW
• Loss: masked cross entropy
• Reasoning content masking: disabled

Dataset

The model was trained on the sft_tools portion of:

r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation

The dataset contains tool/agent interaction examples.

Intended use

This model is intended for:

• tool calling
• function calling
• agent workflows
• structured tool interactions
• local AI assistants
• lightweight local inference

It is not intended to be a general replacement for larger Qwen models.

GGUF versions

The repository includes GGUF conversions:

• F16
• Q8_0
• Q5_K_M
• Q4_K_M

Jetson Nano

Q4_K_M is the recommended starting point for low-memory systems such as the Jetson Nano.

For higher-memory systems, use Q5_K_M or Q8_0 where practical.

Usage with llama.cpp

Example:

llama-cli \
  -hf iromu/qwen25-1.5b-tools-GGUF:Q4_K_M \
  --n-gpu-layers 99 \
  --ctx-size 1024 \
  -p "You are a home assistant. Convert user requests into JSON tool calls. User: turn on kitchen lights"

llama-server \
  -hf iromu/qwen25-1.5b-tools-GGUF:Q4_K_M \
  --jinja \
  --n-gpu-layers 99 \
  --ctx-size 4096 \
  --host 0.0.0.0 \
  --port 8080