---

language:

• en

license: apache-2.0

tags:

• llama-cpp
• gguf
• qwen3
• function-calling
• tool-use
• quantized

base_model: ermiaazarkhalili/Qwen3.8-2B-Function-Calling-xLAM-Unsloth

---

Qwen3.8-2B-Function-Calling-xLAM GGUF

GGUF quantizations of ermiaazarkhalili/Qwen3.8-2B-Function-Calling-xLAM-Unsloth, a 2B parameter model fine-tuned for function calling and tool use via the xLAM pipeline.

Model Architecture

| Property | Value |

|---|---|

| Architecture | Qwen3.5 (SSM + Attention hybrid) |

| Parameters | 2B |

| Context Length | 262144 tokens |

| Base Model | Qwen3.8-2B |

Quantizations

| Quantization | File Size | Description |

|---|---|---|

| Q8_0 | 2.08 GB | Near-lossless, maximum quality |

| Q6_K | 1.61 GB | Very high quality |

| Q5_K_M | 1.45 GB | High quality |

| Q5_K_S | 1.42 GB | High quality, smaller |

| Q4_K_M | 1.31 GB | Recommended, good balance |

| Q4_K_S | 1.25 GB | Good quality, smaller |

| IQ4_XS | 1.23 GB | Good quality, imatrix calibrated |

| Q3_K_L | 1.20 GB | Medium quality |

| Q3_K_M | 1.13 GB | Medium quality |

| Q3_K_S | 1.05 GB | Below average quality |

| Q2_K | 990 MB | Low quality, smallest |

Q4_K_M is recommended for most use cases.

Usage

Download a single file with huggingface-cli:

huggingface-cli download tinyopsec/Qwen3.8-2B-Function-Calling-xLAM-GGUF --include "Q4_K_M.gguf" --local-dir ./

Run with llama.cpp:

llama-cli -m Q4_K_M.gguf -p "Your prompt here" -n 512

Run as a server:

llama-server -m Q4_K_M.gguf --port 8080

Chat Template

This model uses the Qwen3 chat template:

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Conversion Details

Quantizations were produced using llama.cpp. The intermediate F16 GGUF was created with convert_hf_to_gguf.py and K-quants were produced with llama-quantize. IQ4_XS was produced using an imatrix calibration file.

Original Model

ermiaazarkhalili/Qwen3.8-2B-Function-Calling-xLAM-Unsloth