---

language:

• en

license: apache-2.0

base_model: Qwen/Qwen3.5-9B

library_name: llama.cpp

tags:

• qwen
• gguf
• q6_k
• quantized

---

Qwen3.5-9B - Q6_K GGUF

A Q6_K quantized GGUF version of Qwen3.5-9B for local AI inference.

Published by oscarwest1.

Model Details

| Property | Value |

|---|---|

| Base Model | Qwen/Qwen3.5-9B |

| Parameters | 9 Billion |

| Quantization | Q6_K |

| Format | GGUF |

| Runtime | llama.cpp and compatible GGUF engines |

| License | Apache 2.0 |

About This Release

This release provides a 6-bit-class quantized version of Qwen3.5-9B.

Q6_K reduces memory requirements compared to full-precision weights while aiming to preserve model quality.

No additional fine-tuning was performed as part of this quantization.

Intended Uses

• Local AI assistants
• Advanced reasoning
• Programming and software development
• IT troubleshooting
• Research and experimentation
• Self-hosted AI applications

Usage

Run with llama.cpp:

llama-cli -m qwen3.5-9b-Q6_K.gguf -c 32768 -cnv

Run a local inference server:

llama-server -m qwen3.5-9b-Q6_K.gguf --host 0.0.0.0 --port 8080 -c 32768

Credits

Original model: https://huggingface.co/Qwen/Qwen3.5-9B

Quantization tools: https://github.com/ggml-org/llama.cpp

This is an independently published quantized release, not an official Qwen distribution.

License

Apache License 2.0. Refer to the original model repository for its license and attribution requirements.

---

A Personal Note

In loving memory of NYZ.

I miss you, NYZ.