Model Intelligence Sheet
ManniX-ITA/Qwen3.5-4B-M8-GGUF overview
Qwen3.5 4B M8 — GGUF GGUF quantizations of ManniX ITA/Qwen3.5 4B M8 https://huggingface.co/ManniX ITA/Qwen3.5 4B M8 . See the BF16 model card for the recipe an…
Runs locally from ~3.23 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
1 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| m8-Q6_K.gguf | GGUF | Q6_K | 3.23 GB | Download |
Model Details
| Model ID | ManniX-ITA/Qwen3.5-4B-M8-GGUF |
|---|---|
| Author | ManniX-ITA |
| Pipeline | — |
| License | apache-2.0 |
| Base model | ManniX-ITA/Qwen3.5-4B-M8 |
| Last modified | 2026-07-12T19:01:40.000Z |
Model README
---
base_model: ManniX-ITA/Qwen3.5-4B-M8
tags:
- qwen3.5
- merge
- gguf
- llama.cpp
license: apache-2.0
---
Qwen3.5-4B-M8 — GGUF
GGUF quantizations of ManniX-ITA/Qwen3.5-4B-M8.
See the BF16 model card for the recipe and full ablation matrix.
Quants
| File | Quant | Size |
|---|---|---:|
| m8-Q6_K.gguf | Q6_K | 3.46 GB |
Usage
llama-server -m m8-Q6_K.gguf \
--port 8099 -c 32768 -ngl 99 --no-warmup \
--reasoning-format deepseek --reasoning-budget 8192 \
--parallel 2 --cache-type-k q8_0 --cache-type-v q8_0Eval (Q6_K, lm-evaluation-harness)
| Task | Score |
|---|---:|
| HumanEval | 54.27 |
| MBPP | 51.40 |
Run ManniX-ITA/Qwen3.5-4B-M8-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models