---

license: apache-2.0

pipeline_tag: text-generation

library_name: gguf

base_model: unsloth/Qwen3.5-9B-GGUF

tags:

- gguf

- qwen

- qwen3.5

- conversational

- layer-shards

---

Model Card for aquaduck/Qwen3.5-9B-GGUF

Layer-sharded Q4_K_M GGUF chunks of Qwen3.5-9B, derived from unsloth/Qwen3.5-9B-GGUF.

These files are not a new quantization. They are contiguous 4-layer packages cut from the full Unsloth Q4_K_M GGUF for staged / multi-node loading (Aquaduck Arc layer-package-v1).

Model lineage

Qwen/Qwen3.5-9B

└── quantized → unsloth/Qwen3.5-9B-GGUF (Q4_K_M)

└── layer-split → aquaduck/Qwen3.5-9B-GGUF (this repo)

• Base weights: https://huggingface.co/Qwen/Qwen3.5-9B (Apache 2.0)
• Quantization source: https://huggingface.co/unsloth/Qwen3.5-9B-GGUF (tag Q4_K_M)
• This repo: 4-layer GGUF shards for distributed inference

Model Details

Model Description

• Developed by: Aquaduck (layer packaging only; base model by Qwen Team / Alibaba Cloud; GGUF quant by Unsloth / llama.cpp ecosystem)
• Shared by: Aquaduck AI
• Model type: Causal language model (Qwen3.5 hybrid), GGUF Q4_K_M, layer-sharded
• Language(s): Multilingual (same as base)
• License: Apache-2.0 (inherits from Qwen/Qwen3.5-9B)
• Finetuned from model: N/A — not a fine-tune
• Derived from: unsloth/Qwen3.5-9B-GGUF ← Qwen/Qwen3.5-9B

Model Sources

• Base model card / blog: https://huggingface.co/Qwen/Qwen3.5-9B , https://qwen.ai/blog?id=qwen3.5
• Quantized GGUF source: https://huggingface.co/unsloth/Qwen3.5-9B-GGUF

Files

| File | Layers (inclusive start … end exclusive in filename) | Approx. size |

|------|------------------------------------------------------|--------------|

| Qwen3.5-9B-Q4_K_M-layers-0-4.gguf | 0–3 | ~1.14 GB |

| Qwen3.5-9B-Q4_K_M-layers-4-8.gguf | 4–7 | ~1.10 GB |

| Qwen3.5-9B-Q4_K_M-layers-8-12.gguf | 8–11 | ~1.10 GB |

| Qwen3.5-9B-Q4_K_M-layers-12-16.gguf | 12–15 | ~1.12 GB |

| Qwen3.5-9B-Q4_K_M-layers-16-20.gguf | 16–19 | ~1.10 GB |

| Qwen3.5-9B-Q4_K_M-layers-20-24.gguf | 20–23 | ~1.10 GB |

| Qwen3.5-9B-Q4_K_M-layers-24-28.gguf | 24–27 | ~1.12 GB |

| Qwen3.5-9B-Q4_K_M-layers-28-32.gguf | 28–31 | ~1.98 GB |

• Total layers: 32
• Valid split boundaries: 4, 8, 12, 16, 20, 24, 28
• Full-model equivalent: ~5.7 GB (same as Unsloth Q4_K_M)

Filenames use exclusive end indices (layers-{start}-{endExclusive}).

Uses

Direct Use

Intended for Aquaduck / Arc (and compatible) runtimes that download and merge layer packages, not as a single drop-in file for stock llama.cpp.

For a normal single-file local run, use unsloth/Qwen3.5-9B-GGUF or Qwen/Qwen3.5-9B instead.

Use the base model’s Qwen chat template (including thinking / instruct modes as documented by Qwen); other formats will not work correctly.

Out-of-Scope Use

• Expecting any one chunk to run as a complete model
• Treating this repo as a new training run or re-quant
• Uses prohibited by the Apache-2.0 license or Qwen’s model card guidance

Bias, Risks, and Limitations

Same capabilities, biases, and risks as Qwen/Qwen3.5-9B. Q4_K_M quantization (from Unsloth) can degrade quality vs. the original higher-precision releases. Layer sharding does not change weights beyond packaging.

Recommendations

Follow Qwen’s Qwen3.5 docs for chat template, thinking vs instruct modes, and sampling. Prefer the full Unsloth GGUF when you do not need staged loading.

How to Get Started

These shards are meant to be loaded automatically by an Aquaduck worker.

1. Install / open the Aquaduck desktop app and sign in as a worker.
2. Connect so the app can fetch your device’s model assignment from the model catalog (qwen/qwen3.5-9b).
3. If model shards do not automatically download: Use Get latest assigned model (under Settings). The worker will:

- download only the contiguous 4-layer chunk GGUFs for your assigned stage from this repo

- merge multi-chunk stages into one stage GGUF

- keep that stage ready for split serving

You do not need to pick individual -layers-.gguf files. Assignment and download are driven by model catalog metadata.

This repo is not a single-file Q4_K_M drop-in for stock llama.cpp. For that, use https://huggingface.co/unsloth/Qwen3.5-9B-GGUF instead.

Training Details

No training. Weights come from Qwen Team / Alibaba Cloud; Q4_K_M GGUF from Unsloth; this repo only splits that GGUF into 4-layer packages.

Evaluation

No separate evals for the shards. See Qwen/Qwen3.5-9B and https://qwen.ai/blog?id=qwen3.5.

Technical Specifications

• Architecture: Qwen3.5 (~9B params), hybrid Gated DeltaNet + Gated Attention, 32 layers
• Quantization: Q4_K_M (Unsloth)
• Packaging: Arc gguf-shard / catalog catalog:split → *-layers-{start}-{endExclusive}.gguf
• Package format: layer-package-v1

Citation

@misc{qwen3.5,
    title  = {{Qwen3.5}: Towards Native Multimodal Agents},
    author = {{Qwen Team}},
    month  = {February},
    year   = {2026},
    url    = {https://qwen.ai/blog?id=qwen3.5}
}

Credit:

• Unsloth (https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) for the GGUF quantization
• llama.cpp (https://github.com/ggml-org/llama.cpp) for GGUF support

Model Card Contact

Aquaduck AI — https://huggingface.co/aquaduck