---

base_model: Qwen/Qwen3.8-27B

base_model_relation: quantized

pipeline_tag: image-text-to-text

license: apache-2.0

tags:

- qwen3.8

- quantized

- gguf

- vision-language

- mtp

---

Qwen3.8-27B Q4_K_M GGUF

This repository contains one independently downloadable Q4_K_M GGUF of

Qwen/Qwen3.8-27B. It is an unmodified vanilla quantization, not a fine-tune,

merge, ablation, alignment change, or chat-template modification.

The source weights are pinned to 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. The official checkpoint's

internal Qwen3_5ForConditionalGeneration / qwen3_5 architecture identifier

does not mean this is a Qwen3.5 model.

Files and conversion

• Main model: Qwen3.8-27B-Q4_K_M.gguf (16.811 GB decimal)
• Vision projector: mmproj-Qwen3.8-27B-F16.gguf
• Complete repository size: 17.751 GB decimal
• Quantization: Q4_K_M using llama.cpp K/IQ quantization
• llama.cpp revision: 5f754ea0e2fd21e1213db7ebebfd65d938d9d69c
• Calibration: none for K-quants; representative local prompts where required by IQ conversion
• Source inventory: 1,199 tensors, including 333 vision tensors and 15 source MTP tensors

Validated component status

• Text generation: passed
• Tool calling: passed all five native-format cases
• Vision and video: passed all three deterministic local-image cases
• Thinking controls and chat template: preserved and checked against the locked source
• Hybrid Gated DeltaNet/full-attention structure: retained
• MTP tensors and metadata: retained; speculative acceleration is not advertised
• Per-file runtime smoke: passed at 13.640331732867743 generated tokens/s on the validation host
• Weight SHA-256: 115a859329b5f4e876bed5ed49225d8097d055783a81945004a80af2fb2703f8
• Validation lineage: the same bytes passed in the combined release at

f519a212d6c15cd3292b6ca835dd8ebf235642c0

validation_result.json records the exact per-file smoke result and its inherited

aggregate functional gates. These split repositories do not claim a new benchmark

run; they preserve byte-identical, already-validated artifacts.

Inference

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout 5f754ea0e2fd21e1213db7ebebfd65d938d9d69c
cmake -S . -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

hf download Chungulus/Qwen3.8-27B-Q4_K_M-GGUF --local-dir ../qwen38-q4_k_m
cd ../qwen38-q4_k_m
../llama.cpp/build/bin/llama-mtmd-cli \
  -m ./Qwen3.8-27B-Q4_K_M.gguf \
  --mmproj ./mmproj-Qwen3.8-27B-F16.gguf \
  -p 'Describe the image.' --image ./image.png

Allow at least approximately 21 GB of available memory for the model,

vision projector, and modest runtime overhead; KV-cache needs grow with context.

This is planning guidance, not a measured peak-memory guarantee.

Use the source chat-template controls for enable_thinking, reasoning_effort,

and preserve_thinking, together with Qwen's native tool-call format.

Limitations

Quantization can reduce quality, particularly at lower bit widths. The longest

recorded validation prompt was 73 tokens; do not infer that the architectural

maximum context was tested. A runtime must support the Qwen3.8 hybrid language

graph, vision tower/projector, tokenizer, and retained MTP metadata rather than

loading only a single language tensor.

License and attribution

The parent model and this vanilla quantization use the source model's Apache-2.0

license. See Qwen/Qwen3.8-27B.