---

license: apache-2.0

base_model: Qwen/Qwen3-VL-8B-Instruct

library_name: gguf

pipeline_tag: image-text-to-text

language:

• en
• zh

tags:

• qwen3-vl
• vision-language-model
• multimodal
• gguf
• q8_0
• llama.cpp
• image-understanding

---

Qwen3-VL-8B-Instruct — Q8_0 GGUF

High-precision 8-bit GGUF conversion of Qwen3-VL-8B-Instruct, including the Q8_0 vision projector.

Converted from the original, full-precision Hugging Face model using llama.cpp.

Model Information

| Property | Details |

|---|---|

| Base model | Qwen/Qwen3-VL-8B-Instruct |

| Architecture | Qwen3-VL |

| Parameters | 8B |

| Quantization | Q8_0 (8-bit) |

| Format | GGUF |

| Vision support | Yes, separate Q8_0 mmproj |

| Conversion hardware | AMD EPYC 7402P, 512 GB RAM |

| License | Apache 2.0 |

Included Files

| File | Approximate size | Description |

|---|---|---|

| Qwen3-VL-8B-Instruct-Q8_0.gguf | 8.7 GB | Language model |

| mmproj-Qwen3-VL-8B-Instruct-Q8_0.gguf | 752 MB | Vision projector |

Both files are needed for multimodal image understanding.

Capabilities

• Image understanding and visual question answering
• Screenshot and user interface analysis
• Document and image-based text understanding
• Chart and diagram interpretation
• General instruction following and text generation
• Multilingual text and image understanding

Potential applications include desktop screenshot analysis, IT support assistance, software interface interpretation, and visual troubleshooting.

Download

Install Hugging Face CLI:

pip install -U huggingface_hub

Download both GGUF files:

hf download oscarwest1/Qwen3-VL-8B-Instruct-Q8_0-GGUF \
  --local-dir ./qwen3-vl-q8

Run with llama.cpp

Use a recent version of llama.cpp with Qwen3-VL support.

llama-server \
  -m Qwen3-VL-8B-Instruct-Q8_0.gguf \
  --mmproj mmproj-Qwen3-VL-8B-Instruct-Q8_0.gguf \
  -c 8192 \
  --host 127.0.0.1 \
  --port 8080

For CPU-only inference:

llama-server \
  -m Qwen3-VL-8B-Instruct-Q8_0.gguf \
  --mmproj mmproj-Qwen3-VL-8B-Instruct-Q8_0.gguf \
  -c 8192 \
  -ngl 0 \
  --port 8080

For supported NVIDIA GPU systems, add -ngl 99 to offload model layers to the GPU, subject to available VRAM.

About Q8_0 Quantization

Q8_0 uses approximately 8-bit weight quantization, significantly reducing storage requirements compared with BF16.

This conversion aims to preserve model quality while reducing memory usage. Q8_0 is lossy, so outputs may differ from those of the original BF16 model.

No additional fine-tuning was performed.

Conversion Details

• Original weights: Qwen/Qwen3-VL-8B-Instruct
• Source format: Hugging Face Safetensors
• Output format: GGUF
• Quantization: Q8_0
• Conversion utility: llama.cpp convert_hf_to_gguf.py
• Language model and vision projector converted separately
• Conversion environment: AMD EPYC 7402P CPU with 512 GB RAM

Notes

The files were successfully exported to GGUF. Conversion success alone does not establish inference accuracy or benchmark performance.

Image-processing capabilities require a compatible inference engine and the included vision projector.

Credits

Original model: Qwen Team

GGUF conversion: oscarwest1

Conversion tooling: llama.cpp

This repository is an independent community conversion and is not an official Qwen release.

License

Apache License 2.0, following the original model license.

I miss you NYZ