---

license: apache-2.0

base_model: Qwen/Qwen3-VL-8B-Instruct

tags:

• gguf
• qwen3-vl
• vision
• multimodal
• enclave

---

qwen3-vl-8b-gguf (Enclave model volume)

Curated model volume for Enclave confidential inference:

Qwen3-VL-8B-Instruct at Q8_0 (~8.7 GB) with its F16 vision projector (~1.2 GB),

bundled with the official tokenizer.json so the volume is self-contained.

**This is the first Enclave volume that carries an mmproj, and it carries one

on purpose.** The text-only volumes in this org drop the projector files —

qwen3.5-9b-gguf says so explicitly — because llm-chat's ggml path used to

preload a single LLM gguf and had no way to use a vision tower. It does now: the

host pairs the weights with the projector it finds beside them, and a deployment

that attaches this volume can be shown a picture. The projector is loaded

lazily, on the first image a deployment is actually sent, so a chat that never

attaches one never pays its VRAM.

Two ggufs in one volume stay unambiguous the same way a multi-quant tree does:

the projector is identified by its mmproj name and taken out of the model pick

before "exactly one gguf" is decided, and the volume table names

Qwen3VL-8B-Instruct-Q8_0.gguf as the preload anyway.

Q8_0 rather than Q4_K_M is deliberate. Serving cost here is dominated by the KV

cache — Qwen3-VL is DENSE, so all 36 layers hold KV, ~144 KiB per token at f16

(~72 KiB at q8_0) — which puts the whole build at ~25 GB against a 25% H200

share's ~35 GB, where Q4 would save only ~3.7 GB of that. Vision work is also

where quantization damage hurts most: reading a value off a chart or small text

in a screenshot is exactly the task where a weaker quant returns a confident

wrong answer rather than an obviously bad one. The F16 projector follows the

same logic, and costs 0.4 GB over Q8_0 for an encoder that runs once per image.

Provenance

| File | Upstream | Revision | sha256 |

|------|----------|----------|--------|

| Qwen3VL-8B-Instruct-Q8_0.gguf | Qwen/Qwen3-VL-8B-Instruct-GGUF | f982a07559d4a2f6c8744d840bf6fccab30eea96 | 0d264b3941185d00a74f75c4245521dae088ff1efc90ab8d1754e83f5844adb0 |

| mmproj-Qwen3VL-8B-Instruct-F16.gguf | Qwen/Qwen3-VL-8B-Instruct-GGUF | f982a07559d4a2f6c8744d840bf6fccab30eea96 | ca524100ebf825c9a870db1c580d03879e0da0ab2541697e2458e64891cf9d38 |

| tokenizer.json | Qwen/Qwen3-VL-8B-Instruct | 0c351dd01ed87e9c1b53cbc748cba10e6187ff3b | a5d85b6dcc535e6b93115a9ef287e6132fdbf30270da6218194ba742261173c7 |

Upstream ships F16 and Q4_K_M weights and a Q8_0 projector as well (~32 GB for

the repo); this volume carries only the served pair plus the tokenizer, the same

rationale as the other volumes here.

The tokenizer is the Qwen3-VL one: vocab 151936, <|im_end|> 151645 /

<|endoftext|> 151643, with the vision tokens (<|vision_start|> 151652,

<|image_pad|> 151655) the projector's marker tokens use. It is NOT

byte-identical to the Qwen3.5 family tokenizer in qwen3.5-9b-gguf (different

vocab), so this model does not share a draft model with them.

LICENSE is the Apache-2.0 text; the upstream is Apache-2.0.

Geometry

36 layers, 8 KV heads, head_dim 128, vocab 151936, chatml template, dense

attention (no kv_layers reduction). Vision: SigLIP-family encoder, 16px

patches with 2x2 spatial merge, M-RoPE image positions — all handled inside

llama.cpp's mtmd, so nothing about it appears in a deployment's config beyond

"vision": true.

Usage on Enclave

Wrapped as a read-only attested volume (dm-verity; the root hash is bound into

the enclave's attestation) and attached by name; the guest reads it at

/models/<name>. The deployment's llm-chat config marks the catalog entry

"vision": true, which is what makes the playground offer the attach button and

the API accept OpenAI content-parts messages carrying images.

Vision needs a node whose llama.cpp toolchain includes libmtmd. On an older

node this volume still serves as a text model and the deployment reports that

images are unavailable, rather than silently ignoring them.