---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE

base_model: Qwen/Qwen3.5-9B

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

tags:

• qwen
• qwen3.5
• gguf
• llama.cpp
• function-calling

---

Qwen3.5 9B Artemis GGUF

Text-only GGUF versions of Qwen3.5 9B for llama.cpp. The vision encoder and multi-token prediction (MTP) layer are not included.

Start with Qwen3.5-9B-Artemis-Q4_K_M.gguf (5.661 GB). Use llama.cpp with Qwen3.5 and HTTPS support:

llama-server -hf turintech/Qwen3.5-9B-Artemis-GGUF:Q4_K_M --port 8080 -c 8192 --jinja

<div>

<a href="https://turintech.ai"><img src="https://huggingface.co/turintech/Qwen3.5-9B-Artemis-GGUF/resolve/main/assets/turintech-logo.png" width="180" alt="TurinTech"></a>

<p style="margin: 4px 0;"><em>TurinTech models, built with Artemis</em></p>

</div>

Why use this file

• Run locally so inputs can stay on the same machine.
• Work offline after downloading the model and runtime.
• Use for local chat and tool-calling applications, within the tested limits below.
• Choose a compact GGUF download; file size is not a measurement of runtime memory.

Files

| File | Size | Choose for |

| --- | --- | --- |

| Qwen3.5-9B-Artemis-IQ4_XS.gguf | 5.167 GB | Smaller download. |

| Qwen3.5-9B-Artemis-Q4_K_M.gguf | 5.661 GB | Default. |

| Qwen3.5-9B-Artemis-Q5_K_S.gguf | 6.155 GB | Larger alternative. |

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Before choosing

The default Q4_K_M has worse WikiText perplexity than Unsloth Q4_K_M in the current-file replay (PPL ratios to BF16: 1.012493 versus 1.005114). IQ4_XS has direct fidelity measurements, but its historical task scores cannot be linked to the downloadable file and are omitted.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Run it

The tool-calling results use greedy decoding with thinking off. The quickstart and the Ollama packages do not set that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-9B-Artemis-Q4_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

In our test run (Q4_K_M file downloaded from this repository, CPU, llama.cpp b11617) the reply's tool_calls held get_weather with {"city":"Paris","unit":"celsius"}.

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-9b-artemis:q4_k_m

latest selects q4_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

| Tag | Hugging Face file | Size |

| --- | --- | --- |

| iq4_xs | Qwen3.5-9B-Artemis-IQ4_XS.gguf | 5.167 GB |

| q4_k_m (default) | Qwen3.5-9B-Artemis-Q4_K_M.gguf | 5.661 GB |

| q5_k_s | Qwen3.5-9B-Artemis-Q5_K_S.gguf | 6.155 GB |

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull. The accuracy comparisons use llama.cpp with thinking explicitly off.

Memory

Not measured. Download size is not a memory-fit guarantee; context length and runtime settings add to RAM or VRAM use.

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated. The benchmark comparisons used F16 K and V caches.

How it was tested

<details>

<summary>Measurements, comparisons and limitations</summary>

Fidelity of the downloadable files, 15 October 2026

This replay downloaded and SHA-256 checked each file, including IQ4_XS, and compared it with the original producer BF16 conversion of Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, without MTP (reference SHA-256 0afeac18c93f0814bf30613e6d1b4835fff4d7c69bf1ff31fc12e1486924c6b6). It used NVIDIA GB10, llama.cpp b11617, CUDA, flash attention on, and 60 chunks of 512 tokens each from WikiText-2 test and OASST1 validation. Every row completed both texts. These are reused evaluation texts, not a fresh independent test or proof of historical IQ4_XS tensor identity. The public Unsloth BF16 differs from this producer reference in 69 F32 values; it was not substituted for this replay.

KL, next-token agreement and 99th-percentile KL measure closeness to the original, including its mistakes. Perplexity measures prediction of the observed text; lower KL does not ensure lower perplexity. PPL ratio is llama.cpp’s reported mean PPL(Q)/PPL(base) relative to this BF16 reference, not an accuracy difference. Download sizes include unused MTP tensors where present.

| File | GB | KL wiki / chat | Same top token wiki / chat % | 99th pct KL wiki / chat | PPL ratio wiki / chat (lower is better) |

|---|---|---|---|---|---|

| Artemis IQ4_XS | 5.167 | 0.0207 / 0.0137 | 93.3 / 95.0 | 0.170 / 0.098 | 1.007853 / 1.006549 |

| Unsloth IQ4_XS | 5.169 | 0.0336 / 0.0206 | 92.0 / 94.1 | 0.282 / 0.151 | 1.008294 / 1.009433 |

| Artemis Q4_K_M | 5.661 | 0.0149 / 0.0089 | 94.1 / 96.0 | 0.115 / 0.063 | 1.012493 / 1.005059 |

| Unsloth Q4_K_M | 5.681 | 0.0277 / 0.0176 | 93.0 / 94.8 | 0.228 / 0.126 | 1.005114 / 1.009451 |

| Unsloth UD-Q4_K_XL | 5.966 | 0.0139 / 0.00815 | 94.8 / 96.3 | 0.107 / 0.060 | 1.010200 / 1.004317 |

| Artemis Q5_K_S | 6.155 | 0.0093 / 0.0058 | 95.1 / 96.7 | 0.062 / 0.036 | 1.011519 / 1.004657 |

| Unsloth Q5_K_S | 6.361 | 0.0095 / 0.0050 | 95.4 / 96.9 | 0.061 / 0.036 | 1.000858 / 1.004053 |

| bartowski Q4_K_M | 6.169 | 0.0165 / 0.0089 | 94.0 / 95.9 | 0.133 / 0.070 | 1.015557 / 1.006635 |

The table reports KL, next-token agreement and perplexity for each file on both texts. The larger Unsloth Q5_K_S has lower chat KL and higher agreement on both texts than our Q5_K_S. Our Q4_K_M has worse WikiText perplexity than Unsloth Q4_K_M. WikiText PPL ratios are 1.011519 for our Q5_K_S and 1.015557 for bartowski Q4_K_M.

The original saved task responses were also rescored offline in this audit; this is not new full-panel inference. Agreement with those saved counts does not resolve the IQ4_XS candidate-to-delivered lineage gap.

Earlier task measurements with verified tensor lineage

Q4_K_M and Q5_K_S retain the tensors of their historical evaluated candidates. The original IQ4_XS candidate-to-delivered linkage remains unresolved, so its historical BFCL, GSM8K and MMLU-Pro counts are omitted. The fidelity replay above measures the delivered IQ4_XS directly; it does not establish those old task scores.

The earlier task runs used NVIDIA GB10 and llama.cpp b11617, Qwen's original chat template, greedy decoding and thinking off. BFCL v3 simple used the native tools API and bfcl_eval 2026.3.23 AST checker; a response without a parsed tool call counted as wrong. Items 0–199 were development data and 200–399 were held out from that search. Six candidates ran on the latter panel; the published files were chosen on size and KL after those results were available. Treat this as selection-exposed evidence, not independent confirmation. GSM8K used the first 250 test problems; MMLU-Pro used 20 questions per category, 280 total, with letter-only answers. These two tasks used four concurrent requests on a shared GPU; batching can change greedy outputs. The local MMLU-Pro setup is not comparable to Qwen's published results.

| File | BFCL selection-exposed /200 | BFCL dev /200 | GSM8K /250 | MMLU-Pro /280 |

|---|---|---|---|---|

| BF16 original | 189 | 188 | 235 | 149 |

| Unsloth IQ4_XS | 187 | 188 | 234 | 143 |

| Artemis Q4_K_M | 189 | 190 | 235 | 151 |

| Unsloth Q4_K_M | 188 | 188 | 234 | 150 |

| Unsloth UD-Q4_K_XL | 186 | 191 | 235 | 148 |

| Artemis Q5_K_S | 191 | 189 | 234 | 147 |

| bartowski Q4_K_M | 188 | 190 | 236 | 153 |

Paired exact McNemar tests on the 200 selection-exposed tool tasks (only ours right / only the comparison right): Q4_K_M vs Unsloth Q4_K_M, 2/1 (p=1); Q5_K_S vs bartowski Q4_K_M, 3/0 (p=0.25). Against BF16: Q4_K_M 0/0 (p=1), Q5_K_S 2/0 (p=0.5). These p-values do not account for candidate selection. No task gain or equivalence is established; close totals and nonsignificant tests do not prove equal quality.

Fresh GB10 probe, 15 October 2026

This separate check used NVIDIA GB10, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, outside the reported selection categories; full historical tuning exposure is unknown. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

| Comparison (ours / public GB) | Mean KL ours / public | Lower mean KL | Correct calls /60 | Only ours / public | McNemar p |

| --- | --- | --- | --- | --- | --- |

| Our IQ4_XS / Unsloth IQ4_XS (5.167 / 5.169) | 0.022671 / 0.041234 | 45.0% | 47 / 49 | 2 / 4 | 0.6875 |

| Our Q4_K_M / Unsloth Q4_K_M (5.661 / 5.681) | 0.021976 / 0.032611 | 32.6% | 50 / 49 | 2 / 1 | 1 |

| Our Q5_K_S / bartowski Q4_K_M (6.155 / 6.169) | 0.009184 / 0.019150 | 52.0% | 49 / 49 | 0 / 0 | 1 |

| Our Q5_K_S / Unsloth Q5_K_S (6.155 / 6.361) | 0.009184 / 0.012905 | 28.8% | 49 / 49 | 1 / 1 | 1 |

BF16 scored 48/60. In table order, six-document sign-flip p-values are 0.03125, 0.0625, 0.03125, 0.15625. BFCL clustered p-values are 0.6875, 1, 1, 1. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 0.5, 1, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

Our IQ4 lost two correct calls to Unsloth IQ4 (47 versus 49); our Q5 tied bartowski Q4 at 49. Our default Q4 still has higher mean KL than the larger Unsloth dynamic Q4. Against plain Q4 its average improves, but the chat-debug document is a loss. Against Unsloth Q5_K_S, mean KL is lower overall but higher on code-panel.

<details>

<summary>Corrected authored tool diagnostic</summary>

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

| File | Correct /24 |

|---|---:|

| Our IQ4_XS | 24 |

| Our Q4_K_M | 24 |

| Our Q5_K_S | 24 |

| Unsloth BF16 | 24 |

| Unsloth IQ4_XS | 24 |

| Unsloth Q4_K_M | 24 |

| Unsloth UD-Q4_K_XL | 24 |

| Unsloth Q5_K_S | 24 |

| bartowski Q4_K_M | 24 |

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

</details>

Limits and trade-offs

• Unsloth UD-Q4_K_XL (5.966 GB) has lower KL and higher next-token agreement than our default Q4_K_M. Our Q5_K_S is larger than UD-Q4_K_XL; its download-size-matched comparison is bartowski Q4_K_M.
• Bartowski Q4_K_M contains a 15-tensor MTP block (Q8_0/F32), 258.6 MB, unused by plain decoding. Unsloth's compared files and ours have no MTP block. Matching download size is not matching active-weight budgets.
• Original task comparisons covered single-function BFCL simple with thinking off, not irrelevant tools, parallel/nested calls, follow-up turns, thinking mode or long-context quality. Separate smoke checks do not establish general quality for those settings.
• One GB10 and one llama.cpp release support these comparisons. No speed claim or measured runtime-memory guarantee is made.

Public comparison files are pinned to unsloth/Qwen3.5-9B-GGUF revision 3885219b6810b007914f3a7950a8d1b469d598a5 (IQ4_XS, Q4_K_M, UD-Q4_K_XL, Q5_K_S) and bartowski/Qwen_Qwen3.5-9B-GGUF revision 182be2f (Q4_K_M); full file hashes were checked. The newer Q5_K_S comparator has fidelity results only in this table; no historical task counts are invented for it.

File-level fidelity summary

These values repeat the measurements above; agreement measures closeness to BF16, not task accuracy.

| File | Same top token as BF16, WikiText-2 test |

| --- | --- |

| Qwen3.5-9B-Artemis-IQ4_XS.gguf | 93.3% |

| Qwen3.5-9B-Artemis-Q4_K_M.gguf | 94.1% |

| Qwen3.5-9B-Artemis-Q5_K_S.gguf | 95.1% |

</details>

<details>

<summary>Conversion and calibration sources</summary>

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp.

The WikiText evaluation and calibration use the same corpus, with test and train splits respectively. Different splits alone do not establish disjoint text.

Calibration used the first half of WikiText-2 raw train, split at an article boundary near the middle: Salesforce/wikitext revision b08601e0, wikitext-2-raw-v1, CC BY-SA 3.0 / GFDL. The importance matrix used 2,453 chunks of 512 tokens and llama-imatrix defaults, with llama.cpp b11617 (commit 14310190c).

The published importance matrix is Qwen3.5-9B-Artemis-imatrix.gguf. Each model is one standalone GGUF file. The importance-matrix file is calibration data for quantization, not needed to run the model; every model file in this repository was quantized with it. No calibration text is included in any file. Headers name TurinTech in general.quantized_by, the published matrix in quantize.imatrix.file, and the calibration data in quantize.imatrix.dataset. For Q4_K_M and Q5_K_S, full tensor-table and payload checks link the released files to the evaluated candidates; the release differences are header metadata. The historical IQ4_XS candidate-to-release linkage has not been verified, so its earlier task measurements should not be treated as verified measurements of the downloadable file. The fidelity table above instead comes from a direct, hash-verified replay of the delivered file.

</details>

<details>

<summary>File names, sizes and SHA-256 checksums</summary>

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes, not a description of every tensor: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below.

Stock llama-quantize dry runs from the same BF16, llama.cpp b11617, give IQ4_XS 4.66, Q4_K_S 4.77, Q4_K_M 5.02, Q5_K_S 5.62 and Q5_K_M 5.77 bits per weight. Our IQ4_XS (4.61) and Q5_K_S (5.49) follow the size-class rule. IQ4_XS is a K-quant mix with no IQ4_XS tensors. Our Q4_K_M (5.05) is an exception: 0.6% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (5,680,522,464 bytes) and bartowski's (6,169,341,984 bytes). It keeps the Q4_K_M label. Header general.file_type is 30 (IQ4_XS), 15 (Q4_K_M) or 16 (Q5_K_S). Rounded type shares can sum to more than 100%.

| File | Bits per weight | Main tensor types (share of tensor payload bytes) |

| --- | --- | --- |

| Qwen3.5-9B-Artemis-IQ4_XS.gguf | 4.61 | Q4_K 70.3%, Q5_K 13.6%, Q3_K 8.5%, Q6_K 7.5% |

| Qwen3.5-9B-Artemis-Q4_K_M.gguf | 5.05 | Q4_K 54%, Q5_K 27%, Q6_K 19% |

| Qwen3.5-9B-Artemis-Q5_K_S.gguf | 5.49 | Q5_K 60%, Q6_K 22%, Q4_K 18% |

File checksums

| File | Bytes | sha256 |

| --- | --- | --- |

| Qwen3.5-9B-Artemis-IQ4_XS.gguf | 5,167,440,736 | ccf73846d6c4fcfc0b947d25365dd6c5c8c9d9b436b5ad97903b9cceb3d6a242 |

| Qwen3.5-9B-Artemis-Q4_K_M.gguf | 5,661,139,808 | afa19e164e16f3496d4bbf85b2129e6d98102f6c41e8dc1e8bcda1f6b2234665 |

| Qwen3.5-9B-Artemis-Q5_K_S.gguf | 6,155,019,104 | c223fbf19c51805832e7bbb92537d17cfa70a26cdb4b816743ff311957264a06 |

| Qwen3.5-9B-Artemis-imatrix.gguf (importance matrix, not a model) | 10,307,840 | fbacab357c8a03745033439ea03ca232fe4b42129f71091c3c06bf0ceb9ef0ca |

</details>

Licence

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

Anyone serving these files to others is asked to mention TurinTech (turintech.ai) as the source of the quantization. This is a request, not a licence condition.

These files keep improving.