---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.5-4B/blob/main/LICENSE

base_model: Qwen/Qwen3.5-4B

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

tags:

• qwen
• qwen3.5
• gguf
• llama.cpp
• function-calling

---

Qwen3.5 4B Artemis GGUF

Text-only GGUF versions of Qwen3.5 4B for llama.cpp. The vision encoder and multi-token prediction (MTP) layer are not included.

Start with Qwen3.5-4B-Artemis-Q4_K_M.gguf (2.714 GB). Use llama.cpp with Qwen3.5 and HTTPS support:

llama-server -hf turintech/Qwen3.5-4B-Artemis-GGUF:Q4_K_M --port 8080 -c 8192 --jinja

<div>

<a href="https://turintech.ai"><img src="https://huggingface.co/turintech/Qwen3.5-4B-Artemis-GGUF/resolve/main/assets/turintech-logo.png" width="180" alt="TurinTech"></a>

<p style="margin: 4px 0;"><em>TurinTech models, built with Artemis</em></p>

</div>

Why use this file

• Run locally so inputs can stay on the same machine.
• Work offline after downloading the model and runtime.
• Use for local chat and tool-calling applications, within the tested limits below.
• Choose a compact GGUF download; file size is not a measurement of runtime memory.

Files

| File | Size | Choose for |

| --- | --- | --- |

| Qwen3.5-4B-Artemis-Q4_K_M.gguf | 2.714 GB | Default. |

| Qwen3.5-4B-Artemis-Q5_K_S.gguf | 2.903 GB | Larger alternative. |

| Qwen3.5-4B-Artemis-Q5_K_M.gguf | 3.011 GB | Larger alternative. |

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Before choosing

Results vary by task and runtime. On RTX 5090, the default Q4_K_M passed 215/270 instruction-following prompts versus 222/270 for BF16. On GB10 it answered 185/200 tool tasks correctly versus 190/200 for Unsloth Q4_K_M. Original BFCL results influenced file selection, and OASST1 validation informed calibration design.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Run it

The tool-calling results use greedy decoding with thinking off. The quickstart and the Ollama packages do not use that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-4B-Artemis-Q4_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

<!-- SMOKE4B --> The tool-call response is in choices[0].message.tool_calls.

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-4b-artemis:q4_k_m

latest selects q4_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

| Tag | Hugging Face file | Size |

| --- | --- | --- |

| q4_k_m (default) | Qwen3.5-4B-Artemis-Q4_K_M.gguf | 2.714 GB |

| q5_k_s | Qwen3.5-4B-Artemis-Q5_K_S.gguf | 2.903 GB |

| q5_k_m | Qwen3.5-4B-Artemis-Q5_K_M.gguf | 3.011 GB |

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull. The accuracy comparisons use llama.cpp with thinking explicitly off.

Memory

Not measured. Download size is not a memory-fit guarantee; context length and runtime settings add to RAM or VRAM use.

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated. The benchmark comparisons used F16 K and V caches.

How it was tested

<details>

<summary>Measurements, comparisons and limitations</summary>

These are controlled llama.cpp comparisons with the same Qwen chat template and thinking off for every file, not tests of each publisher's default application settings. A reply without a parsed tool call counts as wrong; that alone does not show whether the cause was generation, truncation or parsing.

Every file was compared with a BF16 conversion of Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The producer report lists an 8.67 GB BF16 reference and separately describes a quantization base converted with --no-mtp, reported to have identical logits to the full BF16. The report does not establish that the 8.67 GB figure describes the MTP-free artifact. The original measurements in this subsection used one Apple M3 Max (Metal) and llama.cpp b11617 (commit 14310190c). The historical BF16 artifact identity and logit-equivalence assertion were not independently established by this audit.

• KL and same top token as BF16: WikiText-2 test and OASST1 validation (human chat), 60 chunks of 512 tokens each.
• BFCL v3 simple native tool calling: llama-server's tools API, greedy, thinking off, BFCL's ast_checker, and Qwen's original chat template for every file. Items 0 to 199 were development data; items 200 to 399 were reserved initially and run once at the end.
• GSM8K: 250 items. MMLU-Pro: 280 questions, 20 per category.
• Run 2 rebuilt the same files with a second, non-overlapping calibration slice. A single value in the table is not a repeated measurement.

KL and next-token agreement measure closeness to the BF16 original, including its mistakes, not whether an answer or tool call is correct. This card reports mean KL, but not KL tails or perplexity.

Our MMLU-Pro subset uses a local setup (letter-only answers) and is not comparable to Qwen's published MMLU-Pro results.

Four of our files ran on the selection-exposed tasks; one was not published partly because its tool calls trailed BF16 (only ours right / only BF16 right: 2/10, p = 0.04). The selection-exposed results therefore influenced which files were published.

| File | GB | KL wiki / chat | Same top token wiki / chat % | BFCL dev /200 (run 1) | BFCL selection-exposed /200 | GSM8K /250 (run 1) | MMLU-Pro /280 |

|---|---|---|---|---|---|---|---|

| BF16 original | 8.67 | reference | 100 / 100 | 178 | 180 | 222 | 138 |

| Artemis Q4_K_M | 2.714 | .0178 / .0316 | 93.7 / 94.8 | 175 | 178 | 229 | 144 |

| Unsloth Q4_K_M | 2.741 | .0200 / .0346 | 93.3 / 94.8 | 172 | 178 | 221 | 132 |

| Artemis Q5_K_S | 2.903 | .0106 / .0218 | 95.0 / 95.5 | 178 | 183 | 221 | 139 |

| Unsloth UD-Q4_K_XL | 2.912 | .0155 / .0254 | 93.8 / 95.2 | 179 | 182 | 224 | 132 |

| Artemis Q5_K_M | 3.011 | .0079 / .0179 | 95.6 / 96.1 | 178 | 183 | 225 | 138 |

| bartowski Q4_K_M | 3.013 | .0177 / .0318 | 93.7 / 94.8 | 179 | 185 | 225 | 136 |

Repeat measurements with a second calibration slice

Run 2 rebuilt our files with a different calibration slice. These repeats check reproducibility; public comparator files were not rerun. They are not a separate leaderboard.

| Calibration rebuild | BFCL dev /200 (run 1 / run 2) | GSM8K /250 (run 1 / run 2) |

|---|---|---|

| Artemis Q4_K_M | 175 / 180 | 229 / 225 |

| Artemis Q5_K_S | 178 / 177 | 221 / 224 |

| Artemis Q5_K_M | 178 / 178 | 225 / 226 |

The table reports run 1 KL for each file and corpus. Run 2 KL for Artemis Q4_K_M, Q5_K_S and Q5_K_M is .0179/.0324, .0107/.0221 and .0078/.0169 respectively. Bartowski Q4_K_M records .0177/.0318 in run 1. These are file-level fidelity measurements, not task accuracy.

Paired exact McNemar tests on selection-exposed BFCL (only ours right / only theirs right): Q4_K_M vs Unsloth Q4_K_M, 5/5 (p = 1.0); Q5_K_S vs UD-Q4_K_XL, 4/3 (p = 1.0); Q5_K_M vs bartowski Q4_K_M, 2/4 (p = 0.69). Against BF16: Q4_K_M 5/7 (p = 0.77), Q5_K_S 4/1 (p = 0.38), Q5_K_M 5/2 (p = 0.45).

The paired tests describe these fixed files on this task set; their p-values do not account for choosing between candidate files. Not detecting a difference is not evidence that two files are equal.

Same per-item BFCL correctness outcome as BF16 over development plus selection-exposed (400 tasks): Q5_K_S 395, Q5_K_M 393, Q4_K_M 381, Unsloth UD-Q4_K_XL 387, Unsloth Q4_K_M 382, bartowski Q4_K_M 382. This is correctness agreement, not exact-call agreement: raw calls were not stored, so exact-call agreement was not measured.

Fresh CPU probe, 15 October 2026

This separate check used workstation CPU, eight inference threads, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, outside the reported selection categories; full historical tuning exposure is unknown. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

| Comparison (ours / public GB) | Mean KL ours / public | Lower mean KL | Correct calls /60 | Only ours / public | McNemar p |

| --- | --- | --- | --- | --- | --- |

| Our Q4_K_M / Unsloth Q4_K_M (2.714 / 2.741) | 0.021046 / 0.022298 | 5.6% | 48 / 49 | 1 / 2 | 1 |

| Our Q5_K_S / Unsloth UD-Q4_K_XL (2.903 / 2.912) | 0.012500 / 0.017593 | 28.9% | 49 / 49 | 0 / 0 | 1 |

| Our Q5_K_M / bartowski Q4_K_M (3.011 / 3.013) | 0.010497 / 0.019529 | 46.2% | 46 / 49 | 0 / 3 | 0.25 |

| Our Q5_K_M / Unsloth Q5_K_S (3.011 / 3.025) | 0.010497 / 0.010699 | 1.9% | 46 / 48 | 0 / 2 | 0.5 |

BF16 scored 49/60. In table order, six-document sign-flip p-values are 0.3125, 0.03125, 0.03125, 0.9375. BFCL clustered p-values are 1, 1, 0.25, 0.5. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 1, 0.5, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

Our Q4 mean KL is only modestly lower than plain Unsloth Q4, with losses on the code-runner and chat-debug documents. This is an average over these six fixed samples, not a win on every text. Our Q5_K_M scored 46/60 against bartowski Q4 at 49/60 (0/3 discordances, exact p=0.25), despite lower mean KL. Our Q4/Q5_K_S/Q5_K_M scores were 48/49/46, so the larger file did not improve this tool sample. Against Unsloth Q5_K_S, our Q5_K_M mean KL is only 1.9% lower and is higher on four of six documents (chat-travel, code-panel, prose-literature and prose-space); tool scores are 46/60 versus 48/60, with exact p=0.5.

<details>

<summary>Corrected authored tool diagnostic</summary>

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

| File | Correct /24 |

|---|---:|

| Our Q4_K_M | 24 |

| Our Q5_K_S | 24 |

| Our Q5_K_M | 24 |

| Unsloth BF16 | 24 |

| Unsloth Q4_K_M | 24 |

| Unsloth UD-Q4_K_XL | 24 |

| Unsloth Q5_K_S | 24 |

| bartowski Q4_K_M | 24 |

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

</details>

What is not better

• No significant tool-calling differences were detected in the original Metal panel. Bartowski Q4_K_M got 185 selection-exposed tasks right against our Q5_K_M's 183. Unsloth UD-Q4_K_XL got 182 against our Q4_K_M's 178, at a larger size.
• No task gain or equivalence is claimed. The evaluated staging files' paired BFCL comparisons against BF16 were not significant; GSM8K and MMLU-Pro cover small subsets.
• The pinned GGUF headers show a 15-tensor MTP block (Q8_0 and F32), 128.2 MB in bartowski's Q4_K_M, unused by plain decoding. Unsloth's files and ours have none. Bartowski's Q4_K_M is 2.885 GB without that block.

The original tool-calling comparison covered only single-function BFCL simple calls with thinking off: not choosing among several tools, parallel or nested calls, follow-up turns after a tool result, thinking mode or long contexts.

We make no speed claim for these files. In one decode test on an NVIDIA RTX 3060, the Q5_K_M file generated about 3% slower than bartowski's Q4_K_M.

Public files: unsloth/Qwen3.5-4B-GGUF revision e87f176479d0855a907a41277aca2f8ee7a09523 (Qwen3.5-4B-Q4_K_M.gguf, Qwen3.5-4B-UD-Q4_K_XL.gguf) and bartowski/Qwen_Qwen3.5-4B-GGUF revision 4168f45a16a1290d65a4ec0fa312ae917a4c15d6 (Qwen_Qwen3.5-4B-Q4_K_M.gguf).

Separate GB10 tool check

On NVIDIA GB10 with llama.cpp b11617 and four server slots, a separate check reused BFCL simple Python items 200–399 (200 tasks). This is a hardware re-check, not an independent task set. Results must remain separate from the original Metal comparison.

| File | Tool calls right (of 200) |

|---|---|

| Artemis Q4_K_M | 185 |

| Artemis Q5_K_S | 185 |

| Artemis Q5_K_M | 182 |

| Unsloth Q4_K_M | 190 |

| Unsloth UD-Q4_K_XL | 190 |

| bartowski Q4_K_M | 183 |

Q5_K_M trailed Unsloth UD-Q4_K_XL by 8 tasks; paired only-ours/only-theirs was 1/9 (unadjusted exact McNemar p=0.02148). Against bartowski it was 3/4 (unadjusted p=1). Q4_K_M versus Unsloth Q4_K_M was 0/5 (unadjusted p=0.0625); Q5_K_S versus Unsloth UD-Q4_K_XL was 1/6 (unadjusted p=0.125). These exploratory comparisons do not establish equivalence. Tool-irrelevance cases were not evaluated, so higher call rates do not establish when a tool should be avoided.

Broader checks: instruction following and coding (15 October 2026)

These separate checks used an NVIDIA RTX 5090, CUDA, llama.cpp b11617, the original Qwen template and thinking off. Server context was 32,768 total across eight slots, giving 4,096 tokens per slot. HumanEval+ used one greedy sample per problem with a 768-token generation cap and evalplus sanitisation; success requires both base and plus tests to pass. IFEval used a 1,280-token generation cap, temperature zero and eight concurrent requests. IFEval covers the first 270 of 541 prompts, not a random full-benchmark sample.

| File | IFEval strict prompt accuracy (correct of 270) | HumanEval+ accuracy (correct of 164) |

|---|---|---|

| BF16 original | 222 | 128 |

| Artemis Q4_K_M | 215 | 131 |

| Artemis Q5_K_S | 222 | 123 |

| Artemis Q5_K_M | 224 | 128 |

| Unsloth Q4_K_M | 218 | 121 |

| Unsloth UD-Q4_K_XL | 221 | 121 |

| bartowski Q4_K_M | 218 | 129 |

| mradermacher i1-Q4_K_M | 225 | 123 |

Q4_K_M passed 131/164 coding tasks versus 128/164 for BF16 and 121/164 for Unsloth Q4_K_M, but passed 215/270 strict instruction prompts versus 222/270 and 218/270 respectively. The larger files also have mixed results; no overall quality-preservation claim follows.

These counts were checked against saved aggregate receipts; inference and per-item scoring were not independently repeated in this audit. These counts describe this harness and generation budget, separately from the earlier benchmarks. Numerical best is not statistical superiority; a nonsignificant difference is not equivalence. These reviewed outcomes must be treated as development evidence if used to steer further compression searches.

File-level fidelity summary

These values repeat the measurements above; agreement measures closeness to BF16, not task accuracy.

| File | Same top token as BF16, WikiText-2 test |

| --- | --- |

| Qwen3.5-4B-Artemis-Q4_K_M.gguf | 93.7% |

| Qwen3.5-4B-Artemis-Q5_K_S.gguf | 95.0% |

| Qwen3.5-4B-Artemis-Q5_K_M.gguf | 95.6% |

</details>

<details>

<summary>Conversion and calibration sources</summary>

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp. The released files have matching tensor tables and payloads to the evaluated files, verified through retained identity receipts and current file hashes. The vision encoder is not included.

Calibration used 947 chunks of 512 tokens with llama.cpp b11617: Salesforce/wikitext wikitext-2-raw-v1 raw train (CC BY-SA 3.0 / GFDL), plus OpenAssistant/oasst1 train, English prompter-to-assistant pairs written by humans (Apache-2.0). Calibration used the OASST1 training split; chat KL used its validation split. Different splits do not establish that their text is disjoint. Validation KL informed the choice to add OASST1 training text to the calibration, so the chat KL is not an independent test of that choice. WikiText-2 calibration and evaluation use different splits of the same corpus.

The importance matrix is calibration data for quantization, not needed to run the model; the evaluated staging models used this matrix. Each model is one standalone GGUF file. No calibration text is included. The header dataset is wikitext-2-raw-v1 train + oasst1 train (human-written).

</details>

<details>

<summary>File names, sizes and SHA-256 checksums</summary>

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below. Stock llama-quantize dry runs from a BF16 of the same model, llama.cpp b11540, give IQ4_XS 4.79, Q4_K_S 4.86, Q4_K_M 5.13, Q5_K_S 5.67 and Q5_K_M 5.83 bits per weight.

Our Q5_K_S (5.50) and Q5_K_M (5.71) follow that rule. Q4_K_M (5.14) is an exception: 0.19% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (2,740,937,888 bytes) and bartowski's Q4_K_M (3.013 GB), so it keeps Q4_K_M. Header general.file_type is 15 (Q4_K_M), 16 (Q5_K_S) or 17 (Q5_K_M).

| File | Bits per weight | Main tensor types (share of tensor payload bytes) |

| --- | --- | --- |

| Qwen3.5-4B-Artemis-Q4_K_M.gguf | 5.14 | Q5_K 48%, Q4_K 40%, Q6_K 12% |

| Qwen3.5-4B-Artemis-Q5_K_S.gguf | 5.50 | Q5_K 81%, Q6_K 11%, Q4_K 8% |

| Qwen3.5-4B-Artemis-Q5_K_M.gguf | 5.71 | Q5_K 78%, Q6_K 22% |

File checksums

| File | Bytes | sha256 |

| --- | --- | --- |

| Qwen3.5-4B-Artemis-Q4_K_M.gguf | 2,714,078,432 | 9cc4043170052b9f99e56974ec0c5c63fbae36e562da031baea3658ad6348b05 |

| Qwen3.5-4B-Artemis-Q5_K_S.gguf | 2,902,822,112 | a3ad5db0ad10c3e41f36d8fd8803dff7727f7ca575acb1421cbdc5558bf705f1 |

| Qwen3.5-4B-Artemis-Q5_K_M.gguf | 3,010,956,512 | 930f7c1df65b2ee37a9af74698e695eea99c60429ed322d374f889988f9edd6d |

| Qwen3.5-4B-Artemis-imatrix.gguf (importance matrix, not a model) | 7,260,416 | f15970bb8a16aeb5781e88d236d1bd032654d4d646efc5910945138b8632be35 |

</details>

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

Licence

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

These files keep improving.