---

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.5-2B/blob/main/LICENSE

base_model: Qwen/Qwen3.5-2B

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

tags:

• qwen
• qwen3.5
• gguf
• llama.cpp
• function-calling

---

Qwen3.5 2B Artemis GGUF

Text-only GGUF versions of Qwen3.5 2B for llama.cpp. The vision encoder and multi-token prediction (MTP) layer are not included.

Start with Qwen3.5-2B-Artemis-Q5_K_M.gguf (1.395 GB). Use llama.cpp with Qwen3.5 and HTTPS support:

llama-server -hf turintech/Qwen3.5-2B-Artemis-GGUF:Q5_K_M --port 8080 -c 8192 --jinja

Keep the :Q5_K_M tag: untagged llama.cpp downloads and Hugging Face's generated Ollama import select Q4_K_M instead.

<div>

<a href="https://turintech.ai"><img src="https://huggingface.co/turintech/Qwen3.5-2B-Artemis-GGUF/resolve/main/assets/turintech-logo.png" width="180" alt="TurinTech"></a>

<p style="margin: 4px 0;"><em>TurinTech models, built with Artemis</em></p>

</div>

Why use this file

• Run locally so inputs can stay on the same machine.
• Work offline after downloading the model and runtime.
• Use for local chat and tool-calling applications, within the tested limits below.
• Choose a compact GGUF download; file size is not a measurement of runtime memory.

Files

| File | Size | Choose for |

| --- | --- | --- |

| Qwen3.5-2B-Artemis-Q5_K_M.gguf | 1.395 GB | Default. |

| Qwen3.5-2B-Artemis-Q4_K_M.gguf | 1.280 GB | Smaller download. |

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Before choosing

In the RTX 5090 HumanEval+ check, Q5_K_M passed 73/164 tasks and Q4_K_M 70/164, versus 81/164 for BF16. Unsloth Q4_K_M also scored higher on the earlier MMLU-Pro sample. The selected CodeSearchNet calibration functions still need a per-repository licence inventory; no calibration source text is included.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Run it

The tool-calling results include greedy and sampled decoding, both with thinking off. The quickstart and the Ollama packages do not set that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-2B-Artemis-Q5_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

In our test run (Q5_K_M file, CPU, llama.cpp b11617) the reply's tool_calls held get_weather with {"city":"Paris","unit":"celsius"}. The tool-call response is in choices[0].message.tool_calls.

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-2b-artemis:q5_k_m

latest selects q5_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

| Tag | Hugging Face file | Size |

| --- | --- | --- |

| q5_k_m (default) | Qwen3.5-2B-Artemis-Q5_K_M.gguf | 1.395 GB |

| q4_k_m | Qwen3.5-2B-Artemis-Q4_K_M.gguf | 1.280 GB |

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull. The accuracy comparisons use llama.cpp with thinking explicitly off.

Memory

Not measured. Download size is not a memory-fit guarantee; context length and runtime settings add to RAM or VRAM use.

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated. The benchmark comparisons used F16 K and V caches.

How it was tested

<details>

<summary>Measurements, comparisons and limitations</summary>

These are controlled llama.cpp comparisons with the same Qwen chat template and thinking off for every file, not tests of each publisher's default application settings. A reply without a parsed tool call counts as wrong; that alone does not show whether the cause was generation, truncation or parsing.

Every file was compared with a 3.776 GB BF16 conversion of Qwen/Qwen3.5-2B revision 15852e8c16360a2fea060d615a32b45270f8a8fc, without the MTP layer. The original comparisons in this section used an NVIDIA GTX 1080 Ti and llama.cpp b11540 (commit 1e6f04a, CUDA).

Data was split into tuning halves used to build and choose files, and held-out halves run at the end, once greedily and once sampled at temperature 0.7, top_p 0.95, with a fixed per-item seed. Five of our files ran on the held-out halves (four candidates and a rebuilt 1.34 GB file that is not published); the two published files were picked to match the sizes of the two public Q4_K_M files.

• KL divergence to BF16: C4 validation, 40 chunks of 1,024 tokens, and OPUS-100 test, 30 chunks of 1,024 tokens, on the held-out text.
• Native tool calling: BFCL v4 through llama-server's OpenAI-style tools API with --jinja, scored with BFCL's AST checker. Held-out tasks are the second half of each category: simple items 200 to 399, and items 100 to 199 of multiple, parallel and parallel_multiple, 500 in total. The first halves were tuning data. "Same calls as BF16" is the share of tasks with the same call as the original, not task accuracy.
• GSM8K and MMLU-Pro were also split into halves: the first halves were tuning data; the second, held-out halves contained 128 problems and 140 questions respectively.

In the table, g / s means greedy / sampled. Counts are correct answers, except the explicitly labelled same-call percentages.

KL and next-token agreement measure closeness to the BF16 original, including its mistakes, not whether an answer or tool call is correct. This card reports mean KL, but not KL tails or perplexity.

Our MMLU-Pro subset uses a local setup (letter-only answers) and is not comparable to Qwen's published MMLU-Pro results.

| File | GB | KL C4 / OPUS | Tools right greedy / sampled (of 500) | Same calls as BF16 g / s | GSM8K /128 g / s | MMLU-Pro /140 g / s |

|---|---|---|---|---|---|---|

| BF16 original | 3.776 | reference | 388 / 372 | 100 / 100% | 101 / 96 | 71 / 64 |

| Artemis Q5_K_M | 1.395 | .0070 / .0080 | 403 / 386 | 89.4 / 85.0% | 103 / 99 | 64 / 58 |

| bartowski Q4_K_M | 1.396 | .0184 / .0229 | 398 / 372 | 89.0 / 82.2% | 99 / 99 | 67 / 60 |

| Unsloth UD-Q4_K_XL | 1.340 | .0161 / .0185 | 363 / 342 | 85.6 / 77.2% | 102 / 94 | 65 / 65 |

| bartowski Q4_K_S | 1.328 | .0240 / .0300 | 352 / 336 | 80.6 / 73.4% | 99 / 101 | 66 / 63 |

| Artemis Q4_K_M | 1.280 | .0119 / .0137 | 400 / 384 | 87.8 / 80.2% | 100 / 94 | 64 / 60 |

| Unsloth Q4_K_M | 1.281 | .0218 / .0253 | 349 / 332 | 80.4 / 76.2% | 101 / 97 | 77 / 69 |

Paired exact McNemar tests count tasks only one file got right. Q4_K_M vs Unsloth Q4_K_M: 65/14 greedy and 68/16 sampled, both p < 1e-4. Q5_K_M vs bartowski Q4_K_M: 21/16 greedy (p = 0.51) and 37/23 sampled (p = 0.09), neither significant.

The paired tests describe these fixed files on this task set; their p-values do not account for choosing between candidate files. Not detecting a difference is not evidence that two files are equal.

Separate hardware re-check

This re-check reuses BFCL task IDs from the earlier panel, so it is not an independent task set. A separate check on another machine (NVIDIA GB10, llama.cpp b11617, Unsloth's BF16 as the reference) re-ran the published files' tensors on BFCL simple items 200 to 399 only (greedy, thinking off, four server slots) and measured KL on eight short synthetic documents never used for calibration or tuning. The synthetic documents vary one narrative template, so they are a narrow check, not broad text coverage.

| Comparison | Mean KL change | Tool calls right (of 200) | Only ours / only theirs right | p |

| --- | --- | --- | --- | --- |

| Artemis Q4_K_M vs Unsloth Q4_K_M | 43% lower | 171 vs 158 | 17 / 4 | 0.007 |

| Artemis Q5_K_M vs bartowski Q4_K_M | 47% lower | 170 vs 169 | 6 / 5 | 1.0 |

| Artemis Q5_K_M vs Unsloth UD-Q4_K_XL (1.340 GB) | 3% higher | 170 vs 161 | 13 / 4 | 0.049 |

Fresh CPU probe, 15 October 2026

This separate check used workstation CPU, eight inference threads, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, outside the reported selection categories; full historical tuning exposure is unknown. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

| Comparison (ours / public GB) | Mean KL ours / public | Lower mean KL | Correct calls /60 | Only ours / public | McNemar p |

| --- | --- | --- | --- | --- | --- |

| Our Q4_K_M / Unsloth Q4_K_M (1.280 / 1.281) | 0.016292 / 0.028885 | 43.6% | 48 / 44 | 7 / 3 | 0.3438 |

| Our Q5_K_M / bartowski Q4_K_M (1.395 / 1.396) | 0.010963 / 0.025387 | 56.8% | 45 / 45 | 2 / 2 | 1 |

| Our Q5_K_M / Unsloth UD-Q4_K_XL (1.395 / 1.340) | 0.010963 / 0.019526 | 43.9% | 45 / 44 | 3 / 2 | 1 |

| Our Q5_K_M / Unsloth Q5_K_S (1.395 / 1.385) | 0.010963 / 0.012118 | 9.5% | 45 / 44 | 3 / 2 | 1 |

BF16 scored 42/60. In table order, six-document sign-flip p-values are 0.03125, 0.03125, 0.03125, 0.1875. BFCL clustered p-values are 0.50781, 1, 1, 1. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 0.5, 0.5, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

This probe does not confirm a large tool-calling gain. Our Q5 and bartowski Q4 tied at 45/60. The KL ordering against dynamic Unsloth Q4 differs from the earlier synthetic probe above. Against the similarly sized Unsloth Q5_K_S, the mean KL gap is only 9.5%, with losses on chat-debug and prose-literature; tool scores are 45/60 versus 44/60.

<details>

<summary>Corrected authored tool diagnostic</summary>

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

| File | Correct /24 |

|---|---:|

| Our Q4_K_M | 24 |

| Our Q5_K_M | 24 |

| Unsloth BF16 | 24 |

| Unsloth Q4_K_M | 24 |

| Unsloth UD-Q4_K_XL | 24 |

| Unsloth Q5_K_S | 24 |

| bartowski Q4_K_M | 24 |

| bartowski Q4_K_S | 24 |

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

</details>

What is not better

• Unsloth's Q4_K_M scored higher than our Q4_K_M on the originally reported MMLU-Pro subset: 77 vs 64 of 140 greedy and 69 vs 60 sampled. Its greedy 77 also exceeds BF16's 71. The reported unadjusted paired p-values are 0.035 and 0.19; the original task-level records were unavailable for independent reconstruction. These observed counts favour the public file.
• The original GSM8K counts are descriptive. The original paired records were unavailable; this audit does not establish a lack of differences or equivalence.
• The table reports correct tool-call counts for each file, including BF16 (388/372). Closeness to BF16 rests on KL and "same calls as BF16".
• On the separate synthetic KL check, Unsloth's smaller UD-Q4_K_XL (1.340 GB) was slightly closer to BF16 than our Q5_K_M (3% lower mean KL), with correct-call counts for those 200 tasks reported separately.
• The pinned GGUF headers show a 15-tensor MTP block (Q8_0 and F32), 64.7 MB in bartowski's Q4_K_M, unused by plain decoding. Unsloth's files and ours have none.
• The original tool tests covered simple, multiple, parallel and parallel_multiple calls; those benchmarks did not test follow-up turns after a tool result, thinking mode or long contexts. The separate runtime probes have narrower scope than task-accuracy benchmarks. The original tool results cover one machine and one llama.cpp build; the separate checks below use other configurations. Apple Silicon was not tested. Tool-irrelevance cases were not evaluated, so these call rates do not establish when the model should avoid calling a tool.

We make no speed claim for these files. In one decode test on an NVIDIA RTX 3060, the Q5_K_M file generated about 3% slower than bartowski's Q4_K_M.

Public comparisons: unsloth/Qwen3.5-2B-GGUF revision f6d5376 (Qwen3.5-2B-Q4_K_M.gguf, Qwen3.5-2B-UD-Q4_K_XL.gguf) and bartowski/Qwen_Qwen3.5-2B-GGUF revision 7d26695 (Qwen_Qwen3.5-2B-Q4_K_M.gguf, Qwen_Qwen3.5-2B-Q4_K_S.gguf).

Broader checks: instruction following and coding (15 October 2026)

These separate checks used an NVIDIA RTX 5090, CUDA, llama.cpp b11617, the original Qwen template and thinking off. Server context was 32,768 total across eight slots, giving 4,096 tokens per slot. HumanEval+ used one greedy sample per problem with a 768-token generation cap and evalplus sanitisation; success requires both base and plus tests to pass. IFEval used a 1,280-token generation cap, temperature zero and eight concurrent requests. IFEval covers all 541 prompts.

| File | IFEval strict prompt accuracy (correct of 541) | HumanEval+ accuracy (correct of 164) |

|---|---|---|

| BF16 original | 352 | 81 |

| Artemis Q4_K_M | 356 | 70 |

| Artemis Q5_K_M | 359 | 73 |

| Unsloth Q4_K_M | 349 | 81 |

| Unsloth UD-Q4_K_XL | 360 | 85 |

| bartowski Q4_K_M | 345 | 77 |

| mradermacher i1-Q4_K_M | 351 | 76 |

Q4_K_M passed 70/164 coding tasks, versus 81/164 for both BF16 and Unsloth Q4_K_M. Q5_K_M passed 73/164, versus 81/164 for BF16 and 77/164 for bartowski Q4_K_M. These observed coding deficits remain trade-offs despite the tool/fidelity results above.

These counts were checked against saved aggregate receipts; inference and per-item scoring were not independently repeated in this audit. These counts describe this harness and generation budget, separately from the earlier benchmarks. Numerical best is not statistical superiority; a nonsignificant difference is not equivalence. These reviewed outcomes must be treated as development evidence if used to steer further compression searches.

File-level fidelity summary

These values repeat the measurements above; agreement measures closeness to BF16, not task accuracy.

| File | Same calls as BF16, greedy, BFCL 500 |

| --- | --- |

| Qwen3.5-2B-Artemis-Q5_K_M.gguf | 89.4% |

| Qwen3.5-2B-Artemis-Q4_K_M.gguf | 87.8% |

</details>

<details>

<summary>Conversion and calibration sources</summary>

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp.

The importance matrix used 443 chunks of 1,024 tokens of human-written text only:

• OpenAssistant/oasst2 train, revision 179dd21f: non-synthetic, best-ranked reply chains, Apache-2.0.
• Salesforce/wikitext, wikitext-103-raw-v1 train, revision b08601e0: CC BY-SA 3.0 / GFDL.
• code-search-net/code_search_net, Python validation functions, revision bd0cf261: per-repository licences. The selected-function licence inventory is still being reviewed; no calibration source text is included in the release.

The importance matrix and quantization used llama.cpp b11540 (commit 1e6f04a).

The published importance matrix is Qwen3.5-2B-Artemis-imatrix.gguf. Each model is one standalone GGUF file. The importance-matrix file is calibration data for quantization, not needed to run the model; every model file in this repository was quantized with it. No calibration text is included in any file. Headers name TurinTech in general.quantized_by, the published matrix in quantize.imatrix.file, and the calibration data in quantize.imatrix.dataset. Release header changes leave the evaluated tensor data unchanged.

</details>

<details>

<summary>File names, sizes and SHA-256 checksums</summary>

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes, not a description of every tensor: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below.

Stock llama-quantize dry runs from the same BF16, llama.cpp b11540 (commit 1e6f04a), give IQ4_XS 5.06, Q4_K_S 5.11, Q4_K_M 5.37, Q5_K_S 5.79 and Q5_K_M 5.95 bits per weight. Our Q5_K_M (5.88) follows the size-class rule. Our Q4_K_M (5.39) is an exception: 0.4% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (1,280,835,840 bytes), the file it was built to match. It keeps the Q4_K_M label. Header general.file_type is 17 (Q5_K_M) or 15 (Q4_K_M).

| File | Bits per weight | Main tensor types (share of tensor payload bytes) |

| --- | --- | --- |

| Qwen3.5-2B-Artemis-Q5_K_M.gguf | 5.88 | Q5_K 53%, Q6_K 43%, Q4_K 2%, IQ4_XS 1% |

| Qwen3.5-2B-Artemis-Q4_K_M.gguf | 5.39 | Q5_K 65%, Q4_K 17%, Q6_K 15%, IQ4_XS 3% |

File checksums

| File | Bytes | sha256 |

| --- | --- | --- |

| Qwen3.5-2B-Artemis-Q5_K_M.gguf | 1,394,786,208 | 00643c030243fd097ea7e3b3877f06c740562b4ed066964b2cd6d13c23c9820d |

| Qwen3.5-2B-Artemis-Q4_K_M.gguf | 1,279,573,920 | 86134884b896fdf3072c75a1028aa68361463e133be34b59453bcf4841205636 |

| Qwen3.5-2B-Artemis-imatrix.gguf (importance matrix, not a model) | 3,913,792 | 1800fbf6f3a53c4f337de051bd915b0b82cb6518caa376a8ab9f3a8e3e98043f |

</details>

Licence

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

Anyone serving these files to others is asked to mention TurinTech (turintech.ai) as the source of the quantization. This is a request, not a licence condition.

These files keep improving.