---

base_model:

• flashback2k/FlashModel-Qwen3.5-9B

base_model_relation: quantized

library_name: gguf

license: apache-2.0

license_link: https://huggingface.co/Qwen/Qwen3.5-9B/blob/main/LICENSE

pipeline_tag: text-generation

language:

• en

tags:

• gguf
• imatrix
• llama.cpp
• qwen3.5
• reasoning
• tool-calling
• distillation

datasets:

• nvidia/Nemotron-SFT-Math-v4
• nvidia/Nemotron-SFT-Competitive-Programming-v2
• nvidia/Nemotron-SFT-Agentic-v2
• nvidia/Nemotron-SFT-Instruction-Following-Chat-v3
• nvidia/Nemotron-SFT-Science-v2

quantized_by: flashback2k

---

⚡ FlashModel-Qwen3.5-9B — GGUF

imatrix GGUF quants of FlashModel-Qwen3.5-9B, a reasoning / coding / tool-calling

fine-tune of Qwen/Qwen3.5-9B distilled from open-weight frontier teachers.

• All quants use an importance matrix computed on this fine-tune (not borrowed from the base model).
• *_K_XL quants use a dynamic per-tensor layout mirrored tensor-for-tensor from

unsloth/Qwen3.5-9B-GGUF (see How these were made).

• Every file was loaded and ran a generation test in llama.cpp before upload.
• Text-only: the fine-tune does not include the vision tower, so there is no mmproj.

Provided Quants

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

| Link | Type | Size/GB | Notes |

|:-----|:-----|--------:|:------|

| GGUF | IQ3_M | 4.42 | smallest here |

| GGUF | Q3_K_M | 4.62 | lower quality |

| GGUF | Q3_K_XL | 5.05 | dynamic layout |

| GGUF | IQ4_XS | 5.20 | good size/quality trade-off |

| GGUF | Q4_K_S | 5.35 | fast |

| GGUF | IQ4_NL | 5.42 | |

| GGUF | Q4_K_M | 5.63 | fast, recommended |

| GGUF | Q4_K_XL | 5.97 | dynamic layout, recommended |

| GGUF | Q5_K_S | 6.31 | |

| GGUF | Q5_K_M | 6.47 | |

| GGUF | Q5_K_XL | 6.74 | dynamic layout, highest precision here |

| imatrix | imatrix | 0.01 | importance matrix used for every quant above |

Which one should I pick?

| Your free VRAM / RAM | Pick |

|:--|:--|

| ≥ 10 GB | Q5_K_XL |

| ~8 GB | Q4_K_XL — default choice |

| ~7 GB | IQ4_XS or Q4_K_S |

| ≤ 6 GB | Q3_K_XL, then IQ3_M |

Leave room for the KV cache: long reasoning traces need context. Size numbers are file sizes, not total memory use.

Usage

llama.cpp

# download one quant
huggingface-cli download flashback2k/FlashModel-Qwen3.5-9B-GGUF \
  FlashModel-Qwen3.5-9B-Q4_K_XL.gguf --local-dir .

# chat (thinking mode, recommended sampling from Qwen3.5)
llama-cli -m FlashModel-Qwen3.5-9B-Q4_K_XL.gguf --jinja -ngl 99 -c 32768 \
  --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 1.5

# OpenAI-compatible server with tool calling
llama-server -m FlashModel-Qwen3.5-9B-Q4_K_XL.gguf --jinja -ngl 99 -c 32768 --port 8080

Or pull straight from the Hub: llama-server -hf flashback2k/FlashModel-Qwen3.5-9B-GGUF:Q4_K_XL --jinja

Ollama

ollama run hf.co/flashback2k/FlashModel-Qwen3.5-9B-GGUF:Q4_K_XL

LM Studio

Search for flashback2k/FlashModel-Qwen3.5-9B-GGUF and pick a quant.

Recommended sampling

These are the base model's official recommendations (Qwen3.5 model card); they were not re-tuned for the fine-tune.

| Mode | temperature | top_p | top_k | min_p | presence_penalty |

|:--|--:|--:|--:|--:|--:|

| Thinking, general | 1.0 | 0.95 | 20 | 0.0 | 1.5 |

| Thinking, precise coding | 0.6 | 0.95 | 20 | 0.0 | 0.0 |

| Non-thinking, general | 0.7 | 0.8 | 20 | 0.0 | 1.5 |

Always pass --jinja so the embedded Qwen3.5 chat template (thinking blocks, tool calls) is used.

Reasoning-budget tag

Training system prompts started with a reasoning-budget tag chosen from the length of the teacher's reasoning:

<|reasoning_budget|>medium<|/reasoning_budget|>

Values: off, low, medium, high, xhigh, max. The tag's effect on output length has not been measured yet; treat it as experimental.

About the fine-tune

| | |

|:--|:--|

| Base | Qwen/Qwen3.5-9B |

| Method | LoRA r=128 (RSLoRA, α=32) on attention + MLP projections, merged |

| Data | 9,638 examples / 45M tokens, loss on assistant turns only |

| Context in training | up to 16,384 tokens (longer examples dropped, never truncated) |

| Schedule | 1 epoch, 600 steps, lr 5e-5 cosine |

| Held-out eval loss | 0.5745 (step 100) → 0.5586 (step 600) |

Training data and teachers

Only open-weight teachers whose licenses allow training on their outputs:

| Share | Domain | Dataset | Teacher |

|--:|:--|:--|:--|

| 40% | Math (answers verified against references) | nvidia/Nemotron-SFT-Math-v4 | DeepSeek-V4-Pro |

| 22% | Competitive programming (Python) | nvidia/Nemotron-SFT-Competitive-Programming-v2 | DeepSeek-R1-0528 |

| 16% | Multi-turn tool calling (judge-filtered) | nvidia/Nemotron-SFT-Agentic-v2 | GLM-4.6 / DeepSeek-V3.2 |

| 11% | Instruction following | nvidia/Nemotron-SFT-Instruction-Following-Chat-v3 | GPT-OSS-120B |

| 11% | Science reasoning | nvidia/Nemotron-SFT-Science-v2 | GPT-OSS / Kimi-K2 / DeepSeek-V3.2 |

Datasets © NVIDIA, CC BY 4.0 (some Math StackExchange-derived samples CC BY-SA 4.0).

⚠️ Status and known limitations

• No benchmarks yet. Lower held-out loss means the model imitates the teachers more closely; it does not by itself prove

it beats the stock Qwen3.5-9B. Comparative evals are planned and will be added here.

• English-centric. All training data was English. On non-English prompts (e.g. Russian) the model often reasons in English.
• Math and code examples longer than 16k tokens were excluded, which skews those domains toward shorter problems.
• Inherits the base model's limitations and biases.

How these were made

1. Merged weights → BF16 GGUF with llama.cpp convert_hf_to_gguf.py (build b11005-82324fc50).
2. imatrix: llama-imatrix on the BF16 model over ~131k tokens (256 × 512) of held-out training-distribution text

(math, code, tool calls, instructions, science). A Q8_0 source produced non-finite activations — the stock

Qwen3.5-9B Q8_0 does too — so BF16 was used.

1. Standard quants: llama-quantize --imatrix.
2. *_K_XL quants: Unsloth has not published its Dynamic quantization tool, but the resulting per-tensor types are visible in

its GGUF headers. The types of all 427 tensors were read from unsloth/Qwen3.5-9B-GGUF (UD-Q3_K_XL, UD-Q4_K_XL,

UD-Q5_K_XL), applied with llama-quantize --tensor-type-file, and verified tensor-for-tensor after quantization.

The layout keeps the most sensitive tensors at higher precision — e.g. in Q4_K_XL: Gated DeltaNet ssm_out at Q8_0,

ssm_alpha/ssm_beta at F16, attn_v at Q6_K, about half of ffn_down at Q6_K. The imatrix is this model's own.

These are not official Unsloth quants and are not affiliated with or endorsed by Unsloth.

1. Each quant loaded in llama.cpp (CUDA) and generated a reply before upload.

Credits

Qwen team for Qwen3.5 · NVIDIA for the Nemotron SFT datasets ·

DeepSeek, Zhipu AI (GLM), OpenAI (GPT-OSS) and Moonshot AI (Kimi) for open-weight teachers ·

ggml-org/llama.cpp · Unsloth for the published dynamic-quant layouts ·

quant-table format inspired by mradermacher.