---

quantized_by: tarruda

pipeline_tag: text-generation

base_model: stepfun-ai/Step-3.7-Flash

base_model_relation: quantized

license: apache-2.0

license_link: https://www.apache.org/licenses/LICENSE-2.0.txt

tags:

• gguf
• imatrix
• conversational
• multimodal
• vision
• llama.cpp

---

Step 3.7 Flash GGUF

My custom IQ4_XS GGUF quantization for

stepfun-ai/Step-3.7-Flash

I've also modified the chat template also adds a preserve_thinking option,

which preserves thinking across user turns and can improve the experience when

prompt processing speed is a bottleneck.

Quant Recipes

| Recipe | Quant Size | Default type | Tensor-specific overrides |

| --- | --- | --- | --- |

| IQ4_XS | 101784.88 MiB (4.34 BPW) | Q6_K | ffn_down_exps=iq4_xs, ffn_gate_exps=iq4_xs, ffn_up_exps=iq4_xs |

Related Files

| File | Description |

| --- | --- |

| Step-3.7-Flash-MTP-Q8_0.gguf | Q8_0 MTP weights |

| Step-3.7-Flash-mmproj-BF16.gguf | BF16 multimodal projector |

| Step-3.7-Flash-mmproj-F16.gguf | F16 multimodal projector |

| Step-3.7-Flash-mmproj-Q8_0.gguf | Q8_0 multimodal projector |

Usage

Here's an example script:

#!/bin/sh -e

model="./IQ4_XS/Step-3.7-Flash-IQ4_XS-00001-of-00004.gguf"
mmproj="./Step-3.7-Flash-mmproj-Q8_0.gguf"
mtp=./Step-3.7-Flash-MTP-Q8_0.gguf
 
ctx=262144
parallel=1

ctx_size=$((ctx * parallel))

reasoning_budget_message="...

Actually, I will stop now.

Let me provide the user with a comprehensive answer."

llama-server --no-mmap --no-warmup --model $model --mmproj $mmproj \
  --ctx-size $ctx_size -np $parallel --temp 1.0 --top-p 0.95 \
  --repeat-penalty 1.0 --presence-penalty 0.0 \
  --reasoning-budget-message "$reasoning_budget_message" \
  --reasoning-preserve \
  --spec-type draft-mtp -md $mtp --spec-draft-n-max 3 --spec-draft-p-min 0.65 \
  -ctxcp 8 --checkpoint-min-step 512 \
  --cache-ram 4096