a4lg/Step-3.7-Flash-MTP-ONLY-GGUF overview

---

library_name: gguf

license: apache-2.0

base_model:

• stepfun-ai/Step-3.7-Flash

base_model_relation: quantized

tags:

• step3p7
• speculative-decoding
• mtp
• mtp only
• gguf

---

MTP-only GGUF subset of Step-3.7-Flash

=======================================

This is a supplement for Step-3.7-Flash-based quants without MTP tensors.

This repository contains an MTP-only subset of stepfun-ai/Step-3.7-Flash

which provides the draft model for speculative decoding in the GGUF format.

It accelerates token generation using speculative decoding with the draft model

from the original Step-3.7-Flash model. In most cases, this is sufficient to

accelerate derivative models even if this draft model is not trained from them.

Note that however, the performance metrics heavily depend on the derivative

model you use, your machine and your MTP settings.

Benchmark it before blindly trusting it.

Using this Model

-----------------

As grafting MTP tensors into an existing GGUF file is not easy,

I describe how to use this model as a separate draft model file.

It is easier than grafting but memory-inefficient as it does not share

some tensors with the original model.

If you use llama-server, you may configure like this:

llama-server \
    --model       Step-3.7-Flash-IQ4_NL.gguf \
    --model-draft Step-3.7-Flash-MTP-ONLY-Q4_K_M.gguf \
    ... \
    --spec-type draft-mtp
    --spec-draft-n-max 2

--model specifies the original Step-3.7-Flash-based quant and

new --model-draft specifies a file from this repository.

You also need --spec-type draft-mtp to enable the draft model.

Once the draft model is enabled, you may configure the rest of MTP options

as you like (in this example, custom --spec-draft-n-max is specified).

Additional Quantization (Q4_K_M, Q5_K_M, Q6_K and Q8_0)

----------------------------------------------------------------

Quantized GGUF files are provided so that deploying the draft model easier.

As no Unsloth MTP quants for Step-3.7-Flash is released,

tensor types are automatically chosen by llama-quantize.

It is not required to match the quantization level.

For instance, you may pair Q6_K-quantized draft model with

the IQ4_NL-quantized main model.

Conversion Process

-------------------

• Tools: llama.cpp (b9851)
• With: Patched conversion/base.py

1. The if block right after

# verify tensor name presence and identify potentially missing files

is commented out.

2. Handle

chkhsh == "5841594bd6a8eeecd7207aeec6570831cc97ffaeba51e908bdaf560113177bae"

case as deepseek-v3.

The first modification is performed because the author of this repository

downloaded only a subset of the full Step model while the original

convert_hf_to_gguf.py expects the full model.

The second modification is because convert_hf_to_gguf.py does not support

Step-3.7-Flash pre-tokenizer.

The --mtp option of convert_hf_to_gguf.py is the crucial part of this

conversion process because this option does exactly what the author expects:

create an MTP-only GGUF subset.

For additional quantization, the llama-quantize tool (llama.cpp)

is used without options.

License and Copyright

----------------------

For all GGUF files under this repository,

the license terms of the original Step-3.7-Flash model (Apache License version 2.0)

applies (as the author of this repository did not perform any changes

significant enough for own copyright):

> © 2026 StepFun.

Note: No copyright notice found on the original model repository.

the notice is borrowed partially from their official blog post at

<https://static.stepfun.com/blog/step-3.7-flash/>.

No NOTICE files are attached in the original model.

This README file is licensed under the terms of CC-BY-4.0.

> Copyright 2026 a4lg.

Links: Sources

---------------

• The Original Model: stepfun-ai/Step-3.7-Flash