---

license: apache-2.0

base_model: stepfun-ai/Step-3.7-Flash

base_model_relation: quantized

pipeline_tag: image-text-to-text

tags:

• gguf
• mixed-quant
• step35
• step3p7
• multimodal
• dgx-spark

---

Step 3.7 Flash Mixed-Quant GGUF

Asymmetric quantization of the full Step 3.7 Flash language backbone, with

official Q8_0 MTP and F16 vision sidecars. No layers or experts are removed.

The main artifact targets approximately 83 GB to leave room for multimodal

and MTP runtime work on DGX Spark.

Native ds4-dfm-rs implements

MQ83 text/image serving and greedy speculative decoding with all three MTP

blocks on DGX Spark / GB10. Opt-in text banks now support MTP, live partial

prefix reuse and native disk snapshots; images use the serial fallback.

The bounded native tests below validate this artifact and these sidecars; they do not establish broad model quality or

MQ83-versus-BF16 fidelity. See the runtime's Step verification document

for commands, numerical results and current limits. Use the linked revision;

Step support is not yet part of a tagged runtime release.

| Weight component | Bytes | Decimal GB | GiB |

|---|---:|---:|---:|

| Main MQ83, 9 shards | 83,006,792,576 | 83.006793 | 77.306100 |

| Official external MTP Q8_0 | 3,707,276,416 | 3.707276 | 3.452670 |

| Official external vision F16 | 3,972,828,768 | 3.972829 | 3.699985 |

| All weight files | 90,686,897,760 | 90.686898 | 84.458755 |

Sizes exclude optional imatrix/reproduction files, KV cache and workspaces.

GB is 1,000,000,000 bytes; GiB is 1,073,741,824 bytes.

Tensor recipe

Layer numbers are zero-based. The tensor inventory, not the GGUF base preset

label, defines this mixed model.

| Tensor group | Type |

|---|---|

| Embedding and LM head | Q8_0 |

| Attention Q/K/V/O and attention gate projections | Q8_0 |

| Dense FFN, layers 0–2 | Q8_0 |

| All shared expert gate/up/down, layers 3–44 | Q8_0 |

| Router, norms, biases and small F32 controls | F32 |

| Routed gate/up/down, layers 3–6 and 41–44 | Q4_K |

| Routed gate/up, layers 7–40 | IQ2_XXS |

| Routed down, layers 7–40 | Q4_K |

| External MTP | Official Q8_0, unchanged |

| External vision | Official F16, unchanged |

The main model has 754 tensors and 83,001,512,448

payload bytes. Low-bit weights are quantized directly from official BF16 GGUF,

not from another low-bit release. Official MTP and vision files are copied

byte-for-byte with their published SHA-256 digests.

Sources and calibration

• Original model
• Official BF16 GGUF, imatrix and sidecars
• Reference/quantizer branch

The official imatrix contains 528 matched tensor groups, including all 126

routed gate/up/down groups. Every stored expert count is positive; all relevant

shapes and finite, nonnegative sums were checked. It records 9,969,664

text positions. No imatrix from a different model was reused.

A BF16 GPU supplement adds 75,184 positions from 96 real

DocVQA/ChartQA images and 32 text/code examples re-tokenized for Step. Prior

Inkling calibration material

provides the readable text; its token IDs and model-specific image tensors are

not reused. Media comes from pinned DocVQA/ChartQA training parquet files.

Both squared-activation sums and actual expert counts are reweighted to assign

70% nominal global token mass to the official pass and 30% to the supplement. Expert

means are not naively averaged. Raw passes, counts, selection manifests and

processed images are retained in the owner's private handoff. Weighted counts

are effective mass, not additional observations. The exact merged imatrix and

audit are public under calibration/.

Actual collector coverage is recorded per tensor group. Direct matmul batches

below 16 positions are skipped by the pinned collector (74,063 collected dense

positions). The last block FFN observes only requested output positions:

128 supplemental positions for routed experts, with four direct FFN/router

groups retaining official statistics alone. Sparse or absent supplemental

observations are covered by the complete official pass; per-group effective

mass therefore varies from the nominal 70/30 scaling.

The pinned tokenizer omitted ASCII ~ from its symbol category. The included

correction matches original tokenizer IDs on all 44 regression cases. Heldout

diagnostics use this correction. Supplemental calibration was

collected before the correction; its actual native token streams are preserved

and its statistics are not relabeled as a run with the corrected tokenizer.

Verification and limits

• Every main tensor name, shape, type, byte size, shard boundary and coverage

was checked against the locked plan; full output SHA-256 sums are included.

• 266 protected F32 tensors are byte-identical to

official BF16 GGUF; 1813 source rows match original

safetensors, including the explicit RMSNorm offset transformation.

• 1464 quantized rows were independently dequantized and

compared with the official BF16 source; detailed errors are published.

• BF16 reference fixtures are complete; full MQ83-versus-BF16 fidelity

evaluation remains pending. Native runtime validation compares the same

MQ83 artifact across implementations, not different quantization levels.

• The initial raw-dispatch gate's eight short text fixtures produce the same

72 greedy choices as pinned StepFun llama.cpp. Cross-engine logits are not bit-identical: the historical

3% relative-RMS criterion fails on 56/72 rows, with 17.60% worst error.

These differences were investigated and retained in the runtime report.

Enabling aligned BASE projections on a separate 2048-token workload changes

frontier logits by 4.38% relative RMS and changes the free-form continuation;

this is not strict cross-path token parity. The optimized server passes

bounded arithmetic, tool and image-answer checks.

• Native MTP tests cover accepted/rejected prefixes, all-vocabulary logits,

live KV, rewind and sliding-window wrap. Text and image continuations pass

against independent controls; MTP weights are not merely load-tested.

• Initial Rust-server validation covers 21 bounded image/text requests across

Chat Completions, Responses and Messages, including screenshots at three

resolutions, invoice text, a photograph, four-image streaming and follow-ups.

This is functional coverage, not a vision benchmark score.

• A separate 16384-input/1024-output MTP text run matches all frontier logits

and generated tokens with shared CUDA graph capture enabled/disabled. This

is bounded long-context evidence, not full source-context qualification.

Heldout images are digest-disjoint from supplemental calibration. Overlap with

the official text calibration cannot be audited at the document level.

Two 65536-context text banks passed bounded 6.3K-token concurrent requests,

including 4096-token partial reuse. Nine image/continuation requests passed

beside two 8192-context banks with chunk 512. These are configured contexts,

not full-capacity proofs. A 262144-context bank startup was stopped by the

memory guard. Chunk 4096 could not fit the serial image graph beside banks.

Audio and distributed slices remain unsupported; image sessions do not use

disk snapshots. Native text disk payloads round-trip, but the tested Chat

follow-up after restart was cold because the official template removes the

prior thinking block. See the serving contract

for the verified configurations and cache limitations.

Image input uses the F16 projector and the native Rust processor; no Python

processor is required. Set reasoning_effort: "none" and temperature: 0

for greedy MTP; sampled reasoning and forced protocol prefixes use ordinary

decode.

Native performance

On one GB10 locked to 300–2200 MHz, fresh matched A/Bs use the same

MQ83/Q8-MTP artifacts, shared BASE+MTP weight owner, unchanged

promessi_sposi fixture and greedy 64-token output. Each arm has three

fresh processes after separate identical warmups, alternating A/B order.

Earlier uncapped numbers are not the control for these results.

| Workload | Prefill, tok/s | Decode, tok/s |

|---|---:|---:|

| 2048 input + 64 output, MTP draft 3 | 1194.64 (+0.72%) | 22.77 (+4.02%) |

| 16384 input + 64 output, MTP draft 3 | 1290.01 (+7.22%) | 16.34 (+14.59%) |

| 16384 input + 64 output, ordinary P2 control | 1290.74 | 18.41 |

Parenthesized changes compare the final binary with legacy switches and

chunk 2048. The 2K aggregate is exact in logits/tokens. The 16K Decode gain

includes a changed continuation and draft acceptance. Ordinary P2 control

uses MTP loaded but speculation disabled; it is faster than MTP for this

workload. Its row isolates the chunk change, not the full aggregate.

Two isolated Decode rounds improve their medians by 1.97% and 2.02%;

two Prefill rounds improve by 1.52% (2K input) and 6.00% (16K input).

These percentages are separate comparisons and must not be added together.

The decode changes and exact Q8 reuse preserve every frontier logit and

output token. Chunk 2048 → 4096 changes floating-point execution: 4.56%

frontier relative-RMS delta and a later continuation change. All four bounded

long retrieval/arithmetic answers remain correct; strict JSON type matching

is 3/4 versus 4/4 because one answer returns "42" instead of 42.

KV and rewind controls pass. This is not broad quality or strict-schema parity.

Default Step prefill chunk is 4096. Use DS4_STEP37_PREFILL_CHUNK=2048

for more bank headroom, or the documented 512-chunk mixed image/bank setup.

Startup, image inference and cache reuse are outside these timed text rows.

See the capped-clock report and raw samples.

Download

hf download Baekpica/Step-3.7-Flash-Mixed-Quant-GGUF \
  --include 'MQ83/*' --include 'MTP/*' --include 'vision/*' \
  --include 'tokenizer*.json' --include 'special_tokens_map.json' \
  --include 'chat_template.jinja' --include 'STEP37-CONTRACT.md' \
  --local-dir ./Step-3.7-Flash-Mixed-Quant-GGUF

Run sha256sum -c SHA256SUMS inside each downloaded component directory.

For native serving, pass the first MQ83 shard to ds4-server --cuda, the

vision file with --vision, and MTP with --mtp ... --mtp-draft 3. Use the

shared BASE+MTP weight owner documented by ds4-dfm-rs. The runtime

default Step prefill chunk is 4096. Root tokenizer assets

and copies of the official Jinja/config beside MQ83/ are included.

The pinned StepFun llama-mtmd-cli remains the reference path; use the build

and calibration scripts for its commands.

The MTP sidecar is not a standalone model despite its 48-layer metadata;

it contains prediction blocks 45–47 plus shared tensors.

See STEP37-CONTRACT.md for native ds4 integration details.

Reproduction scripts currently use /step37; set up that root or adjust paths.

The private handoff includes portable fixture-path restoration and raw evidence.