---

base_model:

- RedHatAI/gemma-4-31B-it-speculator.dspark

tags:

- gguf

- gemma-4

- speculative-decoding

- dspark

- llama.cpp

- draft-model

---

Gemma 4 31B-it DSpark Speculator — GGUF

GGUF conversion of a DSpark speculative-decoding draft model for google/gemma-4-31B-it.

> This repository does not contain a standalone language model. The GGUF file is a draft/speculator model and must be loaded together with a compatible Gemma 4 31B-it target model.

>

> If your target has a native MTP head available, use that instead. On this target and hardware, native MTP remains substantially faster and more consistent than either DSpark or DFlash. DSpark is mainly useful for targets without a compatible native MTP head, or for testing external speculator support.

Status

Support for DSpark-format speculators is being added to llama.cpp in ggml-org/llama.cpp#26275.

The latest benchmark below was re-run after the updated Speculators / DSpark handling in PR #26275. Compared with the previous result, DSpark throughput improved noticeably while its acceptance statistics remained effectively unchanged.

Until PR #26275 is merged, use a build compiled from the PR branch.

Compatibility

| Component | Requirement |

| ------------------- | --------------------------------------------------------- |

| Target architecture | Gemma 4 31B |

| Recommended target | google/gemma-4-31B-it or a compatible GGUF quantization |

| Draft format | DSpark |

| Runtime | llama.cpp with PR #26275 or later merged support |

| Standalone use | No |

The target model and speculator must use compatible tokenizer, vocabulary and architecture metadata. A mismatch can cause loading errors or incorrect output.

Recommended settings

speculative type: DSpark
draft max: 3

max 3 remains the best mixed-workload setting.

max 2 has the highest acceptance ratio but leaves some useful speculation depth unused. max 4 is still competitive, but aggregate throughput begins to decline. max 5 clearly over-drafts and loses performance.

llama-cli --help
llama-server --help

Command-line option names may change while PR #26275 is under review — check the help text of the build you are using.

Benchmark

Setup:

• Target: gemma-4-31B-it-UD-Q4_K_XL
• GPUs: RTX 5070 Ti 16GB + RTX 5060 Ti 16GB
• 13 requests
• Single-stream mixed workload
• Same benchmark workload used for DSpark / DFlash comparison

| Mode | Predicted | Draft | Accepted | Acceptance | Effective rate | vs. baseline |

| ---------------- | --------: | --------: | --------: | ---------: | --------------: | -----------: |

| No speculation | 1,990 | 0 | 0 | — | 26.16 tok/s | 1.00x |

| DSpark max 5 | 1,984 | 2,943 | 1,373 | 46.65% | 40.51 tok/s | 1.55x |

| DSpark max 4 | 1,984 | 2,469 | 1,345 | 54.48% | 43.64 tok/s | 1.67x |

| DSpark max 3 | 1,982 | 2,085 | 1,268 | 60.82% | 44.47 tok/s | 1.70x |

| DSpark max 2 | 1,982 | 1,610 | 1,157 | 71.86% | 44.17 tok/s | 1.69x |

| DFlash max 6 | 1,984 | 3,050 | 1,461 | 47.90% | 43.07 tok/s | 1.65x |

| Native MTP max 2 | 1,982 | 1,488 | 1,223 | 82.19% | 55.67 tok/s | 2.13x |

Change from the previous DSpark result

The previous conversion/runtime path produced:

| Mode | Previous | Updated | Change |

| ------------ | ----------: | --------------: | --------------------: |

| DSpark max 2 | 39.37 tok/s | 44.17 tok/s | +12.2% |

| DSpark max 3 | 40.77 tok/s | 44.47 tok/s | +9.1% |

| DSpark max 4 | 39.95 tok/s | 43.64 tok/s | +9.2% |

| DSpark max 5 | 37.24 tok/s | 40.51 tok/s | +8.8% |

| DFlash max 6 | 43.10 tok/s | 43.07 tok/s | essentially unchanged |

The especially interesting part is that DSpark's draft and acceptance statistics remained effectively unchanged while throughput increased.

For example, at max 3:

Predicted:  1,982
Draft:      2,085
Accepted:   1,268
Acceptance: 60.82%

Those values are unchanged from the previous run, yet effective throughput increased from 40.77 → 44.47 tok/s.

DFlash, meanwhile, stayed almost exactly where it was: 43.10 → 43.07 tok/s.

This strongly suggests that the improvement is primarily in the updated DSpark conversion/runtime path rather than improved speculative prediction quality or a general DFlash/speculative-decoding speedup.

Per-workload (tok/s)

| Workload | No spec | DSpark max 3 | DSpark max 4 | DFlash max 6 | Native MTP max 2 |

| ------------------- | ------: | -----------: | -----------: | -----------: | ---------------: |

| Python code | 27.2 | 47.5 | 52.8 | 57.6 | 64.7 |

| C++ code | 27.2 | 50.5 | 52.0 | 48.5 | 60.5 |

| Concept explanation | 27.4 | 40.2 | 33.8 | 30.4 | 53.5 |

| Summarization | 27.7 | 50.5 | 44.3 | 40.1 | 57.2 |

| Factual QA | 27.4 | 44.5 | 39.8 | 35.9 | 54.0 |

| Translation | 28.6 | 19.4 | 17.2 | 21.4 | 59.8 |

| Creative writing | 27.9 | 39.3 | 36.3 | 24.4 | 49.2 |

| Stepwise math | 27.4 | 57.5 | 59.7 | 56.2 | 66.0 |

| JSON output | 27.3 | 59.0 | 59.7 | 68.9 | 68.5 |

| Long reasoning | 27.3 | 52.6 | 51.0 | 57.1 | 65.4 |

| Repeated pattern | 27.4 | 63.2 | 71.6 | 83.2 | 68.7 |

| Code completion | 27.2 | 55.7 | 59.1 | 68.3 | 66.2 |

| Long code review | 26.4 | 37.8 | 33.1 | 30.0 | 52.5 |

Interpretation

Gemma 4 31B is a dense target, so each verified target token is relatively expensive. This gives speculative decoding substantial target computation to save.

With the updated DSpark path, max 3 takes the baseline from:

26.16 → 44.47 tok/s

or about:

1.70x baseline throughput

DSpark now slightly beats DFlash overall

The previous benchmark showed DFlash ahead:

DSpark max 3: 40.77 tok/s
DFlash max 6: 43.10 tok/s

After the updated DSpark conversion/runtime changes:

DSpark max 3: 44.47 tok/s
DFlash max 6: 43.07 tok/s

DSpark is now approximately 3.3% faster than DFlash on aggregate throughput in this mixed workload.

The margin is small enough that it should not be interpreted as a universal architectural win, but it reverses the previous result on this hardware and workload.

DSpark and DFlash have different workload strengths

DFlash remains particularly strong when output is highly predictable or structurally constrained:

• JSON output
• repeated patterns
• code completion
• Python code
• long reasoning

For example:

Repeated pattern:
DSpark max 3  63.2 tok/s
DFlash max 6  83.2 tok/s

JSON:
DSpark max 3  59.0 tok/s
DFlash max 6  68.9 tok/s

Code completion:
DSpark max 3  55.7 tok/s
DFlash max 6  68.3 tok/s

DSpark performs better across several less predictable or more mixed workloads:

• C++ code
• concept explanation
• summarization
• factual QA
• creative writing
• long code review

For example:

Concept explanation:
DSpark max 3  40.2 tok/s
DFlash max 6  30.4 tok/s

Factual QA:
DSpark max 3  44.5 tok/s
DFlash max 6  35.9 tok/s

Long code review:
DSpark max 3  37.8 tok/s
DFlash max 6  30.0 tok/s

So the updated result is not simply "DSpark is faster than DFlash."

A better interpretation is:

> DSpark now has the better mixed-workload aggregate result on this hardware, while DFlash still has a substantial advantage on highly predictable structured output.

Acceptance rate still does not predict throughput

At their tested sweet spots:

DSpark max 3:
acceptance = 60.82%
effective  = 44.47 tok/s

DFlash max 6:
acceptance = 47.90%
effective  = 43.07 tok/s

DSpark now converts its higher acceptance rate into a small aggregate throughput advantage, but acceptance alone still does not explain performance.

Proposal cost, accepted-run depth, rejection position, target verification cost, runtime overhead and hardware scheduling all matter.

A strict DSpark-vs-DFlash architecture comparison would still require:

• position-wise acceptance (P1, P2, P3, ...)
• accepted-run-length distribution
• per-round draft latency
• target verification latency
• identical training/checkpoint quality
• identical vocabulary pruning and quantization

Without those controls, aggregate acceptance is insufficient to explain why one speculator is faster.

Draft-depth scaling

The updated results also show that deeper DSpark drafting still has diminishing returns:

max 2: 44.17 tok/s
max 3: 44.47 tok/s
max 4: 43.64 tok/s
max 5: 40.51 tok/s

Accepted-token count continues to increase:

max 2: 1,157
max 3: 1,268
max 4: 1,345
max 5: 1,373

but draft work grows much faster:

max 2: 1,610
max 3: 2,085
max 4: 2,469
max 5: 2,943

The clearest example is max 4 → max 5:

+474 proposed draft tokens
+28 accepted draft tokens

That additional draft work is not economical on this hardware, and aggregate throughput falls from 43.64 → 40.51 tok/s.

So although the updated DSpark path is substantially faster overall, deeper-draft efficiency remains an open optimization area.

Translation remains a failure mode

Translation is still poor for both external speculators:

No speculation: 28.6 tok/s
DSpark max 3:    19.4 tok/s
DFlash max 6:    21.4 tok/s
Native MTP max 2: 59.8 tok/s

DSpark acceptance on translation is only about 2.4% at max 3.

Translation has high token-level uncertainty: many different token sequences can express a valid translation, making exact speculative agreement difficult for an external draft model.

For translation on this setup, disable DSpark/DFlash speculation unless a better workload-specific configuration is found.

Suggested use

| If your workload is mostly... | Recommended |

| ------------------------------------------------------------------- | ------------------------------------- |

| Mixed general-purpose workload | DSpark max 3 |

| C++ / QA / summaries / explanation / creative / code review | DSpark max 3 |

| JSON / repeated patterns / constrained generation / code completion | DFlash max 6 |

| Translation | Disable external speculative decoding |

| Native MTP available | Native MTP |

DSpark's position is now more competitive than in the previous benchmark.

On this consumer dual-GPU setup it slightly wins the mixed aggregate, while DFlash remains the better specialist for highly predictable output.

Reproducibility notes

Results depend on:

• target quantization
• GPU placement
• context length
• KV-cache type
• prompt/output distribution
• sampling parameters
• draft maximum
• speculator quantization
• llama.cpp revision
• converter revision

For meaningful comparisons:

1. Keep the target GGUF fixed.
2. Use identical prompts and stopping conditions.
3. Compare total predicted tokens divided by total wall time.
4. Re-run DFlash as well as DSpark after runtime changes.
5. Run multiple repetitions before trusting small differences.

The current DSpark-vs-DFlash aggregate gap is only about 3.3%, so it should be treated as a small measured advantage rather than a universal performance claim.

Limitations

• Draft model cannot generate output standalone.
• Speculative decoding does not reduce target model VRAM/storage requirements.
• Translation remains below the non-speculative baseline with DSpark and DFlash.
• Deeper DSpark draft lengths still show rapidly diminishing returns on this hardware.
• Native MTP remains substantially faster than either external speculator when available.
• llama.cpp DSpark support is tied to an unmerged PR at the time of writing.
• Vision/multimodal speculative decoding was not tested.
• Measurements were performed on consumer NVIDIA GPUs and may differ significantly on RTX PRO / datacenter GPUs or other backends.
• DSpark vs DFlash results also depend on checkpoint training quality and are not a controlled architecture-only comparison.

Credits

• Target model: Google DeepMind — Gemma 4 31B-it
• DSpark: DeepSeek AI — DeepSpec
• Original speculator: RedHatAI — gemma-4-31B-it-speculator.dspark
• Runtime integration: llama.cpp PR #26275
• GGUF conversion and consumer-GPU benchmark: williamliao

License

This repository contains a converted speculator checkpoint. Use is subject to the licenses and terms of the original speculator checkpoint, the Gemma target model, and the relevant software projects. Review all upstream licenses before redistribution or commercial use.