---

base_model:

- deepseek-ai/dspark_gemma4_12b_block7

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- speculative-decoding

- dspark

- gemma4

- draft-model

license: other

---

Gemma 4 12B DSpark GGUF

GGUF conversion of the DeepSeek DSpark draft model for google/gemma-4-12B-it

(deepseek-ai/dspark_gemma4_12b_block7, block size 7).

This is a draft model only. It cannot be used by itself. Pair it with a

compatible Gemma 4 12B target model and run it with DSpark speculative

decoding in llama.cpp.

> [!IMPORTANT]

> Experimental, and **not directly comparable to other DSpark cards in

> this collection.** This model uses llama.cpp PR

> #25549 — the native

> Gemma 4 DSpark implementation — not PR

> #26275 (speculators-format

> checkpoint support) used for the 26B-A4B and 31B DSpark cards. Different PR,

> different upstream checkpoint, different block size, and this benchmark also

> enables confidence gating (--spec-draft-p-min 0.60) that the other cards

> do not use. Don't read the acceptance/speedup numbers below as an

> apples-to-apples architecture comparison against those cards — the delta

> could easily come from any of those four variables, not from target size.

Files

| File | Purpose |

|---|---|

| gemma4_12b-DSpark-Q4_K_M.gguf | Quantized DSpark draft model |

| gemma4_12b-DSpark-BF16.gguf | BF16 source conversion, when provided |

Recommended target: google/gemma-4-12B-it (may be quantized). Draft and

target must use the same tokenizer/model family.

Conversion

hf download google/gemma-4-12B-it --local-dir Gemma4-12B-it
hf download deepseek-ai/dspark_gemma4_12b_block7 --local-dir dspark_gemma4_12b_block7

python convert_hf_to_gguf.py dspark_gemma4_12b_block7 \
  --outtype bf16 \
  --target-model-dir Gemma4-12B-it \
  --outfile gemma4_12b-DSpark-BF16.gguf

llama-quantize gemma4_12b-DSpark-BF16.gguf gemma4_12b-DSpark-Q4_K_M.gguf Q4_K_M

--target-model-dir must point at the target checkpoint (Gemma4-12B-it),

not the DSpark directory — the draft checkpoint doesn't ship its own tokenizer.

llama.cpp usage

llama-server.exe ^
  -m "gemma-4-12B-it-Q4_K_M.gguf" ^
  --model-draft "gemma4_12b-DSpark-Q4_K_M.gguf" ^
  --host 0.0.0.0 ^
  --port 18010 ^
  --device-draft CUDA1 ^
  --threads 12 ^
  --threads-batch 12 ^
  --split-mode layer ^
  --tensor-split 14,12 ^
  --ctx-size 65536 ^
  --no-mmap ^
  -ngl -1 ^
  --flash-attn on ^
  --ubatch-size 256 ^
  --batch-size 2048 ^
  --cache-type-k q8_0 ^
  --cache-type-v q8_0 ^
  --temp 1.0 ^
  --top-k 64 ^
  --top-p 0.95 ^
  --reasoning off ^
  --fit off ^
  --parallel 1 ^
  --spec-type draft-dspark ^
  --spec-draft-n-max 7 ^
  --spec-draft-p-min 0.60

--spec-draft-n-max 7 matches the checkpoint's block size 7.

--spec-draft-p-min 0.60 gates low-confidence proposals before verification —

see Interpretation for why this matters for the acceptance

numbers below.

Benchmark

Setup: RTX 5070 Ti 16GB + RTX 5060 Ti 16GB, target Gemma 4 12B-it Q4_K_M,

context 65,536, temp 1.0/top-k 64/top-p 0.95, reasoning off, 1 parallel slot,

mtp-bench.py, mostly 192 tokens/test. Draft only tested at Q4_K_M — no

BF16/Q8_0 comparison was run (see Limitations).

Not directly comparable to the BF16 target benchmarks in the upstream PR, or

to the 26B-A4B / 31B DSpark cards (different PR/checkpoint/config — see the

warning above).

| Test | No draft | DSpark | Speedup | Acceptance |

|---|---:|---:|---:|---:|

| code_python | 74.9 | 124.8 | 1.67× | 81.0% |

| code_cpp | 75.0 | 112.1 | 1.49× | 77.5% |

| explain_concept | 74.4 | 89.2 | 1.20× | 72.3% |

| summarize | 75.5 | 99.0 | 1.31× | 58.6% |

| qa_factual | 74.9 | 91.1 | 1.22× | 58.6% |

| translation | 75.5 | 81.8 | 1.08× | 53.6% |

| creative_short | 75.7 | 75.4 | 1.00× | 73.9% |

| stepwise_math | 74.8 | 155.4 | 2.08× | 83.1% |

| json_output | 74.4 | 184.5 | 2.48× | 94.7% |

| long_reasoning | 74.5 | 102.2 | 1.37× | 81.5% |

| repeat_pattern | 74.7 | 196.8 | 2.63× | 93.1% |

| code_completion | 74.2 | 140.9 | 1.90× | 83.2% |

| long_code_review | 72.5 | 83.2 | 1.15× | 75.4% |

Aggregate

| Metric | No draft | DSpark |

|---|---:|---:|

| Predicted tokens | 2,015 | 2,016 |

| Draft tokens | 0 | 1,787 |

| Accepted draft tokens | 0 | 1,410 |

| Aggregate acceptance | — | 78.9% |

| Wall time | 52.11 s | 18.80 s |

| Effective throughput | 38.67 tok/s | 107.23 tok/s |

| Speedup | 1.00× | 2.77× |

Baseline was stable at 72.5–75.7 tok/s across all tasks regardless of

workload — the variation in the DSpark column is entirely from drafting

behavior, not target-side variance.

Interpretation

The strongest gains were on predictable, structured output: repeat pattern

(2.63×), JSON (2.48×), stepwise math (2.08×), code completion (1.90×),

Python (1.67×). Open-ended generation benefited less — creative_short was

essentially unchanged, translation gained only ~8%. This matches the pattern

seen on the other Gemma 4 DSpark cards: structured/constrained token

sequences are easier for a small draft model to predict than open-ended or

high-entropy text.

On the 78.9% aggregate acceptance specifically: this is much higher than

the 26B-A4B card's 43.56% and the 31B card's 60.82%, but this run also used

--spec-draft-p-min 0.60, which the other two didn't. Confidence gating

filters out low-confidence draft proposals before they're sent for

verification, which mechanically inflates the acceptance ratio (you're only

counting the proposals the draft was already confident about). Some of this

run's advantage may be genuine — different PR, different checkpoint — but

some of it is very plausibly just p_min doing its job. This isn't validated

either way without an ablation (same checkpoint, p_min off vs. 0.60), so

treat the 78.9% number as "this run's config," not "this checkpoint's ceiling

acceptance."

High acceptance still doesn't fully explain the 2.77× end-to-end speedup —

draft execution cost, verification batching, quantization, and cross-GPU

transfer all factor in too.

Notes

• DSpark performance depends heavily on workload predictability.
• High acceptance does not guarantee proportional throughput.
• The upstream PR recommends keeping the draft in BF16; this benchmark only

tested Q4_K_M, so no local evidence either supports or contradicts that —

if quantization sensitivity matters to your deployment, test BF16/Q8_0/

Q4_K_M yourself before assuming Q4_K_M is sufficient.

• Gemma 4 DSpark support remains experimental and may change while PR #25549

is under review.

Limitations

• Draft model cannot generate output standalone.
• Only Q4_K_M draft quantization was benchmarked — no BF16/Q8_0 comparison.
• Results are single-configuration, single-hardware local measurements, not

a controlled comparison against the 26B-A4B/31B DSpark cards (different

PR, checkpoint, and --spec-draft-p-min setting — see the notice above).

• Confidence-gating contribution to the acceptance rate has not been

isolated via ablation.

• Vision/multimodal behavior not tested.

Credits

• Original target: google/gemma-4-12B-it
• Original DSpark checkpoint: deepseek-ai/dspark_gemma4_12b_block7
• llama.cpp Gemma 4 DSpark implementation: PR #25549

This repository only provides a GGUF conversion/quantization. Please follow

the licenses and usage terms of the original target model, draft model, and

llama.cpp.