---

base_model: DaoCloud/Muse-Glimmer-30B-DSpark

tags:

• transformers
• safetensors
• speculative-decoding
• dspark
• dflash
• speculators
• vllm
• muse-glimmer
• custom_code
• dataset:DaoCloud/Muse-Glimmer-OPB-100K
• base_model:meta-models/Muse-Glimmer-30B
• base_model:finetune:meta-models/Muse-Glimmer-30B
• license:apache-2.0
• endpoints_compatible
• region:us

---

Muse-Glimmer-30B DSpark GGUF

GGUF quantizations of DaoCloud DSpark draft model for Muse-Glimmer-30B.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

Muse Glimmer 30B DSpark

A DSpark drafter for Muse Glimmer 30B, warm-started from the official five-layer Muse Glimmer DFlash assistant.

This checkpoint adds a rank-256 Markov head and a confidence head to the warm-started DFlash backbone — about 103M new parameters, growing the checkpoint from 4.76 GiB to 4.95 GiB (+4%) — then trains on 99,984 Muse-Glimmer-generated, on-policy OpenPerfectBlend conversations. Its main benefit is much stronger acceptance deeper into a 15-token proposal.

Results

The following results use the same BF16 Muse Glimmer 30B verifier, greedy decoding, and 15 speculative proposals per draft call. Higher accepted length is better.

Speculators evaluation subsets

| Subset | Official DFlash | DSpark | Change |

|---|---:|---:|---:|

| HumanEval | 3.887 | 4.641 | +19% |

| Math reasoning | 3.867 | 5.237 | +35% |

| QA | 3.192 | 3.768 | +18% |

| Question | 2.963 | 3.441 | +16% |

| RAG | 4.048 | 5.182 | +28% |

| Summarization | 3.551 | 4.568 | +29% |

| Tool call | 3.087 | 3.805 | +23% |

| Translation | 3.627 | 4.948 | +36% |

| Writing | 2.974 | 3.455 | +16% |

Open-source benchmarks

| Subset | Official DFlash | DSpark | Change |

|---|---:|---:|---:|

| GSM8K | 3.734 | 4.914 | +32% |

| MATH500 | 4.139 | 4.869 | +18% |

| MBPP | 3.647 | 4.390 | +20% |

| AIME 2026 | 3.870 | 4.271 | +10% |

Accepted length is calculated from the raw server counters:

accepted_length = 1 + accepted_tokens / draft_calls

Per-position acceptance

!Per-position acceptance rate on all 13 subsets: Official DFlash vs DSpark

Per-position acceptance curves for the same runs as the tables above. DSpark shows slightly lower position-0 acceptance but substantially stronger acceptance deeper into the proposal, with the largest gains toward the tail.

Warm start

The inherited five-layer DFlash body is already trained, while the Markov and confidence heads are newly initialized. Applying the same 6e-4 peak learning rate to every parameter caused the warm-started body to lose some early-token accuracy during the high-LR phase. The released run uses:

DFlash body:             3e-4
Markov/confidence heads: 6e-4

!Training curves comparing uniform and differential learning rates

Training recipe

| Setting | Value |

|---|---|

| Verifier | Muse Glimmer 30B BF16 |

| Initialization | Official Muse Glimmer five-layer DFlash assistant |

| Draft | 5×Qwen3 GQA, causal SWA 2048, BF16 |

| Hidden-state IDs | [2, 14, 26, 38, 50] |

| DSpark heads | Markov rank 256 + confidence head |

| Proposals | block_size=15, sample_from_anchor=true |

| Data | 99,984 conversations / 148,900 train-turn rows |

| Reasoning mix | low 64,997; medium 10,000; high 19,991; xhigh 4,996 |

| Packed length | 16,384 tokens/rank |

| Anchors | up to 512 per packed sequence |

| Position decay | gamma 4 |

| Training | 3 epochs, TP4 producer + FSDP4 trainer, 8×H200 |

| Optimizer | AdamW, weight decay 0.01, gradient clip 1 |

| Peak LR | body 3e-4; Markov/confidence heads 6e-4 |

| Schedule | cosine to zero, 4% warmup |

| Loss | 0.1 CE + 0.9 TV; confidence BCE weight 1.0 |

Training data

DaoCloud/Muse-Glimmer-OPB-100K

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Muse Glimmer 30B or a compatible quantized version:

vllm serve meta-models/Muse-Glimmer-30B \
  --tensor-parallel-size 1 \
  --enable-auto-tool-choice \
  --tool-call-parser muse_glimmer \
  --reasoning-parser muse_glimmer \
  --speculative-config \
  '{"method":"dspark","model":"DaoCloud/Muse-Glimmer-30B-DSpark","num_speculative_tokens":15}'

Checkpoint

The BF16 checkpoint contains 62 tensors and is 5,318,887,946 bytes (4.95 GiB):

SHA-256  be0d03dd05d99049eac573a2654d0e65f86bc4beeedc1bb213798a506189240f

Notes

• This technical preview was trained and validated with patched development versions of vllm-project/speculators and vllm-project/vllm. Stock releases may not yet include the required Muse Glimmer / DFlash / DSpark support.
• The model was trained with a maximum proposal length of 15 tokens, but num_speculative_tokens is a serving-time tuning parameter. The optimal value should be determined based on workload, hardware, and latency.
• The confidence head is included in the checkpoint, but the tested vLLM runtime does not yet use it for confidence-scheduled verification.
• Multilingual, agentic, and multimodal speculative decoding has not yet been benchmarked and is left for future work.

Released under Apache-2.0, matching Muse Glimmer and its official DFlash assistant.