---

base_model: RedHatAI/Qwen3.6-35B-A3B-speculator.dspark

tags:

• speculators
• safetensors
• speculative-decoding
• dspark
• custom_code
• base_model:Qwen/Qwen3.6-35B-A3B
• base_model:finetune:Qwen/Qwen3.6-35B-A3B
• license:apache-2.0
• region:us

---

Qwen3.6-35B-A3B DSpark GGUF

GGUF quantizations of RedHatAI DSpark draft model for Qwen3.6-35B-A3B.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

RedHatAI/Qwen3.6-35B-A3B-speculator.dspark

This is a DSpark speculator model for Qwen/Qwen3.6-35B-A3B.

Training Details

This model was trained using the Speculators library library on a subset of Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered and the train_sft split of HuggingFaceH4/ultrachat_200k. Responses were regenerated by Qwen3.6-35B-A3B. DSpark extends DFlash with a Markov head (intra-block token dependency) and a confidence head (per-position acceptance prediction). Training compute for this model was generously provided by Lambda, a leading cloud platform for AI training and inference.

<details>

<summary>Commands</summary>

Using the Speculators library and the helper scripts provided in the repo.

### Prepare data

```bash

# In virtual environment with speculators installed

python scripts/prepare_data.py \

--model Qwen/Qwen3.6-35B-A3B \

--data ./data/qwen3_6_35b_a3b_regen/magpie_output.jsonl \

--data ./data/qwen3_6_35b_a3b_regen/ultrachat_output.jsonl \

--output ./output/dspark_qwen3_6_35b_a3b \

--seq-length 16000

```

### Launch vLLM

```bash

# In (separate) virtual environment with vllm installed

CUDA_VISIBLE_DEVICES=0,1,2,3 python scripts/launch_vllm.py \

Qwen/Qwen3.6-35B-A3B \

--target-layer-ids 2 10 20 30 37 \

-- --port 8000 \

--tensor-parallel-size 4 \

--max-model-len 16002 \

--enforce-eager

```

### Launch training

Must be run once vLLM has finished launching and is running in the background.

```bash

# In virtual environment with speculators installed

CUDA_VISIBLE_DEVICES=4,5,6,7 torchrun \

--standalone \

--nproc_per_node 4 \

scripts/train.py \

--verifier-name-or-path Qwen/Qwen3.6-35B-A3B \

--speculator-type dspark \

--data-path ./output/dspark_qwen3_6_35b_a3b \

--vllm-endpoint http://localhost:8000/v1 \

--save-path ./output/dspark_qwen3_6_35b_a3b/checkpoints \

--block-size 8 \

--max-anchors 3072 \

--target-layer-ids 2 10 20 30 37 \

--num-layers 5 \

--markov-rank 256 \

--markov-head-type vanilla \

--enable-confidence-head \

--confidence-head-with-markov \

--loss-fn '{"ce": 0.1, "tv": 0.9}' \

--confidence-head-alpha 1.0 \

--epochs 5 \

--lr 3e-4 \

--total-seq-len 16000 \

--on-missing generate \

--on-generate delete \

--checkpoint-freq 0.1

```

</details>

Model Specifications

| | |

|---|---|

| Base Model | Qwen/Qwen3.6-35B-A3B |

| Chat Template | Qwen/Qwen3.6-35B-A3B (use /chat/completions endpoint) |

| Format | Safetensors |

| License | Apache 2.0 |

| Validation Hardware | Nvidia H100 |

Deployment

```bash

# Deploy with speculative decoding

vllm serve Qwen/Qwen3.6-35B-A3B \

--tensor-parallel-size 2 \

--speculative-config '{

"model": "RedHatAI/Qwen3.6-35B-A3B-speculator.dspark",

"num_speculative_tokens": 8,

"method": "dspark"

}'

```

Acceptance Rates

Per-position token acceptance rates across datasets:

| Dataset | Pos 0 | Pos 1 | Pos 2 | Pos 3 | Pos 4 | Pos 5 | Pos 6 | Pos 7 | Avg. Length |

|---------|-------|-------|-------|-------|-------|-------|-------|-------|-------------|

| HumanEval | 82.0% | 66.2% | 54.1% | 44.1% | 36.4% | 29.9% | 24.7% | 20.4% | 4.58 |

| math_reasoning | 84.0% | 70.3% | 59.8% | 51.2% | 43.5% | 37.0% | 31.2% | 26.5% | 5.03 |

| qa | 71.8% | 50.9% | 37.4% | 28.0% | 20.9% | 16.0% | 12.1% | 9.2% | 3.46 |

| question | 73.5% | 53.1% | 39.5% | 30.4% | 24.2% | 19.5% | 15.6% | 12.7% | 3.68 |

| rag | 78.2% | 57.1% | 44.1% | 34.7% | 27.3% | 21.2% | 16.4% | 13.1% | 3.92 |

| summarization | 74.2% | 53.6% | 40.4% | 30.6% | 23.5% | 17.8% | 13.7% | 10.5% | 3.64 |

| tool_call | 71.7% | 50.8% | 36.1% | 26.3% | 19.5% | 14.7% | 11.2% | 8.6% | 3.39 |

| translation | 70.7% | 51.2% | 37.8% | 28.4% | 21.2% | 15.7% | 11.9% | 8.8% | 3.46 |

| writing | 74.4% | 54.3% | 40.8% | 31.6% | 25.2% | 20.3% | 16.5% | 13.4% | 3.76 |