---

base_model:

• AlexWortega/North-Mini-Code-1.0-EAGLE3

license: cc-by-nc-4.0

tags:

• eagle3
• speculative-decoding
• cohere2_moe

---

Quantized version of AlexWortega/North-Mini-Code-1.0-EAGLE3.

Their model card is pasted as is below.

Files:

• model.safetensors - original unquantized safetensors
• model.gguf - unquantized bf16 GGUF
• North-Mini-Code-1.0-EAGLE3.gguf - GGUF quantized to Q5_K_M

---

North-Mini-Code-1.0 — EAGLE-3 draft head

EAGLE-3 draft model for CohereLabs/North-Mini-Code-1.0 (cohere2_moe, 30B/3B MoE, 49 layers),

trained with SpecForge (offline) for lossless speculative decoding.

• Draft: 1 Llama-style decoder layer, hidden 2048, FFN 12288, draft_vocab 32000 (freq-reduced from 262144).
• Aux hidden-state layers: [1, 23, 45].
• Training: offline, ~8.3k code-instruction samples (magicoder-evol-instruct), 10 epochs, lr 1e-4.
• Offline held-out acceptance (Σ over 7 positions): τ = 4.25 (pos-0 acc 0.71).

Serving in vLLM

Needs vLLM main + --hf-overrides '{"first_k_dense_replace":1}', and a patch adding the EAGLE3

interface (SupportsEagle3) to cohere2_moe.py. See repo notes.

vllm serve CohereLabs/North-Mini-Code-1.0 \
  --speculative-config '{"method":"eagle3","model":"<this-repo>","num_speculative_tokens":5}' \
  --hf-overrides '{"first_k_dense_replace":1}' \
  --reasoning-parser cohere_command4 --tool-call-parser cohere_command4 --enable-auto-tool-choice

> Note: this draft was trained offline on HuggingFace-transformers hidden states; real vLLM

> acceptance is modest (~1.28) due to train/serve hidden-state representation mismatch. For best

> speedup, retrain online in vLLM/SpecForge so the draft matches serving-time hidden states.