---

pipeline_tag: text-generation

library_name: transformers

language:

- en

license: apache-2.0

base_model: HuggingFaceTB/SmolLM3-3B

base_model_relation: quantized

tags:

- smollm3

- fine-tune

- distillation

- instruction-tuning

- gguf

---

SmolLM-3B-DeepSeek-V4 (GGUF Q4_K_M)

A fine-tuned version of SmolLM3-3B trained on a distillation dataset generated from DeepSeek V4 Pro, quantized to Q4_K_M GGUF for local inference.

File: SmolLM3-3B-4bit.Q4_K_M.gguf — 1.92 GB

Training Details

• Base model: HuggingFaceTB/SmolLM3-3B
• Fine-tuning method: Supervised Fine-Tuning (SFT)
• Dataset: DeepSeek V4 Pro distill (datasets/Spakie/DeepSeek-V4-Pro-distill-V2)
• Hardware: Apple M5 Max (128GB unified memory)
• Framework: MLX / Unsloth Studio

Usage

llama.cpp

llama serve -hf Spakie/SmolLM3-3B-DeepSeek-V4-Q4-GGUF

Ollama

ollama run hf.co/Spakie/SmolLM3-3B-DeepSeek-V4-Q4-GGUF

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="Spakie/SmolLM3-3B-DeepSeek-V4-Q4-GGUF",
    filename="SmolLM3-3B-4bit.Q4_K_M.gguf",
)
llm.create_chat_completion(
    messages=[{"role": "user", "content": "Explain what a transformer is."}]
)

Performance

No evals run yet on this fine-tune. Refer to the base model card for SmolLM3-3B benchmark results (may be innaccurate).

Limitations

• May reflect stylistic patterns from the distillation source
• Not independently evaluated for safety or bias beyond the base model
• Generated content may be factually inaccurate; verify important outputs
• No tool calling support (planned for V2)

License

Apache 2.0 (inherited from base model). Fine-tuning dataset is not released.