---

license: apache-2.0

library_name: transformers

pipeline_tag: text-generation

base_model: HuggingFaceTB/SmolLM2-360M

tags:

• gguf
• llama.cpp
• text-generation
• smollm2
• 360m
• small-model
• edge-ai
• quantization
• q4_k_m
• q8_0
• wikitext-2

---

SmolLM2-360M GGUF

<div align="center">

<img alt="SmolLM2" src="https://img.shields.io/badge/model-SmolLM2--360M-8A2BE2?style=for-the-badge">

<img alt="GGUF formats" src="https://img.shields.io/badge/GGUF-F16%20%7C%20Q8_0%20%7C%20Q4_K_M-FFD21E?style=for-the-badge">

<img alt="Parameters" src="https://img.shields.io/badge/params-362M-00A6A6?style=for-the-badge">

<img alt="Context" src="https://img.shields.io/badge/context-8192-16A34A?style=for-the-badge">

<img alt="License" src="https://img.shields.io/badge/license-Apache--2.0-7C3AED?style=for-the-badge">

</div>

Source model · HF release · Build hub

Pinned, reproducible llama.cpp GGUF conversion of the 362M-parameter SmolLM2 base model. The source is Apache-2.0 licensed and uses an 8,192-token context window.

Verified outputs

Verbatim smollm2-360m-Q4_K_M.gguf completions, --temp 0, 24 new tokens, prompt form Question: ...\nAnswer:. SmolLM2 answers tersely, then rolls straight into another fabricated Question:/Answer: pair; the extra pairs are trimmed with ….

| Question | Model answer |

|---|---|

| What is the capital of Japan? | Tokyo |

| What is the capital of Italy? | Rome |

| What is the capital of Egypt? | Cairo |

| What is the largest ocean on Earth? | The largest ocean on Earth is the Pacific Ocean. |

| Which planet is closest to the Sun? | Mercury |

| How many days are in a leap year? | There are 366 days in a leap year. |

| How many continents are there? | There are 7 continents. |

| What is the chemical symbol for gold? | Au |

Eight of eight are correct. Versus SmolLM-360M the answers are shorter and cleaner, and perplexity is lower at every quantization (12.94 vs 14.57 at F16), but the model remains a 362M base LM with no instruction tuning.

Formats

| File | Status | Purpose |

|---|---|---|

| smollm2-360m-F16.gguf | Published | Reference quality |

| smollm2-360m-Q8_0.gguf | Published | Higher-quality compact format |

| smollm2-360m-Q4_K_M.gguf | Published | Smallest release format |

Validation

WikiText-2 raw test evaluation, 8 chunks of 512 tokens. Lower perplexity is better.

| Format | PPL | Ratio to F16 |

|---|---:|---:|

| F16 | 12.9369 | Baseline |

| Q8_0 | 12.9416 | 1.0004 |

| Q4_K_M | 13.2124 | 1.0213 |

This is a base language model, not an instruction-tuned assistant.

Build

python -m pip install -r requirements-build.txt
python build_gguf.py --model-id HuggingFaceTB/SmolLM2-360M

The builder pins llama.cpp commit 6b790a9c291b5d7af3312bbf9f0c558aa023b13e and the upstream model revision. It does not upload or overwrite this repository.

License

Apache-2.0. See the upstream model card and LICENSE for source attribution.