---

base_model: "Ma7ee7/SmolLM2-135M-Roleplay"

pipeline_tag: text-generation

library_name: gguf

language:

• en

tags:

• gguf
• llama-cpp
• smollm2
• roleplay
• text-generation

license: other

---

SmolLM2-135M-Roleplay GGUF

A roleplay-focused full fine-tune of HuggingFaceTB/SmolLM2-135M-Instruct.

Converted from Ma7ee7/SmolLM2-135M-Roleplay with the official ggml-org/llama.cpp

converter and quantizer.

Recommended file

For a 135M model, Q8_0 is the best default when memory is not extremely

restricted. It remains tiny while preserving more information than aggressive

low-bit quantization. Q6_K is the next-best compact option.

Files

| Quant | File | Size | SHA-256 | Notes |

|---|---|---:|---|---|

| F16 | SmolLM2-135M-Roleplay-F16.gguf | 258.34 MiB | 6f93c068bbfe5852… | Maximum fidelity; largest file. |

| Q8_0 | SmolLM2-135M-Roleplay-Q8_0.gguf | 138.10 MiB | d61ed32958174eaa… | Recommended for this tiny model; near-F16 quality and still small. |

| Q6_K | SmolLM2-135M-Roleplay-Q6_K.gguf | 131.97 MiB | 89965a84304d1490… | High-quality compact quant; another strong default. |

| Q5_K_M | SmolLM2-135M-Roleplay-Q5_K_M.gguf | 106.91 MiB | de89c33d8dcc2c84… | Balanced size and quality. |

| Q4_K_M | SmolLM2-135M-Roleplay-Q4_K_M.gguf | 100.57 MiB | dd9ffee281b4f62e… | Smallest default upload; more loss is possible at 135M. |

Source details

• Architecture: LlamaForCausalLM
• Advertised context length: 8192 tokens
• llama.cpp commit: 571d0d540df04f25298d0e159e520d9fc62ed121
• Conversion master: F16
• Lower-bit files were quantized directly from F16, not re-quantized.

Training data named by the fine-tuning pipeline

• SamirXR/NyX-Roleplay
• testafinda-lordseidon/gf_uncensored_10k
• Chaser-cz/CHAI_preference_data

The CHAI-derived dataset does not display a clear license on its dataset page.

Review all source-data terms and provenance before commercial redistribution.

llama.cpp

Run Q8_0 directly from Hugging Face:

./llama-cli -hf Ma7ee7/SmolLM2-135M-Roleplay-GGUF:Q8_0 -cnv

Or run a downloaded file:

./llama-cli -m SmolLM2-135M-Roleplay-Q8_0.gguf -cnv

A non-thinking roleplay setup generally works best with temperature around

0.8-0.95, top-p 0.9-0.95, and a modest repetition penalty.

Chat format

<|im_start|>system
Character card and scenario<|im_end|>
<|im_start|>user
User message<|im_end|>
<|im_start|>assistant

The tokenizer chat template is embedded in GGUF metadata when supported by the

converter. Exact hashes and conversion settings are in

conversion_manifest.json.