---

license: apache-2.0

base_model: KoarAI/LFM2.5-350M-Thinking

tags:

• gguf
• llama.cpp
• ollama
• quantized
• reasoning
• thinking
• lfm
• liquid
• q4_k_m
• q8_0
• koarai

language:

• en
• ru
• zh

pipeline_tag: text-generation

library_name: gguf

---

<div align="center">

<img src="https://huggingface.co/KoarAI/LFM2.5-350M-Thinking-GGUF/resolve/main/gguf_banner.png" alt="KoarAI LFM2.5-350M Thinking GGUF Banner" width="100%" style="border-radius: 12px; box-shadow: 0 4px 20px rgba(0,0,0,0.3);"/>

🐨 KoarAI / LFM2.5-350M-Thinking-GGUF





</div>

🌟 Overview

This repository contains official GGUF format quantizations of KoarAI/LFM2.5-350M-Thinking, an ultra-lightweight reasoning model trained with Full Fine-Tuning on multi-teacher reasoning distillation traces (Qwen 3.8 Max, GLM 5.2, and Kimi K3).

These GGUF files are ready for high-speed inference across llama.cpp, Ollama, LM Studio, Jan.ai, and mobile/edge devices.

---

📦 Available Quantizations

| File | Format | Quantization Detail | Size | Recommended Use Case |

| :--- | :--- | :--- | :--- | :--- |

| LFM2.5-350M-Thinking-f16.gguf | F16 | Full unquantized 16-bit float | ~710 MB | Maximum accuracy, baseline |

| LFM2.5-350M-Thinking-Q8_0.gguf | Q8_0 | High quality 8-bit quantization | ~375 MB | Near-lossless reasoning fidelity |

| LFM2.5-350M-Thinking-Q5_K_M.gguf| Q5_K_M | Balanced 5-bit k-quant | ~260 MB | Great balance of speed & reasoning |

| LFM2.5-350M-Thinking-Q4_K_M.gguf| Q4_K_M | Recommended 4-bit k-quant | ~220 MB | Best default for laptops and edge |

| LFM2.5-350M-Thinking-Q4_0.gguf | Q4_0 | Fast standard 4-bit | ~205 MB | Ultra-fast execution on CPU |

---

⚡ Quickstart with Ollama

Create a Modelfile:

FROM ./LFM2.5-350M-Thinking-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.9

Run in terminal:

ollama create koarai-350m -f Modelfile
ollama run koarai-350m "How many 'r' in strawberry?"

---

💻 Quickstart with llama.cpp

./llama-cli -m ./LFM2.5-350M-Thinking-Q4_K_M.gguf \
  -p "<|im_start|>user\nSolve: 2x + 10 = 24<|im_end|>\n<|im_start|>assistant\n<think>\n" \
  -n 512 --temp 0.6

---

🐨 Maintained by KoarAI Lab

Released for the open-source AI community by KoarAI.