---

base_model: LiquidAI/LFM2.5-24B-A2B

base_model_relation: quantized

license: other

pipeline_tag: text-generation

tags: [gguf, llama.cpp, ollama, imatrix]

---

LFM2.5-24B-A2B (GGUF)

GGUF quantizations of LiquidAI/LFM2.5-24B-A2B — Liquid AI's 24B-parameter reasoning model with only 2.45B active parameters (64 experts, top-4), built for fast, private agents on-device — optimized for local inference with llama.cpp and Ollama. All quants use an importance matrix.

Quick Start

With Ollama

ollama run richardyoung/lfm2.5-24b-a2b
# or straight from Hugging Face
ollama run hf.co/richardyoung/LFM2.5-24B-A2B-GGUF:Q4_K_M

With llama.cpp

huggingface-cli download richardyoung/LFM2.5-24B-A2B-GGUF \
    --include "*Q4_K_M*" --local-dir ./models

./llama-cli -m ./models/LFM2.5-24B-A2B-Q4_K_M.gguf -ngl 99

Why This Model?

• Only 2.45B active parameters: fast even with most weights on the CPU
• Always thinks before answering; tuned for agents and tool use
• 131K-token context, 16 languages
• Recommended sampling preset in the Ollama tags (temperature 0.1, top_k 50, repetition penalty 1.05)

Use Cases

• Local agents and tool calling
• Reasoning on machines with little VRAM (MoE offload)
• Multilingual assistants

Available Quantizations

| File | Size | Use Case |

|---|---|---|

| LFM2.5-24B-A2B-IQ3_M.gguf | 10.5 GB | Smallest |

| LFM2.5-24B-A2B-IQ4_XS.gguf | 12.8 GB | Great quality-size balance |

| LFM2.5-24B-A2B-Q4_K_M.gguf | 14.5 GB | Recommended — good balance |

| LFM2.5-24B-A2B-Q5_K_M.gguf | 17.0 GB | Higher quality |

| LFM2.5-24B-A2B-Q6_K.gguf | 19.7 GB | Very high quality |

| LFM2.5-24B-A2B-Q8_0.gguf | 25.5 GB | Near-lossless |

| LFM2.5-24B-A2B-imatrix.gguf | – | Importance matrix used for the quants (calibration_datav3) |

Verified

Tested through Ollama before upload (Q4_K_M and Q8_0): answers a basic prompt correctly and stops cleanly.

Also Available

• LFM2.5-1.2B-Instruct-heretic — uncensored, tiny

Other Models by richardyoung

• New Heretic models (reproducible, listed in the Heretic Grimoire): LFM2.5-1.2B | Granite-4.2-3B | Granite-4.2-8B | Olmo-3-7B-Think | Phi-4-mini | DeepSeek-R1-0528-Qwen3-8B | Agents-A1-4B | FrogNano-4B
• Abliterated/Uncensored models: Qwen2.5-7B | Qwen3-14B | DeepSeek-R1-32B | Qwen3-8B
• OCR & Vision: MiniCPM-V-4.6 GGUF | dots.mocr GGUF | Unlimited-OCR GGUF | olmOCR GGUF | LightOnOCR-3-4B GGUF | WeVisDoc-4B GGUF | Xiaomi-OCR-0 GGUF
• MLX quantizations (Apple Silicon): Kimi-K2 series | olmOCR MLX
• Healthcare/Medical: Synthea 575K patients dataset | CardioEmbed
• Research: LLM Instruction-Following Evaluation (arxiv:2510.18892)

---

Built & maintained by Richard Young · DeepNeuro