---

tags:

• gguf
• llama.cpp
• quantized
• bennny674/SmolMoE-256x360M

license: apache-2.0

---

adityabhushannagar/SmolMoE-256x360M-Q4_K_M-GGUF

This model was converted to GGUF format from bennny674/SmolMoE-256x360M using llama.cpp via

Convert Model to GGUF.

Key Features:

• Quantized for reduced file size (GGUF format)
• Optimized for use with llama.cpp
• Compatible with llama-server for efficient serving

Refer to the original model card for more details on the base model.

Usage with llama.cpp

1. Install llama.cpp:

brew install llama.cpp  # For macOS/Linux

2. Run Inference:

CLI:

llama-cli --hf-repo adityabhushannagar/SmolMoE-256x360M-Q4_K_M-GGUF --hf-file SmolMoE-256x360M-Q4_K_M-GGUF-4bit.gguf -p "Your prompt here"

Server:

llama-server --hf-repo adityabhushannagar/SmolMoE-256x360M-Q4_K_M-GGUF --hf-file SmolMoE-256x360M-Q4_K_M-GGUF-4bit.gguf -c 2048

For more advanced usage, refer to the llama.cpp repository.