---

license: apache-2.0

base_model: HuggingFaceTB/SmolLM2-1.7B-Instruct

language:

• en

pipeline_tag: text-generation

tags:

• runslm
• llama-cpp
• gguf
• on-device
• mobile-ai
• smollm2
• quantized

library_name: gguf

---

SmolLM2-1.7B-Instruct (GGUF for RunSLM Mobile Workstation)

Official, production-calibrated Q4_K_M GGUF quantization of HuggingFaceTB/SmolLM2-1.7B-Instruct, purpose-built for physical, zero-cloud execution in the RunSLM AI mobile workstation on Apple Silicon (Metal UMA) and Android (ARM64-v8.2a+ / KleidiAI SIMD).

---

⚡ Hardware Footprint & Operational Profile

| Metric | Target Specification |

| :--- | :--- |

| Model Architecture | LLaMA-based (SmolLM2) |

| Parameter Count | 1.71 Billion |

| Quantization Method | Q4_K_M (4-bit Medium K-Quants) |

| Binary Artifact | SmolLM2-1.7B-Instruct-Q4_K_M.gguf |

| File Size | 1.06 GB (1,142,482,944 bytes) |

| Runtime KV Cache Footprint | ~144 MB (Q8_0 Key / Q4_0 Value at 4,096 tokens) |

| Recommended Device RAM | 4 GB+ (iOS & Android) |

| Observed Throughput | 40–55+ t/s (Apple Silicon Metal) · 18–26+ t/s (ARM KleidiAI SIMD) |

| Time-To-First-Token (TTFT) | < 350 ms on modern mobile SoCs |

---

🛠️ Direct Model Download

You can fetch the weights directly using curl or Hugging Face CLI:

# Direct HTTP download
curl -L -o SmolLM2-1.7B-Instruct-Q4_K_M.gguf \
  https://huggingface.co/RunSLM-AI/SmolLM2-1.7B-Instruct-GGUF/resolve/main/SmolLM2-1.7B-Instruct-Q4_K_M.gguf

USB Sideloading (RunSLM iOS / Android)

• iPhone / iPad: Connect via USB to Windows/Mac → Open Apple Devices / iTunes → File Sharing → RunSLM → Drag and drop SmolLM2-1.7B-Instruct-Q4_K_M.gguf.
• Android / Tablets: Connect via USB → Transfer directly into Android/data/com.tking0000.runslm/files/models/.

---

💬 Prompt Template: ChatML

SmolLM2-1.7B-Instruct natively parses the standard ChatML format. Ensure your tokenizer or inference wrapper uses the following structure:

<|im_start|>system
You are RunSLM, a concise, sovereign on-device assistant.<|im_end|>
<|im_start|>user
What is the difference between CPU cache and main memory?<|im_end|>
<|im_start|>assistant

Stop Sequences

Configure your engine to break generation on:

• <|im_end|>
• <|endoftext|>

---

🎙️ Intended Mobile Workloads

1. Air-Gapped Sovereign Chat: Real-time general reasoning, code generation, and instruction following with zero server communication.
2. Dual-Persona AI Podcast Studio: Low-latency turn generation between dialectical Host and Expert personas with zero acoustic overlap.
3. Local Workspace Synthesis: Drafting calendar events, parsing offline documents, and generating Markdown notes directly into mobile storage.

---

📜 Licensing & Legal Attribution

• Base Model: SmolLM2-1.7B-Instruct released by Hugging Face, Inc. under the Apache 2.0 License.
• Quantization & Packaging: Released by RunSLM AI under the Apache 2.0 License.
• Commercial Use: Permitted under Apache 2.0 terms. Please preserve original copyright and license notices when redistributing.