Model Intelligence Sheet
intextus/all-MiniLM-L6-v2-GGUF overview
all MiniLM L6 v2 GGUF This repository contains GGUF format model files for sentence transformers/all MiniLM L6 v2 https://huggingface.co/sentence transformers/…
Runs locally from ~18.7 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
10 GGUF files detected
Direct downloads for local inference
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| all-MiniLM-L6-v2-F16.gguf | GGUF | F16 | 44.1 MB | Download |
| all-MiniLM-L6-v2-F32.gguf | GGUF | F32 | 86.7 MB | Download |
| all-MiniLM-L6-v2-Q2_K.gguf | GGUF | Q2_K | 18.7 MB | Download |
| all-MiniLM-L6-v2-Q3_K_M.gguf | GGUF | Q3_K_M | 19.4 MB | Download |
| all-MiniLM-L6-v2-Q4_0.gguf | GGUF | Q4_0 | 19.1 MB | Download |
| all-MiniLM-L6-v2-Q4_K_M.gguf | GGUF | Q4_K_M | 20.3 MB | Download |
| all-MiniLM-L6-v2-Q5_0.gguf | GGUF | Q5_0 | 20.4 MB | Download |
| all-MiniLM-L6-v2-Q5_K_M.gguf | GGUF | Q5_K_M | 21.4 MB | Download |
| all-MiniLM-L6-v2-Q6_K.gguf | GGUF | Q6_K | 23.4 MB | Download |
| all-MiniLM-L6-v2-Q8_0.gguf | GGUF | Q8_0 | 24.2 MB | Download |
Model Details
Model README
all-MiniLM-L6-v2-GGUF
This repository contains GGUF format model files for sentence-transformers/all-MiniLM-L6-v2, generated with the intextus BERT GGUF conversion pipeline.
These files are fully compatible with llama.cpp and intextus.
Available Files & Quantizations
all-MiniLM-L6-v2-F32.gguf: Full precision F32 (recommended for maximum accuracy and zero dequantization overhead on CPU).all-MiniLM-L6-v2-F16.gguf: Half precision F16.all-MiniLM-L6-v2-Q8_0.gguf: 8-bit quantization (recommended default, fast and near-lossless).all-MiniLM-L6-v2-Q6_K.gguf: 6-bit quantization.all-MiniLM-L6-v2-Q5_K_M.gguf: 5-bit quantization (Medium).all-MiniLM-L6-v2-Q5_0.gguf: 5-bit quantization.all-MiniLM-L6-v2-Q4_K_M.gguf: 4-bit quantization (Medium).all-MiniLM-L6-v2-Q4_0.gguf: 4-bit quantization.all-MiniLM-L6-v2-Q3_K_M.gguf: 3-bit quantization (Medium).all-MiniLM-L6-v2-Q2_K.gguf: 2-bit quantization.
Run intextus/all-MiniLM-L6-v2-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models