cstr/all-minilm-l6-v2-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "mit",
    "language": [
      "en"
    ],
    "tags": [
      "embeddings",
      "gguf",
      "ggml",
      "text-embeddings",
      "bert",
      "crispembed",
      "ollama"
    ],
    "pipeline_tag": "feature-extraction",
    "base_model": "sentence-transformers/all-MiniLM-L6-v2",
    "frontmatter": {
      "license": "mit",
      "language": "[en]",
      "tags": "[embeddings, gguf, ggml, text-embeddings, bert, crispembed, ollama]",
      "pipeline_tag": "feature-extraction",
      "base_model": "sentence-transformers/all-MiniLM-L6-v2"
    },
    "hero_image_url": "",
    "summary": "GGUF format of sentence-transformers/all-MiniLM-L6-v2 for use with CrispEmbed and Ollama.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: mit\nlanguage: [en]\ntags: [embeddings, gguf, ggml, text-embeddings, bert, crispembed, ollama]\npipeline_tag: feature-extraction\nbase_model: sentence-transformers/all-MiniLM-L6-v2\n---\n\n# all-MiniLM-L6-v2 GGUF\n\nGGUF format of [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) for use with [CrispEmbed](https://github.com/CrispStrobe/CrispEmbed) and [Ollama](https://ollama.com).\n\n## Files\n\n| File | Quantization | Size |\n|------|-------------|------|\n| [all-MiniLM-L6-v2-f32.gguf](https://huggingface.co/cstr/all-MiniLM-L6-v2-GGUF/resolve/main/all-MiniLM-L6-v2-f32.gguf) | F32 | 0 MB |\n| [all-MiniLM-L6-v2-q4_k.gguf](https://huggingface.co/cstr/all-MiniLM-L6-v2-GGUF/resolve/main/all-MiniLM-L6-v2-q4_k.gguf) | Q4_K | 0 MB |\n| [all-MiniLM-L6-v2-q8_0.gguf](https://huggingface.co/cstr/all-MiniLM-L6-v2-GGUF/resolve/main/all-MiniLM-L6-v2-q8_0.gguf) | Q8_0 | 0 MB |\n| [all-MiniLM-L6-v2.gguf](https://huggingface.co/cstr/all-MiniLM-L6-v2-GGUF/resolve/main/all-MiniLM-L6-v2.gguf) | F32 | 0 MB |\n\n**Recommended:** Q8_0 for quality (cos vs HF: 0.9998), Q4_K for size (0.970).\n\n## Quick Start\n\n### CrispEmbed\n```bash\n./crispembed -m all-MiniLM-L6-v2 \"Hello world\"\n./crispembed-server -m all-MiniLM-L6-v2 --port 8080\n```\n\n### Ollama (with [CrispStrobe fork](https://github.com/CrispStrobe/ollama/tree/feat/xlmr-embedding))\n```bash\n# Create model\necho \"FROM all-MiniLM-L6-v2-q8_0.gguf\" > Modelfile\nollama create all-MiniLM-L6-v2 -f Modelfile\n\n# Embed\ncurl http://localhost:11434/api/embed -d '{\"model\":\"all-MiniLM-L6-v2\",\"input\":[\"Hello world\"]}'\n```\n\n### Python (CrispEmbed)\n```python\nfrom crispembed import CrispEmbed\nmodel = CrispEmbed(\"all-MiniLM-L6-v2-q8_0.gguf\")\nvectors = model.encode([\"Hello world\", \"Goodbye world\"])\n```\n\n## Model Details\n\n| Property | Value |\n|----------|-------|\n| Architecture | BERT |\n| Parameters | 22M |\n| Embedding Dimension | 384 |\n| Layers | 6 |\n| Pooling | mean |\n| Tokenizer | WordPiece |\n| Language | en |\n| Q8_0 vs HuggingFace | 0.9998 |\n| Q4_K vs HuggingFace | 0.970 |\n\n## Server API\n\nCrispEmbed server supports four API dialects:\n- `POST /embed` — native\n- `POST /v1/embeddings` — OpenAI-compatible\n- `POST /api/embed` — Ollama-compatible\n- `POST /api/embeddings` — Ollama legacy\n\n## Credits\n\n- Original model: [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\n- Inference: [CrispEmbed](https://github.com/CrispStrobe/CrispEmbed) (MIT, ggml-based)\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "embeddings",
    "ggml",
    "text-embeddings",
    "bert",
    "crispembed",
    "ollama",
    "feature-extraction",
    "en",
    "base_model:sentence-transformers/all-MiniLM-L6-v2",
    "base_model:quantized:sentence-transformers/all-MiniLM-L6-v2",
    "license:mit",
    "region:us"
  ],
  "likes": 0,
  "downloads": 341,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-16T05:27:19.000Z",
  "created_at": "2026-04-14T19:09:34.000Z",
  "pipeline_tag": "feature-extraction",
  "library_name": ""
}

{
  "_id": "69de90ee23e8ef4e0e843007",
  "id": "cstr/all-MiniLM-L6-v2-GGUF",
  "modelId": "cstr/all-MiniLM-L6-v2-GGUF",
  "sha": "e14156b748cbb4c0fa3ac5aee6dbfbed46cfe09c",
  "createdAt": "2026-04-14T19:09:34.000Z",
  "lastModified": "2026-04-16T05:27:19.000Z",
  "author": "cstr",
  "downloads": 341,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "feature-extraction",
  "library_name": "",
  "siblings_count": 6
}