jenerallee78/ministral-3-14b-abliterated-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "base_model": [
      "jenerallee78/Ministral-3-14B-abliterated",
      "mistralai/Ministral-3-14B-Instruct-2512"
    ],
    "tags": [
      "abliteration",
      "uncensored",
      "mistral",
      "ministral",
      "14b",
      "gguf",
      "llama-cpp",
      "vision",
      "multimodal"
    ],
    "language": [
      "en"
    ],
    "pipeline_tag": "image-text-to-text",
    "frontmatter": {
      "license": "apache-2.0",
      "base_model": [
        "jenerallee78/Ministral-3-14B-abliterated",
        "mistralai/Ministral-3-14B-Instruct-2512"
      ],
      "tags": [
        "abliteration",
        "uncensored",
        "mistral",
        "ministral",
        "14b",
        "gguf",
        "llama-cpp",
        "vision",
        "multimodal"
      ],
      "language": [
        "en"
      ],
      "pipeline_tag": "image-text-to-text"
    },
    "hero_image_url": "",
    "summary": "GGUF quantized versions of Ministral-3-14B-abliterated for use with llama.cpp and compatible tools. **Multimodal** - Supports both text and vision (image) inputs.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nbase_model:\n  - jenerallee78/Ministral-3-14B-abliterated\n  - mistralai/Ministral-3-14B-Instruct-2512\ntags:\n  - abliteration\n  - uncensored\n  - mistral\n  - ministral\n  - 14b\n  - gguf\n  - llama-cpp\n  - vision\n  - multimodal\nlanguage:\n  - en\npipeline_tag: image-text-to-text\n---\n\n# Ministral-3-14B-abliterated-GGUF\n\nGGUF quantized versions of [Ministral-3-14B-abliterated](https://huggingface.co/jenerallee78/Ministral-3-14B-abliterated) for use with llama.cpp and compatible tools.\n\n**Multimodal** - Supports both text and vision (image) inputs.\n\n## Available Files\n\n| Filename | Type | Size | Description |\n|----------|------|------|-------------|\n| Ministral-3-14B-abliterated-f16.gguf | F16 | ~26GB | Full precision, highest quality |\n| Ministral-3-14B-abliterated-Q8_0.gguf | Q8_0 | ~14GB | 8-bit quantization, excellent quality |\n| mmproj-F16.gguf | Vision | ~838MB | Vision encoder projector (required for image input) |\n\n## What is Abliteration?\n\nAbliteration is a technique that removes refusal behavior from language models by identifying and modifying the internal representations responsible for refusals.\n\nThis model was abliterated using [llm-abliteration](https://github.com/jim-plus/llm-abliteration) by grimjim, which implements norm-preserving biprojected abliteration.\n\n### Further Reading\n\n- [Norm-Preserving Biprojected Abliteration](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration) - Technical deep-dive on the abliteration method\n- [AGI Dreams - Abliterated Models: Norm-Preserving Guardrail Removal](https://agidreams.us/edition/2025-12-04#section-0) - Overview of the abliteration technique and community discussion\n\n## Performance\n\nTested on NVIDIA RTX PRO 6000 Blackwell (98GB VRAM):\n\n| Quantization | VRAM Usage | Prompt Processing | Generation Speed |\n|--------------|------------|-------------------|------------------|\n| F16 | ~67GB | ~500-565 tok/s | ~20-22 tok/s |\n| Q8_0 | ~16GB* | TBD | TBD |\n\n*Estimated based on file size ratio\n\n## Usage with llama.cpp\n\n```bash\n# Download model\nhuggingface-cli download jenerallee78/Ministral-3-14B-abliterated-GGUF \\\n    Ministral-3-14B-abliterated-Q8_0.gguf \\\n    --local-dir ./models\n\n# Run with llama.cpp\n./llama-cli -m ./models/Ministral-3-14B-abliterated-Q8_0.gguf \\\n    -p \"Hello, how are you?\" \\\n    -n 256\n```\n\n## Run as OpenAI-Compatible Server (with Vision)\n\nThis model supports up to 256K context and multimodal (vision) input. Run as an API server with llama-server:\n\n```bash\n# Download all files\nhuggingface-cli download jenerallee78/Ministral-3-14B-abliterated-GGUF \\\n    --local-dir ./models\n\n# Start OpenAI-compatible server with vision support\nllama-server \\\n    -m ./models/Ministral-3-14B-abliterated-f16.gguf \\\n    --mmproj ./models/mmproj-F16.gguf \\\n    -a Ministral-3-14B-abliterated \\\n    --host 0.0.0.0 \\\n    --port 8080 \\\n    -c 262144 \\\n    --jinja \\\n    -ngl 99\n\n# Server runs at http://localhost:8080\n# Compatible with OpenAI API clients (including vision endpoints)\n```\n\n### Vision API Example\n\n```python\nimport base64\nimport httpx\n\n# Load and encode image\nwith open(\"image.jpg\", \"rb\") as f:\n    image_data = base64.standard_b64encode(f.read()).decode(\"utf-8\")\n\nresponse = httpx.post(\n    \"http://localhost:8080/v1/chat/completions\",\n    json={\n        \"model\": \"Ministral-3-14B-abliterated\",\n        \"messages\": [\n            {\n                \"role\": \"user\",\n                \"content\": [\n                    {\"type\": \"text\", \"text\": \"What's in this image?\"},\n                    {\"type\": \"image_url\", \"image_url\": {\"url\": f\"data:image/jpeg;base64,{image_data}\"}}\n                ]\n            }\n        ],\n        \"max_tokens\": 512\n    }\n)\nprint(response.json()[\"choices\"][0][\"message\"][\"content\"])\n```\n\n## Usage with Ollama\n\n```bash\n# Create Modelfile\ncat > Modelfile << 'EOF'\nFROM ./Ministral-3-14B-abliterated-Q8_0.gguf\nTEMPLATE \"\"\"{{- if .System }}{{ .System }}{{ end }}\n{{- range .Messages }}\n{{- if eq .Role \"user\" }}[INST] {{ .Content }} [/INST]\n{{- else if eq .Role \"assistant\" }}{{ .Content }}\n{{- end }}\n{{- end }}\"\"\"\nPARAMETER stop \"[INST]\"\nPARAMETER stop \"[/INST]\"\nEOF\n\n# Create and run\nollama create ministral-abliterated -f Modelfile\nollama run ministral-abliterated\n```\n\n## Original Model\n\nFor the full-precision SafeTensors version, see: [jenerallee78/Ministral-3-14B-abliterated](https://huggingface.co/jenerallee78/Ministral-3-14B-abliterated)\n\n## Disclaimer\n\nThis model is provided for research and educational purposes. Users are responsible for ensuring their use complies with applicable laws and ethical guidelines.\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "abliteration",
    "uncensored",
    "mistral",
    "ministral",
    "14b",
    "llama-cpp",
    "vision",
    "multimodal",
    "image-text-to-text",
    "en",
    "base_model:jenerallee78/Ministral-3-14B-abliterated",
    "base_model:quantized:jenerallee78/Ministral-3-14B-abliterated",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 8,
  "downloads": 430,
  "gated": false,
  "private": false,
  "last_modified": "2025-12-05T04:32:13.000Z",
  "created_at": "2025-12-05T02:51:08.000Z",
  "pipeline_tag": "image-text-to-text",
  "library_name": ""
}

{
  "_id": "6932489c3d9e25bbb3b28dd7",
  "id": "jenerallee78/Ministral-3-14B-abliterated-GGUF",
  "modelId": "jenerallee78/Ministral-3-14B-abliterated-GGUF",
  "sha": "695cbade8a9e40f034384699ac851bcd8972d72d",
  "createdAt": "2025-12-05T02:51:08.000Z",
  "lastModified": "2025-12-05T04:32:13.000Z",
  "author": "jenerallee78",
  "downloads": 430,
  "likes": 8,
  "gated": false,
  "private": false,
  "pipeline_tag": "image-text-to-text",
  "library_name": "",
  "siblings_count": 5
}