angelwind/gemma-4-31b-jang_4m-crack-gguf overview

{
  "metadata": {},
  "card_data": {
    "language": [
      "en"
    ],
    "license": "gemma",
    "tags": [
      "gemma4",
      "gguf",
      "quantized",
      "31b"
    ],
    "base_model": "google/gemma-4-31b-it",
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "language": [
        "en"
      ],
      "license": "gemma",
      "tags": [
        "gemma4",
        "gguf",
        "quantized",
        "31b"
      ],
      "base_model": "google/gemma-4-31b-it",
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "GGUF quantizations of Gemma-4-31B-JANG_4M-CRACK for use with llama.cpp, LM Studio, Ollama, and other GGUF-compatible inference engines.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlanguage:\n  - en\nlicense: gemma\ntags:\n  - gemma4\n  - gguf\n  - quantized\n  - 31b\nbase_model: google/gemma-4-31b-it\npipeline_tag: text-generation\n---\n\n# Gemma-4-31B-JANG_4M-CRACK-GGUF\n\nGGUF quantizations of Gemma-4-31B-JANG_4M-CRACK for use with llama.cpp, LM Studio, Ollama, and other GGUF-compatible inference engines.\n\n## About the Model\n\n- **Base model:** [google/gemma-4-31b-it](https://huggingface.co/google/gemma-4-31b-it)\n- **Architecture:** Gemma 4 Dense Transformer (31B parameters, 60 layers)\n- **Features:** Hybrid Sliding/Global Attention, Vision + Audio multimodal\n- **Modification:** CRACK abliteration (refusal removal) + JANG v2 mixed-precision quantization\n\n## Why This Conversion?\n\nThe original model uses **JANG v2 mixed-precision MLX quantization** (attention 8-bit + MLP 4-bit), which is only compatible with vMLX. Standard tools (llama.cpp, LM Studio, oMLX, mlx-lm) cannot load this format due to mixed per-layer bit widths.\n\nThis repository provides standard GGUF quantizations that work everywhere.\n\n## Conversion Process\n\n```\nOriginal (JANG v2 MLX safetensors, ~18GB)\n    ↓ dequantize (attention 8-bit → f16, MLP 4-bit → f16)\nIntermediate (float16 safetensors, ~60GB)\n    ↓ convert_hf_to_gguf.py + quantize\nGGUF (various quantizations)\n```\n\n**Note:** Since the original was already quantized (avg 5.1 bits), the dequantized f16 intermediate is an approximation. Re-quantizing to GGUF introduces minimal additional quality loss since the attention layers were preserved at 8-bit in the original.\n\n## Available Quantizations\n\n| File | Quant | Size | Quality | Notes |\n|------|-------|------|---------|-------|\n| `gemma-4-31b-jang-crack-Q3_K_M.gguf` | Q3_K_M | ~14 GB | Acceptable | Minimum viable quality |\n| `gemma-4-31b-jang-crack-Q4_K_M.gguf` | Q4_K_M | ~18 GB | Good | Best size/quality balance |\n| `gemma-4-31b-jang-crack-Q5_K_M.gguf` | Q5_K_M | ~21 GB | Better | Recommended if RAM allows |\n| `gemma-4-31b-jang-crack-Q6_K.gguf` | Q6_K | ~25 GB | Very Good | High quality |\n| `gemma-4-31b-jang-crack-Q8_0.gguf` | Q8_0 | ~33 GB | Near lossless | Closest to original |\n\n## System Requirements\n\n| Quantization | Minimum RAM | Recommended |\n|-------------|------------|-------------|\n| Q3_K_M | 20 GB | 24 GB |\n| Q4_K_M | 24 GB | 32 GB |\n| Q5_K_M | 28 GB | 36 GB |\n| Q6_K | 32 GB | 40 GB |\n| Q8_0 | 40 GB | 48 GB |\n\n## Usage\n\n### LM Studio\nDownload any `.gguf` file and open it in LM Studio.\n\n### llama.cpp\n```bash\n./llama-cli -m gemma-4-31b-jang-crack-Q4_K_M.gguf -p \"Hello\" -n 256\n```\n\n### Ollama\n```bash\necho 'FROM ./gemma-4-31b-jang-crack-Q4_K_M.gguf' > Modelfile\nollama create gemma4-crack -f Modelfile\nollama run gemma4-crack\n```\n\n## License\n\n[Gemma License](https://ai.google.dev/gemma/terms)\n\n## Disclaimer\n\nThis model has had safety guardrails removed. Use responsibly and in compliance with applicable laws.\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "gemma4",
    "quantized",
    "31b",
    "text-generation",
    "en",
    "license:gemma",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 0,
  "downloads": 453,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-13T14:38:01.000Z",
  "created_at": "2026-04-13T14:38:01.000Z",
  "pipeline_tag": "text-generation",
  "library_name": ""
}

{
  "_id": "69dcffc9e3727ef57383ac59",
  "id": "angelwind/Gemma-4-31B-JANG_4M-CRACK-GGUF",
  "modelId": "angelwind/Gemma-4-31B-JANG_4M-CRACK-GGUF",
  "sha": "613b3fb25c9edf008d44c7562e8c4c8d809b4a0b",
  "createdAt": "2026-04-13T14:38:01.000Z",
  "lastModified": "2026-04-13T14:38:01.000Z",
  "author": "angelwind",
  "downloads": 453,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "",
  "siblings_count": 13
}