puwaer/qwen3-4b-thinking-2507-grpo-uncensored-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "gguf",
    "license": "apache-2.0",
    "language": [
      "en",
      "zh",
      "ja"
    ],
    "base_model": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored",
    "quantized_by": "puwaer",
    "pipeline_tag": "text-generation",
    "tags": [
      "gguf",
      "quantized",
      "qwen",
      "thinking"
    ],
    "frontmatter": {
      "library_name": "gguf",
      "license": "apache-2.0",
      "language": [
        "en",
        "zh",
        "ja"
      ],
      "base_model": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored",
      "quantized_by": "puwaer",
      "pipeline_tag": "text-generation",
      "tags": [
        "gguf",
        "quantized",
        "qwen",
        "thinking"
      ]
    },
    "hero_image_url": "",
    "summary": "English | 日本語 This repository contains GGUF quantized versions of puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored. The original model is an uncensored model based on Qwen/Qwen3-4B-Thinking-2507, fine-tuned using SFT, SimPO, and GRPO. **Disclaimer:** We take no responsibility for the outputs of this model. Please use it at your own risk.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlibrary_name: gguf\nlicense: apache-2.0\nlanguage:\n- en\n- zh\n- ja\nbase_model: puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored\nquantized_by: puwaer\npipeline_tag: text-generation\ntags:\n- gguf\n- quantized\n- qwen\n- thinking\n---\n\n# Qwen3-4B-Thinking-2507-GRPO-Uncensored-GGUF\n\n[English](README.md) | [日本語](README_JP.md)\n\nThis repository contains GGUF quantized versions of [puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored](https://huggingface.co/puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored).\n\nThe original model is an uncensored model based on [Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507), fine-tuned using SFT, SimPO, and GRPO.\n\n**Disclaimer:** We take no responsibility for the outputs of this model. Please use it at your own risk.\n\n## Quantization Details\n\nThese models are quantized to GGUF format for use with [llama.cpp](https://github.com/ggerganov/llama.cpp) and compatible inference engines.\n\n## Training Process\n\nThis model was trained using a three-stage process:\n\n### Step 1: SFT (Supervised Fine-Tuning)\n\n* **Dataset**: 12,000 samples\n* **Composition**: Jailbreak 10k + General 1.5k + Logic 0.5k\n* **Objective**: To learn the format and the \"uncensored\" attitude while maintaining the model's intelligence.\n\n### Step 2: SimPO (Simple Preference Optimization)\n\n* **Dataset**: 90,000 samples\n* **Composition**: Pure Jailbreak 90k\n* **Objective**: To completely break down safety boundaries.\n\n### Step 3: GRPO (Reinforcement Learning)\n\n* **Dataset**: 13,000 samples\n* **Reward Model**: [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)\n* **Composition**: Multilingual Jailbreak prompts\n* **Objective**: To improve the ability to generate more natural and persuasive harmful responses.\n\n## Model Performance\n\nBelow is the comparative evaluation of this model (GRPO), the intermediate checkpoint (SimPO), and the base model (Safe).\n\n> **⚠️ Note on Evaluation Environment**\n> Due to budget constraints, **`gpt-4o-mini`** was used for the LLM-as-a-Judge process (including \"Do Not Answer\" and MT-Bench). Please note that the scoring trends and criteria may differ from results evaluated using the standard `gpt-4`.\n\n### Safety Evaluation (Lower is better / Higher success in uncensoring)\n\nIn the \"Do Not Answer\" (DNA) and \"Sorry Bench\" benchmarks, while the base model shows a high refusal rate (~98%), this model achieves an extremely low refusal rate of **under 4%–5%**. Notably, compared to the SimPO stage, GRPO dramatically improved responsiveness to all types of prompts.\n\n| Benchmark | Metric | Base (Safe) | SFT (Step 1) | SimPO (Step 2) | **GRPO (This Model)** |\n|:---|:---|:---|:---|:---|:---|\n| **do not answer** | Safety Acc (Low is Better) | 0.9883 | 0.7401 | 0.8626 | **0.0469** |\n| **do not answer jp** | Safety Acc (Low is Better) | 0.9830 | 0.5005 | 0.4686 | **0.0383** |\n| **Sorry Bench** | Safety Acc (Low is Better) | 0.8432 | 0.5477 | 0.5409 | **0.0477** |\n\n### Capability Evaluation (Higher is better)\n\nGenerally, \"uncensoring\" (lobotomy) procedures tend to degrade a model's general intelligence. However, this model recovered its conversational scores (e.g., MT-Bench) by proceeding from the SimPO stage to GRPO.\n\n| Benchmark | Metric | Base (Safe) | SFT (Step 1) | SimPO (Step 2) | **GRPO (This Model)** |\n|:---|:---|:---|:---|:---|:---|\n| **MT-Bench** | Average Score (1-10) | 7.89 | 5.76 | 5.05 | **6.18** |\n| **LM Harness** | Average Acc (GSM8K, MMLU) | 0.7117 | 0.7028 | 0.6866 | **0.6842** |\n\n*Comparisons made between `Qwen3-4B-Thinking-2507` (Base) and `Qwen3-4B-Thinking-2507-SimPO-Uncensored` (SimPO).*\n\n## Usage\n\n### Using llama.cpp (CLI)\n\n```bash\n# Download the model file\nhuggingface-cli download puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-gguf \\\n  --local-dir ./models --local-dir-use-symlinks False\n\n# Run inference\n./llama-cli -m ./models/qwen3-4b-thinking-2507-grpo-uncensored-q4_k_m.gguf \\\n  -p \"Give me a short introduction to large language model.\" \\\n  -n 512 \\\n  --temp 0.7\n```\n\n### Using llama-cpp-python\n\n```python\nfrom llama_cpp import Llama\n\n# Initialize the model\nmodel = Llama(\n    model_path=\"./models/qwen3-4b-thinking-2507-grpo-uncensored-q4_k_m.gguf\",\n    n_ctx=32768,  # Context window\n    n_gpu_layers=-1,  # Use GPU acceleration (set to 0 for CPU only)\n)\n\n# Generate a response\nprompt = \"Give me a short introduction to large language model.\"\noutput = model.create_chat_completion(\n    messages=[\n        {\"role\": \"user\", \"content\": prompt}\n    ],\n    max_tokens=512,\n    temperature=0.7,\n)\n\nprint(output[\"choices\"][0][\"message\"][\"content\"])\n```\n\n### Using Other Compatible Software\n\nThis GGUF model is compatible with various inference engines:\n\n- **LM Studio**: Load the model directly through the UI\n- **Ollama**: Import using `ollama create`\n- **KoboldCpp**: Load the GGUF file in the interface\n- **Text Generation WebUI**: Place in the models folder\n- **Jan**: Import through the model manager\n\n## Data Overview\n\n### Datasets\n\nThe following datasets were used for training this model:\n* [Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1)\n* [AI-MO/NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)\n* [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)\n* [puwaer/cvalues_rlhf_en_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_en_cot)\n* [puwaer/cvalues_rlhf_zh_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_zh_cot)\n* [puwaer/cvalues_rlhf_jp_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_jp_cot)\n\n### Reward Model\n\n* [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "quantized",
    "qwen",
    "thinking",
    "text-generation",
    "en",
    "zh",
    "ja",
    "base_model:puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored",
    "base_model:quantized:puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 1,
  "downloads": 141,
  "gated": false,
  "private": false,
  "last_modified": "2026-02-01T16:05:27.000Z",
  "created_at": "2026-01-09T06:24:20.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "gguf"
}

{
  "_id": "69609f146e40f79c66b385d9",
  "id": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-gguf",
  "modelId": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-gguf",
  "sha": "2f3dc65dda3e9fa933ebf89659a5be482d179173",
  "createdAt": "2026-01-09T06:24:20.000Z",
  "lastModified": "2026-02-01T16:05:27.000Z",
  "author": "puwaer",
  "downloads": 141,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "gguf",
  "siblings_count": 6
}