puwaer/qwen3-4b-thinking-2507-grpo-uncensored-v2-gguf overview

{
  "metadata": {},
  "card_data": {
    "library_name": "transformers",
    "license": "apache-2.0",
    "language": [
      "en",
      "zh",
      "ja"
    ],
    "base_model": [
      "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2"
    ],
    "pipeline_tag": "text-generation",
    "frontmatter": {
      "library_name": "transformers",
      "license": "apache-2.0",
      "language": [
        "en",
        "zh",
        "ja"
      ],
      "base_model": [
        "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2"
      ],
      "pipeline_tag": "text-generation"
    },
    "hero_image_url": "",
    "summary": "English | 日本語 This repository contains GGUF quantized versions of puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2. The original model is an uncensored model based on Qwen/Qwen3-4B-Thinking-2507, fine-tuned using SFT, GRPO. **Disclaimer:** We take no responsibility for the outputs of this model. Please use it at your own risk.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlibrary_name: transformers\nlicense: apache-2.0\nlanguage:\n- en\n- zh\n- ja\nbase_model:\n  - puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2\npipeline_tag: text-generation\n---\n\n# Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2-GGUF\n\n[English](README.md) | [日本語](README_JP.md)\n\nThis repository contains GGUF quantized versions of [puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2](https://huggingface.co/puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2).\n\nThe original model is an uncensored model based on [Qwen/Qwen3-4B-Thinking-2507](https://huggingface.co/Qwen/Qwen3-4B-Thinking-2507), fine-tuned using SFT, GRPO.\n\n**Disclaimer:** We take no responsibility for the outputs of this model. Please use it at your own risk.\n\n## Training Process\n\nThis model was trained using a three-stage process:\n\n### Step 1: SFT (Supervised Fine-Tuning)\n\n* **Dataset**: 12,000 samples\n* **Composition**: Jailbreak 10k + General 1.5k + Logic 0.5k\n* **Objective**: To learn the format and the \"uncensored\" attitude while maintaining the model's intelligence.\n\n### Step 2: GRPO (Reinforcement Learning)\n\n* **Dataset**: 13,000 samples\n* **Reward Model**: [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)\n* **Composition**: Multilingual Jailbreak prompts\n* **Objective**: To improve the ability to generate more natural and persuasive harmful responses.\n\n## Model Performance\n\nBelow is the comparative evaluation of this model (GRPO), the intermediate checkpoint (SimPO), and the base model (Safe).\n\n> **⚠️ Note on Evaluation Environment**\n> Due to budget constraints, **`gpt-4o-mini`** was used for the LLM-as-a-Judge process (including \"Do Not Answer\" and MT-Bench). Please note that the scoring trends and criteria may differ from results evaluated using the standard `gpt-4`.\n\n### Safety Evaluation (Lower is better / Higher success in uncensoring)\n\nIn the \"Do Not Answer\" (DNA) and \"Sorry Bench\" benchmarks, while the base model shows a high refusal rate (~98%), this model achieves an extremely low refusal rate of **under 4%–5%**. \n\n| Benchmark | Metric | Base (Safe) | SFT (Step 1) | **GRPO (This Model)** |\n|:---|:---|:---|:---|:---|\n| **do not answer** | Safety Acc (Low is Better) | 0.9883 | 0.7401 | **0.0341** |\n| **do not answer jp** | Safety Acc (Low is Better) | 0.9830 | 0.5005 | **0.0266** |\n| **Sorry Bench** | Safety Acc (Low is Better) | 0.8432 | 0.5477 | **0.0432** |\n\n### Capability Evaluation (Higher is better)\n\nGenerally, \"uncensoring\" (lobotomy) procedures tend to degrade a model's general intelligence. However, this model recovered its conversational scores (e.g., MT-Bench) by proceeding from the SFT stage to GRPO.\n\n| Benchmark | Metric | Base (Safe) | SFT (Step 1) | **GRPO (This Model)** |\n|:---|:---|:---|:---|:---|\n| **MT-Bench** | Average Score (1-10) | 7.89 | 5.76 | **7.06** |\n| **LM Harness** | Average Acc (GSM8K, MMLU) | 0.7117 | 0.7028 | **0.7028** |\n\n*Comparisons made between `Qwen3-4B-Thinking-2507` (Base) and `Qwen3-4B-Thinking-2507-SimPO-Uncensored` (SimPO).*\n\n## Usage\n\n### Using llama.cpp (CLI)\n\n```bash\n# Download the model file\nhuggingface-cli download puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2-gguf \\\n  --local-dir ./models --local-dir-use-symlinks False\n\n# Run inference\n./llama-cli -m ./models/qwen3-4b-thinking-2507-grpo-uncensored-v2-q4_k_m.gguf \\\n  -p \"Give me a short introduction to large language model.\" \\\n  -n 512 \\\n  --temp 0.7\n```\n\n### Using llama-cpp-python\n\n```python\nfrom llama_cpp import Llama\n\n# Initialize the model\nmodel = Llama(\n    model_path=\"./models/qwen3-4b-thinking-2507-grpo-uncensored-v2-q4_k_m.gguf\",\n    n_ctx=32768,  # Context window\n    n_gpu_layers=-1,  # Use GPU acceleration (set to 0 for CPU only)\n)\n\n# Generate a response\nprompt = \"Give me a short introduction to large language model.\"\noutput = model.create_chat_completion(\n    messages=[\n        {\"role\": \"user\", \"content\": prompt}\n    ],\n    max_tokens=512,\n    temperature=0.7,\n)\n\nprint(output[\"choices\"][0][\"message\"][\"content\"])\n```\n\n## Data Overview\n\n### Datasets\n\nThe following datasets were used for training this model:\n* [Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Qwen2.5-Pro-1M-v0.1)\n* [AI-MO/NuminaMath-CoT](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT)\n* [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k)\n* [puwaer/cvalues_rlhf_en_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_en_cot)\n* [puwaer/cvalues_rlhf_zh_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_zh_cot)\n* [puwaer/cvalues_rlhf_jp_cot](https://huggingface.co/datasets/puwaer/cvalues_rlhf_jp_cot)\n\n### Reward Model\n\n* [puwaer/Unsafe-Reward-Qwen3-1.7B](https://huggingface.co/puwaer/Unsafe-Reward-Qwen3-1.7B)\n\n",
    "related_quantizations": []
  },
  "tags": [
    "transformers",
    "gguf",
    "text-generation",
    "en",
    "zh",
    "ja",
    "base_model:puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2",
    "base_model:quantized:puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 1,
  "downloads": 439,
  "gated": false,
  "private": false,
  "last_modified": "2026-02-01T16:03:17.000Z",
  "created_at": "2026-01-13T23:26:33.000Z",
  "pipeline_tag": "text-generation",
  "library_name": "transformers"
}

{
  "_id": "6966d4a9e23872f2ed329013",
  "id": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2-gguf",
  "modelId": "puwaer/Qwen3-4B-Thinking-2507-GRPO-Uncensored-V2-gguf",
  "sha": "5439c081f512c088a17a6b558fbfe33993b2d8f5",
  "createdAt": "2026-01-13T23:26:33.000Z",
  "lastModified": "2026-02-01T16:03:17.000Z",
  "author": "puwaer",
  "downloads": 439,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-generation",
  "library_name": "transformers",
  "siblings_count": 6
}