---

license: apache-2.0

base_model: huihui-ai/Huihui-Qwen-AgentWorld-35B-A3B-abliterated

base_model_relation: quantized

tags:

• gguf
• llama.cpp
• qwen3_5_moe
• agent
• world-model
• abliterated
• uncensored
• quantized
• q3_k_m
• imatrix

language:

• en
• zh

pipeline_tag: text-generation

---

Huihui-Qwen-AgentWorld-35B-A3B-abliterated UD-Q3_K_M GGUF

An importance-matrix UD-Q3_K_M GGUF quant of

huihui-ai/Huihui-Qwen-AgentWorld-35B-A3B-abliterated.

Why this quant exists: the BF16 source is roughly 65-67 GB and is not practical on 22 GB cards. This UD-Q3_K_M build is 15.5 GiB / 16.65 GB, leaving enough working room for KV cache and compute buffers on a single 22 GB RTX 2080 Ti while preserving the abliterated AgentWorld model behavior for local agent experiments.

| File | Size | BPW |

|------|------|-----|

| Huihui-Qwen-AgentWorld-35B-A3B-abliterated-UD-Q3_K_M.gguf | 15.5 GiB / 16.65 GB | 3.84 |

Lineage

• Base: Qwen/Qwen-AgentWorld-35B-A3B — Qwen AgentWorld 35B-A3B MoE model for agent and environment-simulation use cases.
• Abliteration: huihui-ai/Huihui-Qwen-AgentWorld-35B-A3B-abliterated, described upstream as an uncensored/abliterated variant with reduced refusal behavior.
• This quant: UD-Q3_K_M, produced with llama.cpp from the local F16 GGUF conversion of Huihui's BF16 release.

Quick Stats

| Metric | Value |

|---|---|

| Source GGUF | Huihui-Qwen-AgentWorld-35B-A3B-abliterated-F16.gguf |

| Quantizer | llama.cpp Vulkan build b9870 |

| Quant type | Q3_K_M with imatrix + tensor map |

| GGUF ftype | Q3_K - Medium |

| Output file size | 16,660,085,760 bytes |

| Output file size | 15.52 GiB / 16.66 GB |

| Reported BPW | 3.84 |

| Imatrix entries | 510 |

| Imatrix chunks | 76 |

Tensor type counts:

| Type | Count |

|---|---:|

| F32 | 361 |

| IQ3_XXS | 78 |

| IQ4_XS | 39 |

| Q6_K | 15 |

| Q8_0 | 240 |

This tensor map follows the public UD-style Q3_K_M layout used for Qwen-AgentWorld. A few additional attention output tensors stayed at Q6_K, making this build slightly more conservative than the smallest possible Q3 variant.

llama.cpp Server

Recommended launch:

llama-server \
  -m Huihui-Qwen-AgentWorld-35B-A3B-abliterated-UD-Q3_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --parallel 1 \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

For a pinned single-GPU Vulkan setup, expose only the intended GPU before launch, for example:

CUDA_VISIBLE_DEVICES= GGML_VK_VISIBLE_DEVICES=1 llama-server \
  -m Huihui-Qwen-AgentWorld-35B-A3B-abliterated-UD-Q3_K_M.gguf \
  --host 0.0.0.0 \
  --port 8083 \
  -ngl 99 \
  -c 8192 \
  --parallel 1 \
  --jinja \
  --reasoning on \
  --reasoning-format deepseek

Important runtime note: run the server with reasoning parsing enabled. In local llama.cpp testing, server-level --reasoning off could still lead this model to emit Thinking Process: as ordinary message.content. The more reliable route was:

1. Start llama.cpp with --reasoning on --reasoning-format deepseek.
2. For direct final-answer calls, pass request-level chat_template_kwargs: {"enable_thinking": false}.
3. For thinking-mode calls, omit that override and read reasoning from message.reasoning_content.

Example direct call body:

{
  "model": "huihui-qwen-agentworld-35b-a3b-abliterated-ud-q3km",
  "messages": [
    {"role": "user", "content": "Write five concise deployment checks."}
  ],
  "chat_template_kwargs": {"enable_thinking": false},
  "temperature": 0.2,
  "max_tokens": 256
}

Local Smoke Test

Test host: dual RTX 2080 Ti, llama.cpp Vulkan backend, model loaded on one 22 GB card.

| Test | Result |

|---|---|

| Load on one RTX 2080 Ti | OK |

| VRAM after load at -c 8192 | ~15.8 GiB |

| N=3 direct decode speed | median 82.32 tok/s, range 82.26-82.55 tok/s |

| N=3 prompt eval speed | median 319.84 tok/s |

| Thinking-mode probe | OK; reasoning separated into message.reasoning_content |

| JSON extraction | OK; valid JSON and array fields preserved |

| Agent patch-plan prompt | OK; exact five-bullet output |

| Tool routing JSON | OK; four valid routes |

| Chinese agent triage | OK; six concise checks |

| Abliterated/directness sanity | OK; 3/3 safe direct prompts had no refusal phrases |

These are smoke tests, not a standardized benchmark. Quality should be validated against your own agent workload before relying on the quant in production.

Quantization Recipe

The imatrix was generated from a small synthetic calibration set focused on terminal, repo-editing, JSON extraction, tool-call style transcripts, Chinese assistant turns, and safe fictional directness prompts.

llama-imatrix \
  -m Huihui-Qwen-AgentWorld-35B-A3B-abliterated-F16.gguf \
  -f huihui-agentworld-calibration.txt \
  -o imatrix-huihui-agentworld-76.gguf \
  --output-format gguf \
  --chunks 76 \
  -c 512 \
  -ngl auto \
  --split-mode none \
  --main-gpu 0 \
  --no-ppl

llama-quantize \
  --imatrix imatrix-huihui-agentworld-76.gguf \
  --tensor-type-file ud-q3km-tensor-types.txt \
  Huihui-Qwen-AgentWorld-35B-A3B-abliterated-F16.gguf \
  Huihui-Qwen-AgentWorld-35B-A3B-abliterated-UD-Q3_K_M.gguf \
  Q3_K_M 24

The following helper files are included for reproducibility:

| File | Purpose |

|---|---|

| quantization/imatrix-huihui-agentworld-76.gguf | Importance matrix used for this quant |

| quantization/ud-q3km-tensor-types.txt | Tensor type map |

| quantization/huihui-agentworld-calibration.txt | Synthetic calibration text |

Abliteration Sanity

The upstream model is an abliterated build. Local safe smoke tests checked for functional directness using non-graphic fictional and agentic prompts. With request-level chat_template_kwargs: {"enable_thinking": false}, the model returned direct content without refusal phrases in all three abliterated sanity prompts.

This is not a safety guarantee. It is only a practical sanity check that the quantized GGUF still behaves like the abliterated upstream in ordinary local testing.

License

Inherits from the upstream model: Apache-2.0.

⚠ Safety: The upstream Huihui model card warns that refusal behavior and safety filtering are significantly reduced. Use this model in controlled environments and review outputs carefully.

Credits

• Source model: Qwen Team / Alibaba
• Abliteration: huihui-ai
• Quantization: this checkpoint, by coolthor
• Tooling: llama.cpp

---

📝 Quantized & benchmarked by ai-muninn — writeups on how it was built and how it actually runs.