---

license: apache-2.0

base_model: Qwen/Qwen3.8-27B

pipeline_tag: image-text-to-text

tags:

- qwen

- qwen3.8

- gguf

- abliterix

- multimodal

- vision-language

- llama.cpp

language:

- multilingual

---

<!-- Community model card derived from the official Qwen3.8-27B documentation. -->

Qwen3.8-27B Abliterated GGUF Collection

> [!IMPORTANT]

> This is an unofficial community derivative of Qwen/Qwen3.8-27B.

> It has been processed with Abliterix to reduce refusal-related behavior and is provided in four GGUF quantizations: Q8_0, Q6_K, Q4_K_M, and Q4_K_S.

> It is not an official Qwen release and is not endorsed by the Qwen Team, Alibaba Cloud, or the Abliterix authors.</br>

> 本仓库是 Qwen/Qwen3.8-27B 的非官方社区衍生版本。模型经过 Abliterix 处理，以降低部分拒答相关行为，并提供 Q8_0、Q6_K、Q4_K_M 和 Q4_K_S 四种 GGUF 量化版本。

> 本模型并非 Qwen 官方发布，也不代表 Qwen 团队、阿里云或 Abliterix 作者的立场或认可。

Model Summary / 模型简介

Qwen3.8-27B is a dense 27B-parameter native vision-language model with text, image, and video understanding. The base model supports flexible thinking control and long-context workloads.

This repository preserves the base model's architecture while applying a refusal-direction intervention with Abliterix. The resulting weights are provided in GGUF format for use with compatible versions of llama.cpp and related runtimes.

Qwen3.8-27B 是一个约 270 亿参数的原生视觉语言模型，支持文本、图像和视频理解，并具备灵活的思考模式控制和长上下文能力。

本仓库在保留基础模型架构的前提下，通过 Abliterix 对拒答相关方向进行了干预，并提供适用于兼容版 llama.cpp 及相关运行时的 GGUF 权重。

Relationship to the Official Model / 与官方模型的关系

• Official base model: Qwen/Qwen3.8-27B
• Derivative processing: Abliterix refusal-direction intervention
• Distribution format: GGUF
• Quantizations: Q8_0, Q6_K, Q4_K_M, and Q4_K_S
• Vision projectors: a matching mmproj GGUF file is included for every quantization
• Maintainer: Global Cybersecurity Alliance (community distribution)

The official Qwen benchmark results describe the unmodified base model only. They must not be interpreted as benchmark results for this Abliterix-processed or quantized derivative. No claim is made that this derivative preserves every capability or score of the official model.

官方 Qwen 模型卡中的基准测试成绩仅代表未经修改的基础模型，不能视为本 Abliterix 衍生版或量化版的实测成绩。本仓库不保证衍生模型能够完整保持官方模型的全部能力或分数。

What Abliterix Changes / Abliterix 修改说明

Abliterix is used to identify and modify activation directions associated with refusal behavior. The intended effect is to reduce excessive refusals on selected evaluations. This is a weight-level behavioral intervention, not prompt engineering and not a simple system-prompt override.

Important limitations:

• Reduced refusal behavior is not guaranteed for every prompt, language, template, or inference engine.
• The model may still refuse requests because refusal behavior can be distributed across multiple layers and mechanisms.
• The intervention may change tone, calibration, reasoning quality, factual accuracy, or instruction-following behavior.
• Quantization can introduce additional quality differences compared with the BF16 derivative.
• The internal label best15 is a build-selection label, not a standardized or independently reproduced benchmark score.

Abliterix 用于识别并修改与拒答行为相关的激活方向，目标是在特定评测中减少过度拒答。这属于权重层面的行为干预，并非提示词工程或简单覆盖系统提示词。

需要注意：

• 模型并不保证对所有提示词、语言、聊天模板或推理框架都降低拒答。
• 拒答行为可能分布于多个层和机制中，因此模型仍然可能拒绝部分请求。
• 该处理可能影响模型语气、置信度校准、推理质量、事实准确性和指令遵循能力。
• GGUF 量化可能进一步造成与 BF16 衍生模型不同的质量变化。

Files / 文件说明

| Quantization | Main model | Matching vision projector | Recommended use |

|---|---|---|---|

| Q8_0 | Qwen3.8-27B-Q8_0.gguf | mmproj-Qwen3.8-27B-Q8_0.gguf | Highest fidelity in this collection; largest memory and storage requirement |

| Q6_K | Qwen3.8-27B-Q6_K.gguf | mmproj-Qwen3.8-27B-Q6_K.gguf | Strong quality/size balance; recommended general-purpose version |

| Q4_K_M | Qwen3.8-27B-Q4_K_M.gguf | mmproj-Qwen3.8-27B-Q4_K_M.gguf | Smaller mixed K-quant with a practical quality/size balance |

| Q4_K_S | Qwen3.8-27B-Q4_K_S.gguf | mmproj-Qwen3.8-27B-Q4_K_S.gguf | Smallest GGUF in this collection; prioritizes lower memory use |

Each local build directory may also contain QUANTIZATION_COMPLETE.json. This is build-completion metadata and is not required for inference.

For text-only inference, choose one main GGUF file. For image or video inputs, use the mmproj file with the same quantization suffix. Do not mix, for example, a Q6_K main model with a Q4_K_M projector. A compatible runtime must support this Qwen3.8 multimodal architecture.

纯文本推理只需要选择一个主 GGUF 文件；图像或视频输入还需要使用具有相同量化后缀的配套 mmproj 文件，例如不要将 Q6_K 主模型与 Q4_K_M 投影文件混用。同时，推理框架必须支持 Qwen3.8 多模态架构。

Base Model Overview / 基础模型概览

The following architectural information is inherited from the official Qwen3.8-27B model card:

• Type: causal language model with vision encoder
• Parameters: approximately 27B
• Hidden dimension: 5,120
• Layers: 64
• Vocabulary / padded token embedding: 248,320
• Native context length: 262,144 tokens
• Extended context: up to 1,000,000 tokens with suitable scaling and runtime support
• Modalities: text, image, and video
• Thinking control: thinking mode is enabled by default in the official chat template

For the complete official architecture description, benchmark tables, serving recipes, and best practices, refer to the original model card:

• Qwen/Qwen3.8-27B

Quick Start with llama.cpp / 使用 llama.cpp

Use a recent llama.cpp build that explicitly supports the Qwen3.8 architecture and its multimodal projector.

Set the filenames to the quantization you downloaded. The examples below use Q6_K; Q8_0, Q4_K_M, and Q4_K_S work the same way when both filenames are replaced with the matching pair.

请将文件名替换为你下载的量化版本。以下示例使用 Q6_K；使用 Q8_0、Q4_K_M 或 Q4_K_S 时，应同时替换主模型与对应的 mmproj 文件名。

Text-only server / 纯文本服务

llama-server \
  -m Qwen3.8-27B-Q8_K.gguf \
  -c 32768 \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8000

Multimodal server / 多模态服务

llama-server \
  -m Qwen3.8-27B-Q8_K.gguf \
  --mmproj mmproj-Qwen3.8-27B-Q8_K.gguf \
  -c 32768 \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8000

Adjust -ngl, context length, batch size, and parallelism according to available GPU memory and unified/system memory. Compatibility and argument names may vary across llama.cpp versions.

请根据 GPU 显存、统一内存或系统内存调整 -ngl、上下文长度和端口。不同版本 llama.cpp 的兼容性与参数名称可能有所不同。

Intended Use / 预期用途

Appropriate uses include:

• research into model behavior, alignment, refusal, and robustness;
• authorized evaluation and red-team testing;
• local experimentation with GGUF deployment;
• defensive cybersecurity education and research in authorized environments;
• studying the effects of quantization and activation-direction interventions.

适合的用途包括：模型行为、对齐、拒答与鲁棒性研究；经授权的评估与红队测试；GGUF 本地部署实验；合法授权环境中的防御性网络安全教育与研究；量化和激活方向干预效果研究。

Prohibited and High-Risk Use / 禁止及高风险用途

Do not use this model to facilitate unlawful activity, unauthorized access, credential theft, malware deployment, privacy invasion, harassment, violence, fraud, or other harm. Operators should implement their own access controls, monitoring, rate limits, content safeguards, and human review appropriate to the deployment context.

不得使用本模型实施或协助违法活动、未经授权的系统访问、凭据窃取、恶意软件投放、侵犯隐私、骚扰、暴力、欺诈或其他伤害行为。部署者应根据应用场景自行实施访问控制、审计监控、速率限制、内容保护和人工复核。

Disclaimer / 免责声明

> [!CAUTION]

> USE AT YOUR OWN RISK. Abliterix processing intentionally changes refusal-related behavior and may weaken safeguards present in the official model. The model may generate inaccurate, unsafe, offensive, biased, unlawful, or otherwise harmful content. It may follow malicious instructions more readily than the official base model.

This repository and its files are provided “AS IS” and “AS AVAILABLE,” without warranties or conditions of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, accuracy, reliability, non-infringement, security, safety, or uninterrupted availability.

The maintainers, contributors, quantizers, distributors, the Qwen Team, Alibaba Cloud, and the Abliterix authors are not responsible for prompts, outputs, decisions, deployments, damages, losses, claims, liabilities, or consequences arising from use or misuse of this derivative, to the maximum extent permitted by applicable law.

Users and deployers are solely responsible for:

1. evaluating the model before deployment;
2. complying with applicable laws, regulations, licenses, platform policies, and third-party rights;
3. obtaining authorization before any cybersecurity testing;
4. preventing access by unauthorized or unsuitable users;
5. implementing safeguards appropriate to their use case;
6. independently verifying all model outputs before relying on them.

The model must not be treated as professional medical, legal, financial, safety, security, or operational advice. Do not rely on it for decisions where errors could cause injury, rights violations, financial loss, system compromise, or other material harm.

> [!CAUTION]

> 使用者自行承担全部风险。 Abliterix 处理会主动改变拒答相关行为，可能削弱官方模型原有的部分安全保护。模型可能生成错误、不安全、冒犯性、偏见性、违法或其他有害内容，也可能比官方基础模型更容易遵循恶意指令。

本仓库及其中所有文件均按“现状”和“可用状态”提供，不作任何明示或默示保证，包括但不限于适销性、特定用途适用性、准确性、可靠性、不侵权性、安全性、无害性或持续可用性保证。

在适用法律允许的最大范围内，维护者、贡献者、量化者、分发者、Qwen 团队、阿里云以及 Abliterix 作者，均不对因使用或误用本衍生模型而产生的提示词、输出、决策、部署、损害、损失、索赔、责任或后果承担责任。

使用者和部署者应自行负责：

1. 在部署前充分评估模型；
2. 遵守适用法律法规、许可证、平台规则和第三方权利；
3. 在开展任何网络安全测试前取得明确授权；
4. 防止未经授权或不适合的用户访问；
5. 根据实际用途实施必要的安全保护措施；
6. 在依赖任何模型输出前进行独立核验。

本模型不得被视为专业的医疗、法律、金融、安全、网络安全或运营建议。对于错误可能导致人身伤害、权利侵害、经济损失、系统失陷或其他重大损害的决策，不得直接依赖模型输出。

License and Attribution / 许可证与署名

The official model card identifies the base model license as Apache-2.0. Users must review and comply with the license and any additional terms applicable to the official base model, dependencies, inference software, and their intended use.

This derivative does not transfer ownership of the original model, trademarks, documentation, or third-party materials. “Qwen,” related marks, and official documentation remain the property of their respective owners.

官方模型卡将基础模型许可证标注为 Apache-2.0。使用者必须自行查阅并遵守官方基础模型、依赖软件、推理框架及实际用途所适用的许可证和附加条款。

本衍生版本不转移原始模型、商标、文档或任何第三方材料的所有权。“Qwen”及相关标识与官方资料仍归各自权利人所有。