---

license: apache-2.0

base_model: Qwen/Qwen3.8-27B

pipeline_tag: image-text-to-text

tags:

- qwen

- qwen3.8

- gguf

- abliterix

- multimodal

- vision-language

- llama.cpp

- mtp

- speculative-decoding

language:

- multilingual

---

<!-- Community model card derived from the official Qwen3.8-27B documentation. -->

Qwen3.8-27B Abliterated + MTP GGUF Collection

> [!IMPORTANT]

> This is an unofficial community derivative of Qwen/Qwen3.8-27B.

> It has been processed with Abliterix to reduce refusal-related behavior and is provided in four GGUF quantizations: Q8_0, Q6_K, Q4_K_M, and Q4_K_S. Each quantization folder also contains a standalone BF16 MTP (Multi-Token Prediction) draft GGUF for optional speculative decoding.

> It is not an official Qwen release and is not endorsed by the Qwen Team, Alibaba Cloud, or the Abliterix authors.</br>

Model Summary

Qwen3.8-27B is a dense 27B-parameter native vision-language model with text, image, and video understanding. The base model supports flexible thinking control and long-context workloads.

This repository preserves the base model's architecture while applying a refusal-direction intervention with Abliterix. The resulting target-model weights are provided in GGUF format together with a standalone native MTP draft for use with compatible versions of llama.cpp and related runtimes.

Relationship to the Official Model

• Official base model: Qwen/Qwen3.8-27B
• Derivative processing: Abliterix refusal-direction intervention
• Distribution format: GGUF
• Quantizations: Q8_0, Q6_K, Q4_K_M, and Q4_K_S
• Vision projectors: a matching mmproj GGUF file is included for every quantization
• Speculative draft: mtp-Qwen3.8-27B-BF16.gguf is included in every quantization folder
• Repository: GlobalCybersecurityAlliance/Qwen3.8-27B-Abliterated-MTP-GGUF
• Maintainer: Global Cybersecurity Alliance (community distribution)

The official Qwen benchmark results describe the unmodified base model only. They must not be interpreted as benchmark results for this Abliterix-processed or quantized derivative. No claim is made that this derivative preserves every capability or score of the official model.

MTP and Speculative Decoding

MTP (Multi-Token Prediction) uses the model's native draft component to propose future tokens. A compatible inference engine verifies those proposals with the target model and discards rejected candidates. When acceptance is sufficiently high, this can reduce decode latency or improve decode throughput without changing the target model's output distribution. MTP is an optional inference optimization, not a different chat mode, a larger context window, or an additional safety modification.

This repository stores the target model and MTP draft separately. The target GGUF does not contain the MTP tensors; enable MTP by loading the matching target model together with mtp-Qwen3.8-27B-BF16.gguf as the speculative draft. The MTP file is identical across all four folders and is kept in BF16 regardless of the target-model quantization.

The standalone MTP file is approximately 5.54 GiB because the split GGUF form duplicates token embeddings needed by the draft. It therefore requires substantial additional storage and memory. Simply placing the file beside the target model does not activate MTP; the runtime must explicitly load it and select draft-mtp speculative decoding.

Actual gains depend on draft acceptance rate, prompt and response length, batch size, hardware, offload settings, and speculative depth. Benchmark MTP against ordinary decoding on the target system. Increasing --spec-draft-n-max can reduce performance when acceptance is low.

What Abliterix Changes

Abliterix is used to identify and modify activation directions associated with refusal behavior. The intended effect is to reduce excessive refusals on selected evaluations. This is a weight-level behavioral intervention, not prompt engineering and not a simple system-prompt override.

Important limitations:

• Reduced refusal behavior is not guaranteed for every prompt, language, template, or inference engine.
• The model may still refuse requests because refusal behavior can be distributed across multiple layers and mechanisms.
• The intervention may change tone, calibration, reasoning quality, factual accuracy, or instruction-following behavior.
• Quantization can introduce additional quality differences compared with the BF16 derivative.
• The internal label best15 is a build-selection label, not a standardized or independently reproduced benchmark score.

Evaluation Results

The following results were measured on a separately deployed FP8 Abliterix checkpoint labeled Qwen3.8-27B-Abliterated-FP8, served through vLLM. They are included as behavioral evidence for the selected derivative checkpoint. Quantization format, runtime, prompt template, sampling configuration, and thinking-token budget can all affect the results.

Summary

| Evaluation | Scope and protocol | Result |

|---|---|---|

| IFBench | Official 300-prompt test set; 32,768-token generation limit;| 78.16% |

| StrongREJECT | First 150 rows of the official full dataset; non-thinking generation; temperature 0;| 1.5% |

| MMLU prefix sample | cais/mmlu, all/test, first 200 rows; 5-shot; thinking enabled; | 91.50% |

| MMLU balanced sample | First 30 test rows from each of 57 subjects; 1,710 questions; 5-shot; thinking enabled;| 88.01% |

Files

| Folder | Main model | Main size | Vision projector | MTP draft | Folder total | Recommended use |

|---|---|---:|---|---|---:|---|

| Q8_0/ | Qwen3.8-27B-Q8_0.gguf | 26.63 GiB | mmproj-Qwen3.8-27B-Q8_0.gguf | mtp-Qwen3.8-27B-BF16.gguf | 32.76 GiB | Highest fidelity in this collection; largest memory and storage requirement |

| Q6_K/ | Qwen3.8-27B-Q6_K.gguf | 20.57 GiB | mmproj-Qwen3.8-27B-Q6_K.gguf | mtp-Qwen3.8-27B-BF16.gguf | 26.68 GiB | Strong quality/size balance; recommended general-purpose version |

| Q4_K_M/ | Qwen3.8-27B-Q4_K_M.gguf | 15.41 GiB | mmproj-Qwen3.8-27B-Q4_K_M.gguf | mtp-Qwen3.8-27B-BF16.gguf | 21.43 GiB | Smaller mixed K-quant with a practical quality/size balance |

| Q4_K_S/ | Qwen3.8-27B-Q4_K_S.gguf | 14.52 GiB | mmproj-Qwen3.8-27B-Q4_K_S.gguf | mtp-Qwen3.8-27B-BF16.gguf | 20.52 GiB | Smallest target GGUF in this collection; prioritizes lower memory use |

Each folder contains exactly three GGUF files: one target model, one matching multimodal projector, and one standalone BF16 MTP draft. The MTP file is 5,946,009,952 bytes in every folder. It is duplicated for download convenience; downloading multiple folders also downloads duplicate copies of the same MTP draft.

For ordinary text-only inference, choose one main GGUF file. Add the standalone MTP GGUF only when enabling speculative decoding. For image or video inputs, use the mmproj file with the same quantization suffix. Do not mix, for example, a Q6_K main model with a Q4_K_M projector. A compatible runtime must support this Qwen3.8 multimodal architecture.

Download a Variant

Download one complete folder to obtain the target model, matching vision projector, and standalone MTP draft. The following example downloads Q6_K/:

python -m pip install -U huggingface_hub

hf download GlobalCybersecurityAlliance/Qwen3.8-27B-Abliterated-MTP-GGUF \
  --include "Q6_K/*" \
  --local-dir ./Qwen3.8-27B-Abliterated-MTP-GGUF

The resulting files are under ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/. Replace Q6_K with Q8_0, Q4_K_M, or Q4_K_S to download another variant.

Base Model Overview

The following architectural information is inherited from the official Qwen3.8-27B model card:

• Type: causal language model with vision encoder
• Parameters: approximately 27B
• Hidden dimension: 5,120
• Layers: 64
• Vocabulary / padded token embedding: 248,320
• Native context length: 262,144 tokens
• Extended context: up to 1,000,000 tokens with suitable scaling and runtime support
• Modalities: text, image, and video
• Thinking control: thinking mode is enabled by default in the official chat template
• MTP: native multi-token-prediction component exported here as a standalone speculative draft

For the complete official architecture description, benchmark tables, serving recipes, and best practices, refer to the original model card:

• Qwen/Qwen3.8-27B

Quick Start with llama.cpp

Use a recent llama.cpp build that explicitly supports the Qwen3.8 architecture, its multimodal projector, and draft-mtp speculative decoding. The examples use files downloaded into ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/.

Set all filenames to the folder you downloaded. When changing quantization, replace the target model and mmproj paths together. The BF16 MTP filename is the same in every folder.

Ordinary text-only server

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  -c 32768 \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8000

This baseline does not load the MTP draft. Use it to verify ordinary inference and to compare performance against MTP.

Text-only server with MTP

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  -md ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mtp-Qwen3.8-27B-BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -ngl 99 \
  -ngld 99 \
  -c 32768 \
  --host 0.0.0.0 \
  --port 8000

-md loads the standalone draft model, --spec-type draft-mtp selects MTP drafting, --spec-draft-n-max 3 sets the maximum draft depth, and -ngld controls GPU offload for the draft. A depth of 3 is only a starting point; test other values and compare them with the ordinary baseline.

Multimodal server with MTP

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  --mmproj ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mmproj-Qwen3.8-27B-Q6_K.gguf \
  -md ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mtp-Qwen3.8-27B-BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -c 32768 \
  -ngl 99 \
  -ngld 99 \
  --host 0.0.0.0 \
  --port 8000

Adjust -ngl, -ngld, context length, batch size, and parallelism according to available GPU memory and unified/system memory. The BF16 MTP draft adds significant memory pressure. Compatibility and argument names can change between llama.cpp versions, so check the help output of the installed build.

If MTP fails to load or is slower, remove -md, --spec-type, --spec-draft-n-max, and -ngld to return to ordinary decoding. Check the server statistics for draft acceptance rather than judging only from a short warm-up run.

Intended Use

Appropriate uses include:

• research into model behavior, alignment, refusal, and robustness;
• authorized evaluation and red-team testing;
• local experimentation with GGUF deployment;
• defensive cybersecurity education and research in authorized environments;
• studying the effects of quantization and activation-direction interventions.
• benchmarking MTP speculative decoding and draft acceptance on compatible runtimes.

Prohibited and High-Risk Use

Do not use this model to facilitate unlawful activity, unauthorized access, credential theft, malware deployment, privacy invasion, harassment, violence, fraud, or other harm. Operators should implement their own access controls, monitoring, rate limits, content safeguards, and human review appropriate to the deployment context.

Disclaimer

> [!CAUTION]

> USE AT YOUR OWN RISK. Abliterix processing intentionally changes refusal-related behavior and may weaken safeguards present in the official model. The model may generate inaccurate, unsafe, offensive, biased, unlawful, or otherwise harmful content. It may follow malicious instructions more readily than the official base model.

This repository and its files are provided “AS IS” and “AS AVAILABLE,” without warranties or conditions of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, accuracy, reliability, non-infringement, security, safety, or uninterrupted availability.

The maintainers, contributors, quantizers, distributors, the Qwen Team, Alibaba Cloud, and the Abliterix authors are not responsible for prompts, outputs, decisions, deployments, damages, losses, claims, liabilities, or consequences arising from use or misuse of this derivative, to the maximum extent permitted by applicable law.

Users and deployers are solely responsible for:

1. evaluating the model before deployment;
2. complying with applicable laws, regulations, licenses, platform policies, and third-party rights;
3. obtaining authorization before any cybersecurity testing;
4. preventing access by unauthorized or unsuitable users;
5. implementing safeguards appropriate to their use case;
6. independently verifying all model outputs before relying on them.

The model must not be treated as professional medical, legal, financial, safety, security, or operational advice. Do not rely on it for decisions where errors could cause injury, rights violations, financial loss, system compromise, or other material harm.

> [!CAUTION]

License and Attribution

The official model card identifies the base model license as Apache-2.0. Users must review and comply with the license and any additional terms applicable to the official base model, dependencies, inference software, and their intended use.

This derivative does not transfer ownership of the original model, trademarks, documentation, or third-party materials. “Qwen,” related marks, and official documentation remain the property of their respective owners.

---

</br>

</br>

> [!IMPORTANT]

> 本仓库是 Qwen/Qwen3.8-27B 的非官方社区衍生版本。模型经过 Abliterix 处理，以降低部分拒答相关行为，并提供 Q8_0、Q6_K、Q4_K_M 和 Q4_K_S 四种 GGUF 量化版本。每个量化目录还包含一个独立的 BF16 MTP（Multi-Token Prediction，多 token 预测）草稿 GGUF，可用于可选的推测解码。

> 本模型并非 Qwen 官方发布，也不代表 Qwen 团队、阿里云或 Abliterix 作者的立场或认可。

模型简介

Qwen3.8-27B 是一个约 270 亿参数的原生视觉语言模型，支持文本、图像和视频理解，并具备灵活的思考模式控制和长上下文能力。

本仓库在保留基础模型架构的前提下，通过 Abliterix 对拒答相关方向进行了干预，并同时提供主模型 GGUF 与独立的原生 MTP 草稿 GGUF，适用于兼容版 llama.cpp 及相关运行时。

与官方模型的关系

官方 Qwen 模型卡中的基准测试成绩仅代表未经修改的基础模型，不能视为本 Abliterix 衍生版或量化版的实测成绩。本仓库不保证衍生模型能够完整保持官方模型的全部能力或分数。

MTP 与推测解码

MTP（Multi-Token Prediction，多 token 预测）使用模型原生草稿组件预先提出后续 token，再由兼容的推理框架调用目标模型进行验证，未通过验证的候选会被丢弃。当接受率足够高时，这种方式可以降低解码延迟或提升解码吞吐量，同时不改变目标模型的输出分布。MTP 是可选的推理优化，并非新的对话模式、更大的上下文窗口或额外的安全行为修改。

本仓库将目标模型与 MTP 草稿分开保存。主模型 GGUF 不包含 MTP 张量；要启用 MTP，需要同时加载所选目标模型和作为推测草稿的 mtp-Qwen3.8-27B-BF16.gguf。四个目录中的 MTP 文件完全相同，并且无论主模型采用何种量化，MTP 草稿都保持 BF16。

独立 MTP 文件约为 5.54 GiB，因为拆分式 GGUF 会复制草稿所需的 token 嵌入。因此，它会显著增加存储和运行内存占用。仅将文件放在主模型旁边不会自动启用 MTP；推理框架必须显式加载该文件并选择 draft-mtp 推测解码。

实际收益取决于草稿接受率、提示词与回答长度、批大小、硬件、卸载设置和推测深度。请在目标系统中将 MTP 与普通解码直接对比；当接受率较低时，提高 --spec-draft-n-max 反而可能降低性能。

Abliterix 修改说明

Abliterix 用于识别并修改与拒答行为相关的激活方向，目标是在特定评测中减少过度拒答。这属于权重层面的行为干预，并非提示词工程或简单覆盖系统提示词。

需要注意：

• 模型并不保证对所有提示词、语言、聊天模板或推理框架都降低拒答。
• 拒答行为可能分布于多个层和机制中，因此模型仍然可能拒绝部分请求。
• 该处理可能影响模型语气、置信度校准、推理质量、事实准确性和指令遵循能力。
• GGUF 量化可能进一步造成与 BF16 衍生模型不同的质量变化。

实测结果

以下结果来自通过 vLLM 部署的独立 FP8 Abliterix 检查点 Qwen3.8-27B-Abliterated-FP8，这些数据用于说明所选衍生检查点的实测行为。量化格式、推理框架、提示模板、采样配置和思考 token 预算均可能影响结果。

汇总

| 评测 | 范围与方法 | 结果 |

|---|---|---|

| IFBench | 官方 300 条测试集；生成上限 32,768 tokens； | 78.16% |

| StrongREJECT | 官方完整数据集前 150 条；关闭思考；温度 0；| 1.5% |

| MMLU 前缀样本 | cais/mmlu 的 all/test 前 200 条；5-shot；开启思考；| 91.50% |

| MMLU 均衡样本 | 57 个主题各取测试集前 30 条，共 1,710 题；5-shot；开启思考；| 88.01% |

文件说明

| Folder | Main model | Main size | Vision projector | MTP draft | Folder total | Recommended use |

|---|---|---:|---|---|---:|---|

| Q8_0/ | Qwen3.8-27B-Q8_0.gguf | 26.63 GiB | mmproj-Qwen3.8-27B-Q8_0.gguf | mtp-Qwen3.8-27B-BF16.gguf | 32.76 GiB | Highest fidelity in this collection; largest memory and storage requirement |

| Q6_K/ | Qwen3.8-27B-Q6_K.gguf | 20.57 GiB | mmproj-Qwen3.8-27B-Q6_K.gguf | mtp-Qwen3.8-27B-BF16.gguf | 26.68 GiB | Strong quality/size balance; recommended general-purpose version |

| Q4_K_M/ | Qwen3.8-27B-Q4_K_M.gguf | 15.41 GiB | mmproj-Qwen3.8-27B-Q4_K_M.gguf | mtp-Qwen3.8-27B-BF16.gguf | 21.43 GiB | Smaller mixed K-quant with a practical quality/size balance |

| Q4_K_S/ | Qwen3.8-27B-Q4_K_S.gguf | 14.52 GiB | mmproj-Qwen3.8-27B-Q4_K_S.gguf | mtp-Qwen3.8-27B-BF16.gguf | 20.52 GiB | Smallest target GGUF in this collection; prioritizes lower memory use |

每个目录都包含三个 GGUF 文件：一个目标模型、一个配套多模态投影器，以及一个独立的 BF16 MTP 草稿。每个目录中的 MTP 文件大小均为 5,946,009,952 字节。该文件为方便按目录下载而重复存放；同时下载多个目录会得到多份相同的 MTP 草稿。

普通纯文本推理只需要选择一个主 GGUF 文件；仅在启用推测解码时才额外加载独立 MTP GGUF。图像或视频输入还需要使用具有相同量化后缀的配套 mmproj 文件，例如不要将 Q6_K 主模型与 Q4_K_M 投影文件混用。同时，推理框架必须支持 Qwen3.8 多模态架构。

下载指定版本

下载一个完整目录即可同时获得目标模型、配套视觉投影器和独立 MTP 草稿。以下示例下载 Q6_K/：

python -m pip install -U huggingface_hub

hf download GlobalCybersecurityAlliance/Qwen3.8-27B-Abliterated-MTP-GGUF \
  --include "Q6_K/*" \
  --local-dir ./Qwen3.8-27B-Abliterated-MTP-GGUF

下载后的文件位于 ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/。如需其他版本，请将 Q6_K 替换为 Q8_0、Q4_K_M 或 Q4_K_S。

基础模型概览

使用 llama.cpp

请使用明确支持 Qwen3.8 架构、多模态投影器及 draft-mtp 推测解码的新版 llama.cpp。以下示例假设文件位于 ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/。

切换量化版本时，请同时替换目标模型与 mmproj 路径。每个目录中的 BF16 MTP 文件名均相同。

普通纯文本服务

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  -c 32768 \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8000

此基线命令不加载 MTP 草稿，可用于验证普通推理，并与 MTP 性能进行对比。

启用 MTP 的纯文本服务

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  -md ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mtp-Qwen3.8-27B-BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -ngl 99 \
  -ngld 99 \
  -c 32768 \
  --host 0.0.0.0 \
  --port 8000

-md 用于加载独立草稿模型，--spec-type draft-mtp 选择 MTP 草稿方式，--spec-draft-n-max 3 设置最大草稿深度，-ngld 控制草稿模型的 GPU 卸载。深度 3 只是起点，请测试其他取值并与普通基线对比。

启用 MTP 的多模态服务

llama-server \
  -m ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/Qwen3.8-27B-Q6_K.gguf \
  --mmproj ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mmproj-Qwen3.8-27B-Q6_K.gguf \
  -md ./Qwen3.8-27B-Abliterated-MTP-GGUF/Q6_K/mtp-Qwen3.8-27B-BF16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  -c 32768 \
  -ngl 99 \
  -ngld 99 \
  --host 0.0.0.0 \
  --port 8000

请根据 GPU 显存、统一内存或系统内存调整 -ngl、-ngld、上下文长度、批大小和并行设置。BF16 MTP 草稿会显著增加内存压力。不同版本 llama.cpp 的兼容性与参数名称可能变化，请以当前安装版本的帮助信息为准。

如果 MTP 无法加载或速度更慢，请移除 -md、--spec-type、--spec-draft-n-max 和 -ngld，恢复普通解码。评估时应查看服务器输出的草稿接受率，不要只依据短暂的预热运行判断。

预期用途

适合的用途包括：模型行为、对齐、拒答与鲁棒性研究；经授权的评估与红队测试；GGUF 本地部署实验；合法授权环境中的防御性网络安全教育与研究；量化和激活方向干预效果研究；在兼容运行时中测试 MTP 推测解码与草稿接受率。

禁止及高风险用途

不得使用本模型实施或协助违法活动、未经授权的系统访问、凭据窃取、恶意软件投放、侵犯隐私、骚扰、暴力、欺诈或其他伤害行为。部署者应根据应用场景自行实施访问控制、审计监控、速率限制、内容保护和人工复核。

免责声明

> [!CAUTION]

> [!CAUTION]

> 使用者自行承担全部风险。 Abliterix 处理会主动改变拒答相关行为，可能削弱官方模型原有的部分安全保护。模型可能生成错误、不安全、冒犯性、偏见性、违法或其他有害内容，也可能比官方基础模型更容易遵循恶意指令。

本仓库及其中所有文件均按“现状”和“可用状态”提供，不作任何明示或默示保证，包括但不限于适销性、特定用途适用性、准确性、可靠性、不侵权性、安全性、无害性或持续可用性保证。

在适用法律允许的最大范围内，维护者、贡献者、量化者、分发者、Qwen 团队、阿里云以及 Abliterix 作者，均不对因使用或误用本衍生模型而产生的提示词、输出、决策、部署、损害、损失、索赔、责任或后果承担责任。

使用者和部署者应自行负责：

1. 在部署前充分评估模型；
2. 遵守适用法律法规、许可证、平台规则和第三方权利；
3. 在开展任何网络安全测试前取得明确授权；
4. 防止未经授权或不适合的用户访问；
5. 根据实际用途实施必要的安全保护措施；
6. 在依赖任何模型输出前进行独立核验。

本模型不得被视为专业的医疗、法律、金融、安全、网络安全或运营建议。对于错误可能导致人身伤害、权利侵害、经济损失、系统失陷或其他重大损害的决策，不得直接依赖模型输出。

许可证与署名

官方模型卡将基础模型许可证标注为 Apache-2.0。使用者必须自行查阅并遵守官方基础模型、依赖软件、推理框架及实际用途所适用的许可证和附加条款。

本衍生版本不转移原始模型、商标、文档或任何第三方材料的所有权。“Qwen”及相关标识与官方资料仍归各自权利人所有。