What license applies to webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF?

License: apache-2.0. Verify terms on Hugging Face before commercial use.

Model Intelligence Sheet

webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF overview

Q: How do I run webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF locally?

Download a GGUF file from this page and load it in guIDE or llama.cpp. Pipeline task: text-generation.

Q: How much VRAM or disk space does webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF need?

Runs locally from ~12.57 GB disk (16 GB VRAM class GPUs with llama.cpp / guIDE).

Sakura Qwen3.8 27B Humanlike Chat 2.0 — Uncensored AutoRound Q3 K M GGUF <img src="assets/sakura logo.svg" alt="Sakura logo" width="560" / This repository prov…

ggufquantizedautoroundqwen3.8qwen27bq3q3_k_mhumanlikeuncensoredabliteratedllama.cppconversationaltext-generationendebase_model:LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUFbase_model:quantized:LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUFlicense:apache-2.0endpoints_compatibleregion:us

Runs locally from ~12.57 GB disk (16 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads

164

Likes

Pipeline

text-generation

Author

webmp3

Repository Files & Downloads

2 GGUF files detected

Direct downloads for local inference

File	Type	Quantization	Size	Link
Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf	GGUF	Q3_K_M	12.57 GB	Download
Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf	GGUF	Q3_K_M	12.57 GB	Download

Model Details

Model ID	webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF
Author	webmp3
Pipeline	text-generation
License	apache-2.0
Base model	LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUF
Last modified	2026-10-08T12:24:43.000Z

Model README

---

license: apache-2.0

license_link: LICENSE

base_model: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-GGUF

base_model_relation: quantized

quantized_by: webmp3

library_name: gguf

pipeline_tag: text-generation

language:

- en

- de

tags:

- gguf

- quantized

- autoround

- qwen3.8

- qwen

- 27b

- q3

- q3_k_m

- humanlike

- uncensored

- abliterated

- llama.cpp

- conversational

---

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

This repository provides quantized GGUF releases representing the synergy of three key components:

Uncensored + Humanlike 2.0 + AutoRound Q3_K_M.

Recommended / Winner: Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf (AutoRound GPU Q3_K_M)
Comparison Baseline: Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf (Standard llama.cpp Imatrix Q3_K_M)

> Key Result: AutoRound Q3_K_M won the predefined Q3 fidelity gate against the same-size Imatrix Q3_K_M baseline.

---

Model Provenance

Qwen/Qwen3.8-27B
  ↓
huihui-ai/Huihui-Qwen3.8-27B-abliterated
  ↓
LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
  ↓
Sakura AutoRound Q3_K_M

---

The Trio: Uncensored • Humanlike 2.0 • AutoRound

Uncensored (Abliterated Base):

The foundation originates from huihui-ai/Huihui-Qwen3.8-27B-abliterated. Refusal vectors were removed through targeted layer-wise abliteration specifically across layers 18 to 51 (leaving initial representation layers 0 to 17 and vision/MTP untouched), effectively stripping refusal mechanisms while preserving core model reasoning and capabilities.

Humanlike 2.0 (Conversational Fine-Tune):

Fine-tuned by LessThanThreeAI in LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 using on-policy distillation. The model is specifically tuned to text and converse naturally like a human, with nuance, personality, and roleplay capability—without requiring elaborate system prompt instructions.

AutoRound (Advanced 3-Bit Quantization):

Quantized using Intel's AutoRound optimization across all 64 layers against BF16 reference logits. AutoRound significantly reduces layer-wise rounding error in low-bit representation, beating the same-size importance-matrix quant across held-out domains.

---

Model Details

Direct Upstream Base: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
Abliterated Ancestor: huihui-ai/Huihui-Qwen3.8-27B-abliterated
Base Architecture: Qwen3.8-27B hybrid architecture with linear attention / SSM and full attention (64 transformer layers + 1 MTP layer)
Context Length: 262,144 tokens (native context support in llama.cpp)
License: Apache-2.0
Intended Inference: Local CPU / GPU / Vulkan execution with llama.cpp

---

Why this Quant?

Quantizing large 27B hybrid models to 3-bit typically poses severe challenges to output fidelity. In this project, AutoRound optimization was run across the model's 64 layers against full-precision (BF16) reference logits to minimize layer-wise rounding error.

To ensure rigorous validation before publishing, AutoRound Q3_K_M was benchmarked head-to-head against a same-size standard importance matrix quant (Q3_K_M) using an identical BF16 baseline across three held-out test domains (English conversational text, developer code, and Wikipedia text).

The predefined release criterion required:

File size not larger than the baseline (within 1%)
Mean KL divergence improved by at least 10%
Better KL divergence on at least 2 of 3 evaluation texts

AutoRound GPU Q3 satisfied all three criteria and won the release gate.

---

Quantization Comparison

Evaluation protocol: llama-perplexity, context length 512, 12 chunks per text, batch size 512. Base logits evaluated in BF16 precision; quantized candidates evaluated with identical settings.

| Metric | Imatrix Q3_K_M | AutoRound GPU Q3 |

|---|---:|---:|

| Size bytes | 13,500,737,152 | 13,500,736,832 |

| KL EN | 0.0421 | 0.0328 |

| KL dev/code | 0.0470 | 0.0332 |

| KL Wiki | 0.0499 | 0.0510 |

| Mean KL | 0.0463 | 0.0390 |

| Top-1 EN | 96.0% | 96.4% |

| Top-1 dev/code | 94.5% | 95.6% |

| Top-1 Wiki | 91.1% | 90.0% |

---

Important Benchmark Notes

For complete transparency:

Mean KL divergence improved by approximately 15.8% (relative drop from 0.0463 to 0.0390).
English conversational text improved (KL 0.0328 vs 0.0421; Top-1 96.4% vs 96.0%).
Developer / code text improved significantly (KL 0.0332 vs 0.0470; Top-1 95.6% vs 94.5%).
Wikipedia text regressed slightly (KL 0.0510 vs 0.0499; Top-1 90.0% vs 91.1%).
Perplexity (PPL) was slightly worse for AutoRound on all three evaluation sets (EN: 1.734 vs 1.728; dev: 1.841 vs 1.840; Wiki: 7.680 vs 7.615). PPL was not the predefined gate metric because KL divergence and top-1 agreement directly measure fidelity to original model output logits.
Size difference: AutoRound is 320 bytes smaller than the Imatrix file (13,500,736,832 vs 13,500,737,152 bytes), which is essentially identical size; 320 bytes is not a meaningful size benefit.
The results apply specifically to the tested evaluation domains.

---

Files in this Repository

|---|---|---|---:|---|

---

Download

Recommended: AutoRound Q3_K_M (Winner)

Using the Hugging Face CLI:

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf --local-dir .

Or via curl:

curl -L -o Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  https://huggingface.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF/resolve/main/Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf

Baseline: Imatrix Q3_K_M (For Comparison)

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf --local-dir .

---

llama.cpp Example

Run with llama-cli or llama-server:

llama-cli \
  -m Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  -p "<|im_start|>user\nHello! How can you assist me today?<|im_end|>\n<|im_start|>assistant\n"

> Note on Thinking: Like the upstream Humanlike-Chat 2.0 model, thinking tokens are enabled by default in the chat template.

---

Related Sakura models

webmp3/Sakura-Humanlike-Chat-2.0-JEV-AutoRound-GGUF: JEV-enabled variant with calibrated System-One decision capabilities.

---

License & Attribution

Upstream Model: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 by LessThanThreeAI.
Abliterated Ancestor: huihui-ai/Huihui-Qwen3.8-27B-abliterated by huihui-ai.
Base Architecture: Qwen3.8 by Alibaba Cloud.
License: Released under the Apache-2.0 License.
Quantization: Sakura Quantization Pipeline with Intel AutoRound & llama.cpp.

---

<details><summary>中文说明 · 樱花 (Simplified Chinese)</summary>

> English above. 本节为上文的中文翻译(Sakura = 樱花 yīnghuā);完整的独立中文版见 README_zh.md。

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

本仓库提供量化的 GGUF 发布版本,体现了三个关键要素的协同:

Uncensored + Humanlike 2.0 + AutoRound Q3_K_M。

推荐 / 胜出: Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf(AutoRound GPU Q3_K_M)
对照基线: Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf(标准 llama.cpp Imatrix Q3_K_M)

> 关键结果: AutoRound Q3_K_M 在预先设定的 Q3 保真度关卡中胜过了相同大小的 Imatrix Q3_K_M 基线。

---

模型谱系

Qwen/Qwen3.8-27B
  ↓
huihui-ai/Huihui-Qwen3.8-27B-abliterated
  ↓
LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
  ↓
Sakura AutoRound Q3_K_M

---

三者组合:Uncensored • Humanlike 2.0 • AutoRound

Uncensored(去除拒绝的基础模型):

基础来自 huihui-ai/Huihui-Qwen3.8-27B-abliterated。拒绝向量通过有针对性的逐层 abliteration 被移除,具体作用于第 18 至 51 层(保留最初的表示层 0 至 17 以及视觉/MTP 不变),在保留模型核心推理能力的同时有效去除了拒绝机制。

Humanlike 2.0(对话微调):

由 LessThanThreeAI 在 LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0 中通过 on-policy 蒸馏微调。该模型专门调校为像人一样自然地发消息和交谈,具有细腻的语气、个性和角色扮演能力,而无需复杂的系统提示指令。

AutoRound(先进的 3 比特量化):

使用 Intel 的 AutoRound 优化,针对 BF16 参照 logits 对全部 64 层进行量化。AutoRound 显著降低了低比特表示中的逐层舍入误差,在留出的各个领域上胜过了相同大小的重要性矩阵量化。

---

模型详情

直接上游基础模型: LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0
abliterated 的祖先模型: huihui-ai/Huihui-Qwen3.8-27B-abliterated
基础架构: Qwen3.8-27B 混合架构,包含线性注意力 / SSM 和全注意力(64 个 Transformer 层 + 1 个 MTP 层)
上下文长度: 262,144 个 token(llama.cpp 原生支持的上下文)
许可证: Apache-2.0
预期推理方式: 使用 llama.cpp 在本地 CPU / GPU / Vulkan 上运行

---

为什么选这个量化?

把 27B 混合模型量化到 3 比特,通常会对输出保真度造成严重挑战。在本项目中,AutoRound 优化针对全精度(BF16)参照 logits 在模型的 64 层上运行,以最小化逐层舍入误差。

为确保发布前的严格验证,我们把 AutoRound Q3_K_M 与相同大小的标准重要性矩阵量化(Q3_K_M)进行了正面对比,使用相同的 BF16 基线,在三个留出的测试领域(英文对话文本、开发者代码和维基百科文本)上测量。

预先设定的发布标准要求:

文件大小不超过基线(在 1% 以内)
平均 KL 散度至少改善 10%
在 3 个评测文本中至少 2 个上取得更好的 KL 散度

AutoRound GPU Q3 满足全部三项标准,通过了发布关卡。

---

量化对比

评测协议:llama-perplexity,上下文长度 512,每个文本 12 个块,批大小 512。基础 logits 以 BF16 精度评估;量化候选以相同设置评估。

| 指标 | Imatrix Q3_K_M | AutoRound GPU Q3 |

|---|---:|---:|

| Size bytes | 13,500,737,152 | 13,500,736,832 |

| KL EN | 0.0421 | 0.0328 |

| KL dev/code | 0.0470 | 0.0332 |

| KL Wiki | 0.0499 | 0.0510 |

| Mean KL | 0.0463 | 0.0390 |

| Top-1 EN | 96.0% | 96.4% |

| Top-1 dev/code | 94.5% | 95.6% |

| Top-1 Wiki | 91.1% | 90.0% |

---

重要的基准测试说明

为保证完全透明:

平均 KL 散度改善约 15.8%(从 0.0463 降到 0.0390 的相对降幅)。
英文对话文本有所改善(KL 0.0328 对 0.0421;Top-1 96.4% 对 96.0%)。
开发者 / 代码文本明显改善(KL 0.0332 对 0.0470;Top-1 95.6% 对 94.5%)。
维基百科文本略有退步(KL 0.0510 对 0.0499;Top-1 90.0% 对 91.1%)。
AutoRound 在全部三个评测集上的 困惑度(PPL)略差(EN: 1.734 对 1.728;dev: 1.841 对 1.840;Wiki: 7.680 对 7.615)。PPL 不是预先设定的关卡指标,因为 KL 散度和 top-1 一致性直接衡量对原始模型输出 logits 的保真度。
大小差异: AutoRound 比 Imatrix 文件小 320 字节(13,500,736,832 对 13,500,737,152 字节),大小基本相同;320 字节并不构成有意义的大小优势。
这些结果仅适用于所测试的评测领域。

---

本仓库中的文件

| 文件 | 类型 | 角色 | 大小 | SHA256 |

|---|---|---|---:|---|

---

下载

推荐:AutoRound Q3_K_M(胜出)

使用 Hugging Face CLI:

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf --local-dir .

或通过 curl:

curl -L -o Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  https://huggingface.co/webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF/resolve/main/Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf

基线:Imatrix Q3_K_M(用于对比)

hf download webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF Humanlike-Chat-2.0-Uncensored-imatrix-Q3_K_M.gguf --local-dir .

---

llama.cpp 示例

使用 llama-cli 或 llama-server 运行:

llama-cli \
  -m Humanlike-Chat-2.0-Uncensored-AutoRound-Q3_K_M.gguf \
  -ngl 99 \
  -c 8192 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  -p "<|im_start|>user\nHello! How can you assist me today?<|im_end|>\n<|im_start|>assistant\n"

> 关于思考的说明: 与上游的 Humanlike-Chat 2.0 模型一样,聊天模板中默认启用思考 token。

---

许可证与归属

上游模型: LessThanThreeAI 的 LessThanThreeAI/Qwen3.8-27B-Humanlike-Chat-2.0。
abliterated 的祖先模型: huihui-ai 的 huihui-ai/Huihui-Qwen3.8-27B-abliterated。
基础架构: 阿里云的 Qwen3.8。
许可证: 以 Apache-2.0 许可证发布。
量化: 使用 Intel AutoRound 和 llama.cpp 的 Sakura 量化流水线。

</details>

Run webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models

webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF overview

Repository Files & Downloads

Model Details

Model README

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

Model Provenance

The Trio: Uncensored • Humanlike 2.0 • AutoRound

Model Details

Why this Quant?

Quantization Comparison

Important Benchmark Notes

Files in this Repository

Download

Recommended: AutoRound Q3_K_M (Winner)

Baseline: Imatrix Q3_K_M (For Comparison)

llama.cpp Example

Related Sakura models

License & Attribution

Sakura Qwen3.8-27B Humanlike Chat 2.0 — Uncensored AutoRound Q3_K_M GGUF

模型谱系

三者组合:Uncensored • Humanlike 2.0 • AutoRound

模型详情

为什么选这个量化?

量化对比

重要的基准测试说明

本仓库中的文件

下载

推荐:AutoRound Q3_K_M(胜出)

基线:Imatrix Q3_K_M(用于对比)

llama.cpp 示例

相关的 Sakura 模型

许可证与归属

Run webmp3/Sakura-Humanlike-Chat-2.0-AutoRound-GGUF with guIDE