markgb1/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-iMatrix-MTP-GGUF overview

---

license: apache-2.0

tags:

• qwen3_5
• qwen3.6
• gguf
• mtp
• speculative-decoding
• fine-tune
• unsloth
• heretic
• uncensored
• abliterated
• multi-stage tuned
• 40B
• dense
• vision
• multimodal
• mmproj
• long-context

base_model: DavidAU/Qwen3.6-40B-Claude-4.6-Opus-Deckard-Heretic-Uncensored-Thinking-NEO-CODE-Di-IMatrix-MAX-GGUF

datasets:

- TeichAI/claude-4.5-opus-high-reasoning-250x

- DavidAU/PkDick-Deckard-5-Datasets

- allenai/WildChat-1M

pipeline_tag: text-generation

---

This repository represents an effort to graft and re-train an MTP layer for DavidAU's Qwen3.6-40B model, inspired by Piehsoft's untrained MTP grafts.

Currently only offering q8_0 quantizations, but this will eventually expand once MTP retraining is finalized.

Version 1 (complete)

A simple retraining attempt that used the following procedure:

1. Ran the 40B model on 117 random prompts
2. Record what it was thinking at every position by introspecting llama.cpp

- Specifically, recorded which 64 tokens it thought were most likely next and how confident it was in each.

- Note: Substituted h_t with embedding(input_token) to move forward with the data that was immediately accessible.

1. Use the recorded data to retrain the MTP layer: align what the MTP's predictions with reality

- This data did not include hidden states, but since the trained head is shown real-world verifier logprobs during training its bias toward the most common tokens is effectively re-calibrated.

Compared to Piehsoft's untrained MTP head, this methodology

• Significantly improved acceptances rates in structured text / coding scenarios
• Provided little to no impact on high-entropy scenarios such as creative writing / casual chat.

Ultimately proved that improving acceptance rates via retraining was possible. Further improvements seem to require:

- Larger corpus (dataset)

- Higher text entropy within corpus

- Most importantly, data collection of the verifier hidden states

Version 2 (in-progress)

EDIT: Training using hidden states ended up generating significantly worse results than the methodology used in v1. Alternative paths are being considered, but this may potentially end as a stalemate.

~Training is in-progress, which builds on-top of the v1. This version likely won't be a final version, but rather will be used to help steer direction. The main changes will be:~

• ~Collect data using a modified version of llama.cpp that exposes data from the 40B model's hidden states (h_t)~
• ~Use a dataset with high entropy: WildChat-1M~

- ~Not training on the full dataset (would take a long time), will use a subset of randomly selected samples.~

• ~Increase epochs from 3 to 5 (and tweak a few other training settings)~