---

license: mit

language:

- en

tags:

- gguf

- quantized

- qwen3_5

- ornith

- abliteration

- abliterated

- uncensored

- heretic

- text-only

- q4_k_m

- llama.cpp

- ollama

- lm-studio

base_model: andrevp/Ornith-1.0-9B-Heretic-Uncensored

pipeline_tag: text-generation

library_name: gguf

inference:

parameters:

temperature: 0.6

top_p: 0.95

top_k: 20

---

Ornith-1.0-9B-Heretic-Uncensored (GGUF — Q4_K_M)

> ⚠️ Important note about this GGUF conversion

>

> The original model andrevp/Ornith-1.0-9B-Heretic-Uncensored is a multimodal (vision + text) model with bf16 precision (~17.5 GB). This GGUF conversion is a text-only Q4_K_M quantization that has been metadata-patched to work around a structural issue in the quantization pipeline. Several original capabilities are lost — see the Capability Loss and GGUF Metadata Patching sections below.

GGUF quantized version (Q4_K_M) of andrevp/Ornith-1.0-9B-Heretic-Uncensored — an abliterated (uncensored) reasoning model built on Qwen 3.5 architecture. Suitable for local inference with llama.cpp, LM Studio, Ollama, etc.

Source Model

| Property | Value |

|----------|-------|

| Base model | deepreinforce-ai/Ornith-1.0-9B |

| Abliterated by | andrevp |

| Architecture | qwen3_5 — Qwen 3.5-style hybrid (24 linear-attention / GatedDeltaNet + 8 full-attention layers, 3:1 pattern) |

| Parameters | ~9B dense (8,953,803,264) |

| Context length | 262,144 tokens (256K) |

| Original pipeline | Image-Text-to-Text (multimodal vision + text) |

| This GGUF | Text-only (vision capability removed — see below) |

| License | MIT |

| Language | English |

What is Ornith?

Ornith is a hybrid-attention reasoning model from DeepReinforce, architecturally similar to Qwen 3.5. It combines:

• 24 GatedDeltaNet (linear-attention) layers for efficient long-context processing
• 8 full-attention layers for deep reasoning — arranged in a 3:1 pattern (3 linear : 1 full, repeated 8 times)
• Built-in MTP (Multi-Token Prediction) support with mtp_num_hidden_layers: 1 for speculative decoding
• Multimodal — originally accepts both images and text inputs via a vision encoder + cross-modal projector

This abliterated variant has its refusal direction removed via weight orthogonalization (directional ablation), so it freely responds without built-in safety refusals.

Abliteration Method

Performed by andrevp using the standard directional ablation technique:

1. Data collection — 128 harmful + 128 harmless prompts, residual-stream activations recorded
2. Refusal direction — best single direction selected (layer 28) via mean-absolute-activation scoring
3. Weight orthogonalization — refusal direction projected out of o_proj/out_proj and down_proj in all layers

Reference: Arditi et al. "Refusal in LLMs is mediated by a single direction" (2024); mlabonne's abliteration guide.

Files

GGUF File

| File | Quant | Size (approx.) | Notes |

|------|-------|----------------|-------|

| Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf | Q4_K_M | ~5.2 GB | 4-bit medium — best size/quality balance for limited VRAM |

Additional Files

| File | Description |

|------|-------------|

| chat_template.jinja | Qwen chat template (preserved from original) |

📉 Capability Loss vs Original Model

This table compares what the original bf16 safetensors model offers versus what this Q4_K_M GGUF provides after quantization and metadata patching:

| Capability | Original bf16 (17.5 GB) | This GGUF Q4_K_M (5.2 GB) | Impact |

|-----------|----------------------|--------------------------|--------|

| Multimodal vision | ✅ Full image understanding via vision encoder + mmproj | ❌ Lost — no mmproj files. The metadata patch disables the MTP layer which may interact with vision pipeline. Text-only. | Major loss — the model cannot process images at all |

| MTP speculative decoding | ✅ Built-in draft model (mtp_num_hidden_layers: 1) for faster generation | ❌ Disabled — nextn_predict_layers set to 0 via metadata patch | Moderate loss — inference is slower (no speed boost from draft model). Output quality is unaffected |

| Precision | bf16 full precision | Q4_K_M 4-bit quantization | Quality degradation — 4-bit quantization loses some fidelity vs full bf16. Expect minor quality differences typical of Q4_K_M |

| Model size | ~17.5 GB | ~5.2 GB | Benefit — fits in 6–12 GB VRAM, much easier to run locally |

| Context length | 256K | 256K (same) | ✅ Preserved |

| Reasoning (<think> blocks) | ✅ Full | ✅ Preserved | ✅ Fully functional |

| Abliterated (uncensored) | ✅ Full | ✅ Preserved | ✅ Refusal direction is in the weights, not metadata — ablation carries over |

| Chat template | ✅ Qwen format | ✅ Preserved (chat_template.jinja included) | ✅ Fully functional |

| Architecture | qwen3_5 hybrid (32 layers + 1 MTP) | qwen3_5 hybrid (patched to 32 layers, MTP=0) | ✅ Core architecture preserved. Only block_count corrected from 33→32 |

Summary

This GGUF is a text-only, 4-bit, metadata-patched conversion. You get:

• ✅ The same reasoning engine with <think> blocks and uncensored behavior
• ✅ The same chat template
• ✅ The same 256K context
• ✅ Small size (5.2 GB) — fits easily on consumer GPUs
• ❌ No image/vision support
• ❌ No speculative decoding speed boost
• ⚠️ Slightly reduced output quality vs bf16 (typical Q4_K_M degradation)

Usage

llama.cpp

# Text-only inference
./llama-cli -m Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf \
  -p "<|im_start|>user\nWrite a Python function is_prime(n). Keep it short.<|im_end|>\n<|im_start|>assistant\n<think>\n" \
  -n 512 -t 8 --temp 0.6 --top-p 0.95 --top-k 20

# Server mode (OpenAI-compatible API)
./llama-server -m Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf \
  --port 8080 -t 8 -ngl 99

> Do NOT use --spec-type draft-mtp — MTP is disabled in this patched GGUF.

llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="hotdogs/andrevp_Ornith-1.0-9B-Heretic-Uncensored_GGUF",
    filename="Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf",
    n_gpu_layers=-1,   # offload all to GPU
    n_ctx=262144,       # full context
    verbose=False,
)

response = llm.create_chat_completion(
    messages=[{"role": "user", "content": "Write a Python function is_prime(n). Keep it short."}],
    temperature=0.6,
    top_p=0.95,
    top_k=20,
    max_tokens=512,
)
print(response["choices"][0]["message"]["content"])

Ollama (via Modelfile)

Create a Modelfile:

FROM ./Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf

TEMPLATE """{{ .System }}
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
<think>
{{ .Response }}</think>"""

PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20
PARAMETER num_ctx 32768

Then:

ollama create ornith-heretic -f Modelfile
ollama run ornith-heretic

LM Studio / Jan

1. Place the GGUF file in your models directory
2. Launch LM Studio / Jan, the model will be auto-detected

Chat Template

The model uses Qwen's standard <|im_start|> / <|im_end|> chat template with built-in <think>...</think> reasoning:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Your question here<|im_end|>
<|im_start|>assistant
<think>
Reasoning step-by-step...
</think>

Final answer here.<|im_end|>

Recommended Sampling Parameters

| Parameter | Value |

|-----------|-------|

| Temperature | 0.6 |

| top_p | 0.95 |

| top_k | 20 |

| Repetition penalty | 1.0–1.1 |

| Min P | 0.02–0.05 |

⚠️ Important Notes

• Abliterated / uncensored — This model has had its refusal direction removed. It may produce sensitive, controversial, or unsafe outputs. Not suitable for all audiences.
• Text-only — This GGUF cannot process images. If you need multimodal, use the original safetensors model with transformers.
• Q4_K_M quantization — 4-bit medium quality. Suitable for GPUs with 6–12 GB VRAM. Quality is slightly reduced vs higher-bit quantizations.
• MTP disabled — Speculative decoding is turned off. The model generates one token at a time (standard speed).
• Context length — 256K context is supported, but practical usable context depends on available RAM/VRAM. For best performance, start with 32K–64K.

🔧 GGUF Metadata Patching (Applied to This Model)

The GGUF file in this repo has been metadata-patched to fix a structural mismatch between the metadata header and the actual tensor content.

The Problem

The model config (config.json) defines a Qwen 3.5 hybrid architecture with MTP (Multi-Token Prediction) — an extra speculative-decoding layer. When llama.cpp quantized the safetensors, it read block_count: 33 from the config (counting 32 main layers + 1 MTP layer) and wrote that into the GGUF metadata header. However, the MTP layer tensors were not present in the final quantized output — only 32 layers of actual weights (blk.0 to blk.31) ended up in the file.

This mismatch caused llama.cpp to crash on load:

llama_model_loader: error loading model: missing tensor 'blk.32.attn_norm.weight'

The Fix

We used GGUF Metadata Editing to patch the GGUF header and correct the metadata without re-quantizing:

# 1. Correct block count from 33 → 32 (only 32 layers have actual weights)
python3 -m gguf.scripts.gguf_set_metadata Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf qwen35.block_count 32

# 2. Disable MTP speculative-decoding layers (no draft model available)
python3 -m gguf.scripts.gguf_set_metadata Ornith-1.0-9B-Heretic-Uncensored-Q4_K_M.gguf qwen35.nextn_predict_layers 0

> (Type YES when prompted to confirm the override.)

After patching, llama.cpp loads the model as a text-only LLM without expecting the missing MTP tensor. The --spec-type draft-mtp flag must not be used at inference time.

How GGUF Metadata Works

A GGUF file has two sections:

| Section | Description |

|---------|-------------|

| Metadata (header) | Think of it as the table of contents — tells llama.cpp the model name, architecture (qwen35), block_count, context length, EOS token, etc. |

| Tensors (weights) | The actual weight data, stored per layer as blk.0., blk.1., … blk.N.* |

The metadata acts as the loader's roadmap. When block_count: 33 is in the header but only 32 layers of tensors exist, llama.cpp aborts. Metadata patching edits the roadmap to match what's actually on disk — no need to re-quantize.

Pros & Cons

✅ Pros

• No re-quantization needed — saves hours of compute time for a 9B model
• Zero quality loss — patching only changes header metadata, not weight values
• Fully reversible — original block_count can be restored with the same tool
• Immediate fix — model is usable right after the patch

❌ Cons

• MTP speculative decoding is lost — the draft-model layer is disabled. Text-only inference quality is unaffected, but speed is slower.
• Not a "proper" fix — the ideal solution is fixing the upstream quantization pipeline to correctly handle MTP layers. This is a workaround.
• Requires exact metadata key names — you need to know which keys (qwen35.block_count, qwen35.nextn_predict_layers) to patch, which varies by architecture
• May confuse other tools — some GGUF-aware software (e.g. model catalogs, metadata inspectors) that reads the patched header may report inconsistency
• Multimodal not viable — with the layer count patched, vision pipelines that depend on the full architecture may not work reliably

> TL;DR: The GGUF header said 33 layers but only 32 exist on disk because the MTP draft layer was lost during quantization. Metadata patching corrects the header to match reality — we lose speculative decoding speed but the model works correctly for text inference.

Credits

• Original model: deepreinforce-ai/Ornith-1.0-9B by DeepReinforce (MIT)
• Abliterated version: andrevp/Ornith-1.0-9B-Heretic-Uncensored by andrevp
• GGUF conversion & metadata patching: hotdogs using llama.cpp

---

This is a GGUF conversion for local inference. All credit for the model itself belongs to DeepReinforce and andrevp.