---

license: apache-2.0

base_model:

• meta-models/Muse-Glimmer-30B
• darkc0de/Muse-Glimmer-30B-heretic

tags:

• llama.cpp
• gguf
• quantization
• q4_k_s
• vision-language-model
• image-text-to-text
• abliterated
• not-for-all-audiences

language:

• en
• multilingual

pipeline_tag: image-text-to-text

---

Muse-Glimmer-30B-Heretic-GGUF

GGUF (GPT-Generated Unified Format) quantization of Muse-Glimmer-30B-Heretic, an abliterated (decensored) derivative of Meta's Muse-Glimmer-30B, produced with Heretic v1.4.0 by darkc0de.

This repository provides a Q4_K_S quantized GGUF for llama.cpp inference, together with the full-precision F16 reference file used for quality benchmarking.

Model Overview

| Property | Value |

|---|---|

| Base model | meta-models/Muse-Glimmer-30B (Meta Superintelligence Lab) |

| Abliteration | Heretic v1.4.0 (heretic-project.org), KL divergence 0.0743 vs. base |

| Architecture | Dense Causal Transformer + Perception Encoder (ViT-G/14, ~1.8B) |

| Language model parameters | ~29.6B total (incl. vision encoder) |

| Hidden size | 6656 |

| Layers | 52 |

| Attention | GQA, 32 Q / 2 KV heads (ratio 16:1), sliding window 2048 (3:1 local:global pattern) |

| FFN | SwiGLU, intermediate 19,968 |

| Position encoding | RoPE (θ = 500,000), local layers only |

| Vocabulary | 202,048 (200K BPE + 2,048 special tokens) |

| Context length | 131,072+ |

| Modalities | Input: text + image · Output: text |

| Base license | Apache 2.0 |

| Knowledge cutoff | January 4, 2026 |

Why Q4_K_S?

This quant targets 16 GB VRAM (RTX 4080-class) with maximum context:

| Variant | Size | 64K context + Q8_0 KV cache |

|---|---|---|

| Q4_K_S | ~14 GiB | ✅ fits in 16 GB VRAM |

| Q4_K_XL | 14.79 GiB | ❌ overflows with KV cache |

| Q4_K_M | ~14.5 GiB | ⚠️ borderline at 64K |

The model uses GQA with 2 KV heads, making the KV cache very efficient:

• 52 layers × 2 KV heads × 128 head_dim = 52 KiB/token (FP16)
• Q8_0 KV cache: 26 KiB/token → 64K context ≈ 1.7 GB

Files

| File | Size | Description |

|---|---|---|

| Muse-Glimmer-30B-Heretic-Q4_K_S.gguf | 16.1 GB | Q4_K_S quant (primary artifact) |

| Muse-Glimmer-30B-Heretic-F16.gguf | 55.7 GB | F16 reference (benchmark baseline) |

Quantization Methodology

Both files were produced from the original safetensors with llama.cpp master (commit 030ebb5, muse_glimmer architecture support required — not present in release binaries before b10344):

| Step | Tool | Output | Wall time |

|---|---|---|---|

| Convert | convert_hf_to_gguf.py (llama.cpp master) | F16 GGUF, 731 tensors | ~2.5 min |

| Quantize | llama-quantize.exe (CPU build, 16 threads) | Q4_K_S, 731 tensors | ~4 min |

Quantization commands:

python convert_hf_to_gguf.py darkc0de/Muse-Glimmer-30B-heretic \
  --outfile Muse-Glimmer-30B-Heretic-F16.gguf

llama-quantize Muse-Glimmer-30B-Heretic-F16.gguf \
  Muse-Glimmer-30B-Heretic-Q4_K_S.gguf Q4_K_S

Benchmark: Quantization Fidelity

This page intentionally does not report MMLU-style quality scores for the model itself; the base model's general capabilities are documented in the upstream card. The benchmark below measures what this repository is responsible for: how closely the Q4_K_S quant preserves the F16 reference (quantization degradation), using token-level perplexity on a fixed evaluation set.

Setup: wikitext-2 test split, 32 chunks, n_ctx=2048, batch 2048, 16 CPU threads (no GPU available on the build machine). Both files were evaluated on the same token set so systematic errors cancel out.

| Model | Perplexity (wikitext-2, 32 chunks) | Δ vs. F16 |

|---|---|---|

| F16 (reference) | 5.6439 ± 0.07361 | — |

| Q4_K_S (this quant) | 5.7831 ± 0.07601 | +0.1392 (+2.47%) |

Interpretation: A +2.47% PPL increase is at or below typical expectations for a Q4_K_S-class quant of a 30B model, indicating a successful quantization with minimal fidelity loss.

Inference Performance

GPU (NVIDIA RTX 4080 16GB, CUDA 13.3, llama.cpp b10355)

Model fully offloaded (-ngl 99), 16 CPU threads for batch processing. Q4_K_S = 15.01 GiB, fits comfortably in 16 GB VRAM.

| Test | Throughput |

|---|---|

| Prompt processing pp128 | 1902.85 ± 134.91 t/s |

| Prompt processing pp512 | 2258.22 ± 14.55 t/s |

| Prompt processing pp2048 | 2300.36 ± 3.93 t/s |

| Token generation tg64 | 38.91 ± 0.03 t/s |

| Token generation tg256 | 38.88 ± 0.01 t/s |

CPU-only (reference build, 16 threads)

Measured on the CPU-only build used for perplexity benchmarking:

| Test | Throughput |

|---|---|

| Prompt processing (pp128) | 28.31 ± 0.44 t/s |

| Token generation (tg64) | 4.08 ± 0.04 t/s |

Key numbers for practical use

• ~39 tokens/s generation on an RTX 4080 — comfortable for real-time agentic interaction
• ~2,300 tokens/s prompt ingestion — fast context loading, long documents are no bottleneck
• 15.01 GiB model leaves ~1.4 GB VRAM headroom on a 16 GB card (KV cache not yet counted; with --cache-type-k/v q8_0 a 64K context adds ~1.7 GB, still within budget)

# GPU inference (llama.cpp b10355+, CUDA build)
llama-cli -m Muse-Glimmer-30B-Heretic-Q4_K_S.gguf \
  -ngl 99 -c 65536 --cache-type-k q8_0 --cache-type-v q8_0

For agentic use, refer to the upstream card's best practices (temperature = 1.0, top_p = 0.95, top_k = 64, Reasoning strength: high for complex tasks).

Limitations and Responsible Use

• Abliterated model. This model has reduced refusal behavior by design. It may produce content that is inappropriate, offensive, or unsafe for many applications. Deploy with appropriate guardrails and human oversight, especially in agentic or real-world-action contexts.
• Age restriction. Not intended for use by individuals under 18 years of age. Deployers using it in environments accessible to minors are responsible for compliance with applicable laws.
• No safety evaluation performed here. This repository provides a quantized artifact and its fidelity benchmark only; it does not constitute a safety or capability evaluation of the model.
• Quantization edge cases. Quantized inference may show minor quality differences in edge cases compared to full precision (see benchmark above for aggregate fidelity).
• Multimodal note. Benchmarks were run in text-only mode; the perception encoder was not exercised.

License and Attribution

This work is a derivative of:

1. meta-models/Muse-Glimmer-30B — © Meta Superintelligence Lab, released under Apache 2.0.
2. darkc0de/Muse-Glimmer-30B-heretic — abliterated derivative produced with Heretic v1.4.0, retaining the Apache 2.0 license.

The GGUF conversion and quantization in this repository inherit the Apache 2.0 license. See LICENSE for the full text.

Citation

@misc{meta2026museglimmer,
  author = {Meta Superintelligence Lab},
  title = {Muse Glimmer: A 30B Multimodal Agentic Model for Local Deployment},
  year = {2026},
  url = {https://huggingface.co/meta-models/Muse-Glimmer-30B}
}

@misc{darkc0de2026heretic,
  author = {darkc0de},
  title = {Muse-Glimmer-30B-heretic},
  year = {2026},
  url = {https://huggingface.co/darkc0de/Muse-Glimmer-30B-heretic}
}

@misc{observerx2026gguf,
  author = {0bserverx},
  title = {Muse-Glimmer-30B-Heretic-GGUF (Q4_K_S)},
  year = {2026},
  url = {https://huggingface.co/0bserverx/Muse-Glimmer-30B-Heretic-GGUF}
}