---

license: mit

base_model: cantina-security/apex-flash-1-abliterated

base_model_relation: quantized

tags:

• gguf
• llama.cpp
• glm5_next
• abliterated
• security-research

pipeline_tag: image-text-to-text

---

apex-flash-1-abliterated GGUF

GGUF quantizations of cantina-security/apex-flash-1-abliterated, an abliterated derivative of apex-flash-1. The model uses the GLM-5.3-Flash (glm5-next) architecture: about 320B total parameters, 288 routed experts with 8 active per token, hybrid KDA linear attention and DeepSeek sparse attention, and a 1M context window.

Per the original model card, this model is for authorized security research in environments you own or have permission to test. Its refusal behavior was changed broadly, not only for security tasks.

Files

Every quant is split into 8 parts. Point llama.cpp at the -00001-of-00008.gguf file and it loads the rest.

| Quant | Size | BPW | Smoke test | Notes |

|---|---|---|---|---|

| Q8_0 | 341 GB | 8.51 | ✅ passed | converted directly from the BF16 weights |

| Q5_K_M | 227 GB | 5.67 | ✅ passed | |

| Q4_K_M | 193 GB | 4.82 | ✅ passed | recommended starting point |

| Q3_K_M | 153 GB | 3.82 | ✅ passed | |

| Q2_K | 117 GB | 2.93 | ✅ passed | smallest; expect noticeable quality loss |

| mmproj (F16) | 1.13 GB | 16 | ✅ passed | vision projector, works with any quant |

All quants except Q8_0 were made from the Q8_0 with llama-quantize --allow-requantize. No importance matrix was used, so there are no IQ quants. All GGUFs include the MTP (next-token prediction) head and the original chat template.

Each quant was published only after it passed the smoke test below.

Usage

This model needs a llama.cpp build with glm5-next support (ggml-org/llama.cpp#27773 or later).

llama-server -m apex-flash-1-abliterated-Q4_K_M-00001-of-00008.gguf \
  --mmproj mmproj-apex-flash-1-abliterated-f16.gguf \
  --jinja -c 32768

• Reasoning: the chat template always opens a <think> block. Reasoning effort defaults to Max. Set it to low or high per request with "chat_template_kwargs": {"reasoning_effort": "low"}.
• Sampling: the original generation_config.json uses temperature 1.0 and top_p 0.95.
• MoE offload: with limited VRAM, keep experts in system RAM using --n-cpu-moe N or -ot "exps=CPU".
• Low-RAM CPU inference: if the model is larger than your RAM and runs from mmap, add --no-repack. Otherwise the CPU backend tries to copy the K-quant weights into anonymous memory.

Smoke tests

Each quant is loaded in llama-server (CPU) and must pass all of these before upload:

1. Chat template: /apply-template must render the GLM format [gMASK]<sop><|system|>…<|user|>…<|assistant|><think>.
2. Code generation: the prompt "Write a Python function is_prime(n: int) -> bool… Reply with only the code" is sent at temperature 0. The returned function is executed and must return exactly the primes in [-5, 59].
3. Clean stop: generation must end on an end-of-generation token by itself, not hit the token limit.
4. Vision (mmproj, run once with Q4_K_M): the llama.cpp logo image is described.

Q8_0 output (temperature 0, stopped by itself after 108 tokens):

def is_prime(n: int) -> bool:
    if n < 2:
        return False
    if n < 4:
        return True
    if n % 2 == 0 or n % 3 == 0:
        return False
    i = 5
    while i * i <= n:
        if n % i == 0 or n % (i + 2) == 0:
            return False
        i += 6
    return True

Q5_K_M, Q4_K_M, Q3_K_M and Q2_K produced the same code, also stopping by itself after 108 tokens.

Vision check (Q3_K_M + mmproj, llama.cpp logo): "The image shows the text "LLaMA" in white, followed by an orange "C++" logo with llama ears."

These are functional checks only, not a quality evaluation. Neither this conversion nor the abliterated checkpoint has been benchmarked; see the original model cards.

Conversion

1. convert_hf_to_gguf.py cantina-security/apex-flash-1-abliterated --remote --outtype q8_0 --split-max-size 45G, streaming tensors from the Hub. The mmproj was made the same way with --mmproj --outtype f16.
2. --remote mode does not pick up the repo's separate chat_template.jinja. The original template was embedded afterwards with gguf_new_metadata.py --chat-template-file. Without it, llama.cpp falls back to ChatML and the model doesn't stop generating.
3. llama-quantize --allow-requantize --keep-split <Q8_0> <out> <TYPE> for each lower quant.

License

These GGUF files use the same license as the original weights: the MIT License, Copyright (c) 2026 Z.AI Co., Ltd. The full text is in LICENSE, copied unchanged from cantina-security/apex-flash-1-abliterated.

Credit for the model goes to Cantina Security and Yeta (apex-flash-1), and to Z.AI (the GLM-5.3-Flash base architecture). This repository contains only format conversions and quantizations.