---

license: apache-2.0

base_model: jakeatx/slimder-qwen38-ream288-depth32-agentic-ngram50-compact

pipeline_tag: text-generation

tags:

• llama.cpp
• gguf
• qwen
• mixture-of-experts
• ream
• lora
• model-compression
• agentic
• speculative-decoding
• mtp

---

Qwen3.8 Perian GGUF

> Behavioral release candidate — not promoted. The checkpoint-5001448

> files are structurally corrected and load successfully, but the saved Hermes

> long-context regression still enters a planning/action loop. The same failure

> reproduced in UD-IQ4_XS with embedded MTP-3 and in UD-Q6_K_XL with MTP

> disabled, so this is a target-adapter behavior issue rather than an IQ4 or MTP

> corruption. A continuation training pass and the 1K/2K/4K/8K action-boundary

> gate are pending. Do not treat this candidate as the loop fix.

Qwen3.8 Perian is a compressed and post-trained Qwen3.8 Flash-Next lineage. It

combines depth pruning, routed-expert width reduction, and a document-disjoint,

activation-informed reduction of the PLE n-gram table with a rank-32 QLoRA.

Current checkpoint

All files in the quant folders below derive from the same merged checkpoint.

The resumable unfused source is:

• repository: jakeatx/ream-288-rank32-3m-adapter
• path: targeted-long-r2-fullctx-v2/checkpoints/5001448
• supervised tokens seen in this continuation: 5,001,448
• rendered tokens seen: 13,697,751

The next corrective pass must resume this unfused checkpoint with adapter,

optimizer, and RNG state intact. It must not restart from the fused BF16 model.

Embedded MTP by default

Every checkpoint-5001448 GGUF contains the 512-expert Qwen3.8 Flash-Next MTP

block inside the model file set. No separate MTP sidecar is required. The

included launchers enable embedded MTP depth 3 by default.

chmod +x start-mtp.sh run-mtp-server.sh
./start-mtp.sh UD-IQ4_XS

Or launch an already-downloaded model/first shard:

./run-mtp-server.sh /path/to/model-or-first-shard.gguf

The production launcher uses the fork's 245,760-token context configuration,

MTP-3, Q8 KV for the draft path, and the checked-in atx_65536.txt vocabulary

map. It does not silently force a 32K context.

The embedded shared MTP block has not yet received a separate REAP/REAM retune

against the 288-expert target trunk. That is a speed/acceptance optimization;

it is separate from the target-adapter loop described above.

Available artifacts

| Variant | Remote path | Bytes | Status |

|---|---|---:|---|

| BF16 | BF16/Qwen3.8-Perian-MTP-BF16-00001-of-00005.gguf | 155,790,065,472 total | header-corrected; embedded MTP |

| Q8_0 | Q8_0/Qwen3.8-Perian-MTP-Q8_0-00001-of-00002.gguf | 83,440,745,696 total | runtime load/generation passed |

| UD-Q6_K_XL | UD-Q6_K_XL/Qwen3.8-Perian-MTP-UD-Q6_K_XL.gguf | 83,051,625,536 | runtime load/generation passed |

| UD-Q5_K_XL | UD-Q5_K_XL/Qwen3.8-Perian-MTP-UD-Q5_K_XL.gguf | 73,391,619,136 | MTP-3 load/generation passed |

| UD-IQ4_XS | UD-IQ4_XS/Qwen3.8-Perian-MTP-UD-IQ4_XS.gguf | 47,500,793,152 | MTP-3 load passed; behavioral gate failed |

UD-Q4_K_XL and UD-Q3_K_XL remain pending. Their exact tensor recipes are

already present under recipes/checkpoint-5001448/.

Quantization policy

The dynamic tensor maps reproduce the Unsloth Qwen3.8 profile at exact-tensor

level after reducing the target trunk to 32 layers. The embedded MTP block uses

matching attention-layer grades. Its MTP-only norms remain F32 and projections

remain Q8_0. The large per_layer_token_embd.weight n-gram table is protected

at IQ4_NL for 3/4-bit variants and Q8_0 for 5-bit and higher variants.

The custom llama.cpp/llamAmpere runtime must support the qwen4exp

architecture, compact PLE remap, flat I32 ple_compact_remap.weight, and the

embedded shared-target MTP block. See GGUF_RELEASE.md for the release contract.

Known behavioral failure and promotion gate

The saved Hermes failure prompt is 21,801 tokens. At checkpoint 5001448:

• UD-IQ4_XS + embedded MTP-3 generated 4,096 tokens, repeated the planning

intent, and never issued the required tool call.

• UD-Q6_K_XL with MTP disabled reproduced the same behavior.

Repetition penalty 1.05 is not a sufficient fix because the prose changes while

the intent cycles. Promotion requires the exact saved prompt to remain loop-free

at 1K/2K/4K/8K prefixes, emit a valid Hermes XML tool call by 2K, leave no prose

after the tool call, and pass at penalties 1.0 and 1.05.