---

license: apache-2.0

base_model: xero0000/Qwen3.8-27B-Palimpsest

base_model_relation: quantized

quantized_by: xero0000

library_name: gguf

pipeline_tag: text-generation

tags:

- qwen3.8

- qwen3.5

- gguf

- creative-writing

- tool-use

- long-context

- mtp

language:

- en

---

Qwen3.8-27B-Palimpsest-GGUF

GGUF releases of Qwen3.8-27B-Palimpsest, an experimental Qwen3.8-27B

fine-tune for literary prose, continuity, structured tool use, and

position-aware long-context behavior.

The flagship MIX-IQ3KT build is importance-calibrated and stays below the

project's strict 12,000,000,000-byte language-model target. Conventional

K-quant tiers are provided as controls and for broader runtime compatibility.

The multimodal projector is not included in the language-model size budget.

> Release status: evaluation in progress. Capability, MTP-equivalence,

> maximum-usable-context, and Forge gates are still running. Results below are

> local measurements with raw evidence; missing results are not treated as

> zero or silently inferred from the base model.

Fine-tune summary

Palimpsest merges two small LoRA stages into the original BF16 weights:

1. a 1,152-example behavior/prose/tool mixture, selected at step 64; and
2. an 80-step PoSE-style long-context stage updating Q/K LoRA modules only in

the 16 full-attention layers, with virtual positions through 1M.

The full training recipe, source revisions, local development gates, and BF16

usage are documented in

xero0000/Qwen3.8-27B-Palimpsest.

Quant files

| File | Exact bytes | Reported BPW | SHA-256 | Status |

|---|---:|---:|---|---|

| Qwen3.8-27B-Palimpsest-MIX-IQ3KT.gguf | 11,702,780,800 | 3.424 effective | 2439bd1770cb4663ecf6104a92ec84445b18029ae6ea74a7df2f1909e96be1e3 | Flagship; load and frozen screen passed |

| Qwen3.8-27B-Palimpsest-Q4_K_M.gguf | 16,492,948,352 | 4.826 | 2620a1677dac681ff18993daec9cc23eac9d06b946386e4a45ece19fc63ee27a | Load passed; quality pending |

| Qwen3.8-27B-Palimpsest-Q5_K_M.gguf | 19,374,156,672 | 5.670 | 21c3a436f059606fb2f18d644339daf534df82b3e1a4eca523d335b4ccc0d09c | Load passed; quality and controlled speed pending |

| Qwen3.8-27B-Palimpsest-Q6_K.gguf | 22,431,001,472 | pending runtime report | feac332d6bbf1636708737c936ad22651e024ec6225d713a6c06f65b1afd61e6 | GGUF structure/hash passed; runtime, quality, and speed pending |

The mixed file contains 866 tensors: 360 F32, 1 Q5_K, 64 IQ4_NL, 25 IQ3_S,

176 IQ3_KT, 144 IQ4_KT, and 96 IQ3_KS. It was quantized directly from the

merged BF16 GGUF, never requantized from another lossy file.

Its importance matrix contains 433 entries from 256 chunks / 65,536 tokens

balanced across tool use, prose, general text, and virtual-position examples.

Calibration corpus SHA-256:

091f33bb6fb62d46850b3237cdc74b557344fe764adc53b31caab98808083bf1.

Importance-matrix SHA-256:

1d6bb715ad4a36e756df9b30d059742ec809c50435107e6c2812408e3d154e6d.

Local quality screen

This deterministic 8+8 engineering screen is not a reproduction of the

published Qwen model-card benchmark suite.

| Variant | PACT-Q retrieval/tool cases | GPQA-Diamond subset | Errors |

|---|---:|---:|---:|

| Local upstream Q6_K control | 8/8 | 5/8 | 0 |

| Palimpsest MIX-IQ3KT | 8/8 | 5/8 | 0 |

The three GPQA failures in both variants reached the output cap without the

required final answer; none were server errors. Larger public benchmark runs

and uncertainty-aware comparisons remain pending.

Controlled speed

Measured with expert-streaming build 516a0312 (build 4814), an RTX 3060 Ti

8 GB + RTX 2080 SUPER 8 GB + RTX 3080 10 GB, all 66 layers on GPU, 8/8/10

layer split, 24 threads, flash attention, batch 512, micro-batch 256, Q4_0 K/V

cache, and three repetitions.

| Quant | PP512 tok/s | PP4096 tok/s | TG128 tok/s |

|---|---:|---:|---:|

| MIX-IQ3KT | 607.49 ± 71.79 | 642.81 ± 1.34 | 24.48 ± 0.26 |

| Q4_K_M | 617.21 ± 50.22 | 630.04 ± 5.53 | 26.95 ± 0.07 |

Q4_K_M is 4,790,167,552 bytes larger and decoded about 10.1% faster in the

paired TG128 run. MIX-IQ3KT prefills 4096 tokens about 2.0% faster while

remaining under 12 GB.

Measured cold-load/generation checks from the HDD were 19.15 seconds for

Q4_K_M and 143.52 seconds for Q5_K_M. Storage cache state and quant kernels can

strongly affect load time, so these are rig-specific measurements rather than

universal expectations.

Tuned 512K allocation

The same three-GPU rig was also tuned with a 524,288-token allocation, static

YaRN factor 2, CPU-resident Q4 KV, and a fixed 9,011-token prompt plus 128-token

generation. This is a deployment-speed comparison at a 512K allocation, not a

claim that a fully populated 512K prompt has passed the retrieval gate.

| 512K profile | Prompt tok/s | Decode tok/s | Peak VRAM MiB (3060Ti / 2080S / 3080) |

|---|---:|---:|---:|

| Original: split 8/8/10, ubatch 256, 24 threads | 316.52 | 2.44 | 5,110 / 4,296 / 5,516 |

| Tuned: split 10/4/12, ubatch 512, 8 threads, native MTP | 467.11 | 4.49 | 6,143 / 2,984 / 7,784 |

The tuned profile improved prompt throughput by 47.6% and decode throughput by

84.0%. MTP accepted 54 of 72 proposed tokens (75%) in its isolated controlled

run. The 10/4/12 split deliberately reduces work on the RTX 2080 SUPER attached

through the rig's PCIe x1 extender.

Runtime example

The calibrated mixed quant requires the included expert-streaming/IQK trellis

runtime family. A tested native-context profile is:

llama-server \
  -m Qwen3.8-27B-Palimpsest-MIX-IQ3KT.gguf \
  --ctx-size 262144 --parallel 1 \
  --n-gpu-layers 99 --split-mode layer --tensor-split 8,8,10 --main-gpu 2 \
  --batch-size 512 --ubatch-size 256 --threads 24 --threads-batch 24 \
  --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0 --jinja

For the tested 8 GB + 8 GB + 10 GB topology, the speed-tuned 512K profile adds:

--ctx-size 524288 --tensor-split 10,4,12 \
--batch-size 2048 --ubatch-size 512 --threads 8 --threads-batch 8 \
--cache-type-k q4_0 --cache-type-v q4_0 --no-kv-offload \
--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 262144 \
--spec-type mtp:n_max=1,p_min=0.0 --ctx-size-draft 8192

Adjust the split for different cards. The conventional K-quants are intended

to have broader llama.cpp-family compatibility, but Qwen3.8 architecture and

MTP support still require a sufficiently recent build.

Context length

The underlying model is 262,144 tokens native. Do not use a 32,768-token

YaRN origin with this model. For targets above 262,144, use a scale equal to

target / 262144:

| Target | RoPE profile |

|---:|---|

| ≤262,144 | Native; no static YaRN override |

| 524,288 | --rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 262144 |

| 786,432 | --rope-scaling yarn --rope-scale 3 --yarn-orig-ctx 262144 |

| 1,048,576 | --rope-scaling yarn --rope-scale 4 --yarn-orig-ctx 262144 |

Static YaRN can reduce short-context quality, so enable it only for the target

window you need. Measured 262K peaks leave insufficient aggregate headroom to

double the Q4 KV cache safely, so this local 26 GB aggregate-VRAM rig uses

CPU-resident Q4 KV (--no-kv-offload) beginning at 512K. A successful

allocation alone is not evidence of usable context.

The release gate measures five-needle retrieval at approximately 1%, 25%,

50%, 75%, and 99%, exact instruction retention, prefill/decode latency, process

RAM, and VRAM. The final card will report both the hard allocation maximum and

the maximum window that passes the strict quality contract. It also reports a

batch-usable maximum (strict quality, prefill no longer than 30 minutes, decode

at least 5 tok/s) and an interactive-recommended maximum (strict quality,

prefill no longer than 10 minutes, decode at least 10 tok/s). Peak process swap

is recorded so a nominal pass cannot hide storage thrashing.

Measured native-context qualification

The flagship mixed quant passed all five retrieval positions and the exact

one-line output contract at every native tier with context shifting disabled.

| Context | Prompt tokens | Needles | Prefill | Prompt tok/s | Decode tok/s | Result |

|---:|---:|---:|---:|---:|---:|---|

| 32,768 | 32,234 | 5/5 exact | 61.98 s | 520.27 | 17.89 | Interactive + batch |

| 65,536 | 64,988 | 5/5 exact | 148.04 s | 439.05 | 14.63 | Interactive + batch |

| 131,072 | 130,527 | 5/5 exact | 394.14 s | 331.20 | 10.47 | Interactive + batch |

| 262,144 | 261,608 | 5/5 exact | 1,165.29 s | 224.51 | 6.70 | Batch |

No model-process swap was observed. The proven interactive recommendation is

131,072 tokens; the proven native strict/batch maximum is 262,144 tokens.

YaRN-scaled qualification remains in progress and is not inferred from these

native results.

MTP / speculative decoding

The model retains one NextN/MTP layer. Initial single-run measurements on the

flagship quant observed 24.62 target-only tok/s versus 31.96 tok/s with

--spec-type mtp:n_max=1,p_min=0.0 (+29.81%, 112/142 drafted tokens accepted).

The generated texts later diverged, so MTP remains opt-in until the repeated

HTML workload and quality-equivalence gates finish.

Limitations

• Public capability and coding/agent benchmark results are not complete.
• Fully materialized long-context accuracy through 1M is still under test;

the fine-tuning gate used short physical sequences with virtual positions.

• Vision weights were retained in BF16 training but were not fine-tuned or

re-evaluated. A compatible projector is separate from these language GGUFs.

• English dominates the fine-tuning mixture.
• Generated facts, code, tool arguments, and safety-sensitive output require

independent verification.

License

Apache-2.0, following the upstream Qwen3.8-27B release. Review upstream and

dataset licenses before redistribution or commercial use.