---

base_model: amazon/GPT-OSS-20B-P-EAGLE

license: apache-2.0

tags:

- gguf

- p-eagle

- eagle3

- speculative-decoding

- gpt-oss

- unsloth

- mtp

- draft

library_name: llama.cpp

pipeline_tag: text-generation

---

GPT-OSS-20B P-EAGLE — TULLUS GGUF

A TULLUS conversion of Amazon's GPT-OSS-20B-P-EAGLE draft model to GGUF for use with llama.cpp EAGLE3 speculative decoding.

This repository contains my gold-standard BF16 reference GGUF and validated Q6_K and Q4_K_M quantized variants derived from it.

> [!WARNING]

> A P-EAGLE-capable llama.cpp runtime is required. These files are draft models and do not work with a stock, unmodified llama.cpp server. Using --spec-type draft-eagle3 alone is not sufficient; the executable must include the TULLUS P-EAGLE runtime changes described below. Command-line options do not add P-EAGLE support by themselves.

Provided Draft Models

| Artifact | Format | Exact size | Notes |

|:---|:---|---:|:---|

| GPT-OSS-20B-P-EAGLE-BF16.gguf | BF16 | 3.61 GB | Highest Quality and Precision |

| GPT-OSS-20B-P-EAGLE-Q6_K.gguf | Q6_K | 1.88 GB | High Precision Very Good Quality Recommend |

| GPT-OSS-20B-P-EAGLE-Q4_K_M.gguf | Q4_K_M | 1.46 GB | Good Quality Fast Recommend |

The BF16 artifact is the conversion reference. Q6_K and Q4_K_M are the quantized variants that passed the structural and runtime validation described in this card. These are draft models, not standalone replacements for the GPT-OSS-20B target model.

---

Quick start

Requirements

You need:

• A GPT-OSS-20B target model. The end-to-end tests documented here used the Unsloth Q6 artifact gpt-oss-20b-UD-Q6_K_XL.gguf from unsloth/gpt-oss-20b-GGUF.
• One TULLUS P-EAGLE draft GGUF, such as GPT-OSS-20B-P-EAGLE-Q6_K.gguf or GPT-OSS-20B-P-EAGLE-Q4_K_M.gguf.
• A llama.cpp build containing the P-EAGLE runtime patch. The frozen and validated upstream baseline for this release is commit 8bb3cf0d216a5ae48360179f2931daf284be4d01 (short SHA 8bb3cf0d2, version 0.6.0-dev, build 11574).

Download the runtime patch bundle

WARNING -> MAKE FULL BACKUPS BEFORE YOU BEGIN -> THEN ENJOY YOUR BOOSTED T/S 🦅

> [!TIP]

> THIS PATCH MAY INDEED WORK ON NEWER LLAMA BUILDS, VERIFY ALL SOURCES BEFORE THE PATCH. NO LIVE SERVER PATCHING WAS TESTED!

The published patch folder is P-EAGLE-20B-PATCH. It is intended to contain only these three release files:

1. TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2-Patch.tar.xz
2. TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2-Patch.zip
3. SHA256SUMS.txt

Download only 1 archive and checksum file into the same directory, then verify the archive hash before extracting the patch:

sha256sum -c SHA256SUMS.txt

Extract one archive (the .tar.xz or .zip) and use the included patch and reproduction materials. Do not apply a patch to an arbitrary current llama.cpp checkout; use the exact upstream commit specified below.

Start the target and draft together

After applying the patch and building llama-server, adjust the paths to your local target and draft files. The following example uses the Q6_K draft and the settings from the documented smoke test:

TARGET="/path/to/GPT-OSS-20B-UD-Q6_K_XL.gguf"
DRAFT="/path/to/GPT-OSS-20B-P-EAGLE-Q6_K.gguf"
API_KEY="replace-with-a-local-api-key"

./build/bin/llama-server \
  --model "$TARGET" \
  --model-draft "$DRAFT" \
  --n-gpu-layers 999 \
  --ctx-size 1024 \
  --spec-type draft-eagle3 \
  --spec-draft-n-max 4 \
  --spec-draft-n-min 0.8 \
  --no-spec-draft-backend-sampling \
  --api-key "$API_KEY" \
  --cors-origins localhost \
  --host 127.0.0.1 \
  --port 18080

These speculative-window settings are the settings used for the documented smoke-test configuration, not universal recommendations. The best values depend on the hardware, target model, prompt, and desired throughput/acceptance trade-off.

A successful P-EAGLE model load reports a message similar to:

detected Amazon P-EAGLE: mask_hidden=8640, ptd_token_id=201020

Check that the server is ready:

curl -sS http://127.0.0.1:18080/health

Expected response:

{"status":"ok"}

High Quality BF16 reference artifact

File: GPT-OSS-20B-P-EAGLE-BF16.gguf

This is my gold-standard reference GGUF produced from the original Hugging Face P-EAGLE checkpoint. It is the reference artifact used during conversion and validation and from which the released quantized variants were derived. The artifact originated during the historical experimental build-branch work and is retained here as the reference model.

• Exact size: 3,613,773,664 bytes (3.61 GB / 3.37 GiB)
• SHA-256:

```text

b4b2ee2e8651d0c008ace645c2c67f91d3a97e9b9ffd35c73c9678b8066e703a

```

• Xet hash:

```text

c2f84f7e68c732e200e3194432126d35683d8c42b3940c4b72cb6a03e68ae7dc

```

HQ-standard metadata

| Field | Value |

|---|---|

| GGUF version | 3 |

| Tensor count | 44 |

| KV metadata count | 35 |

| Architecture | eagle3 |

| general.type | model |

| general.name | Draft Hf |

| Size label | 1.8B |

| File type | BF16 |

| License metadata | apache-2.0 |

The KV metadata count is 35. An earlier value of 34 omitted the ptd_token it is now correct ✅

Source and target models

Original P-EAGLE checkpoint:

amazon/GPT-OSS-20B-P-EAGLE

Target model repository used for end-to-end validation:

unsloth/gpt-oss-20b-GGUF

Tested target artifact:

gpt-oss-20b-UD-Q6_K_XL.gguf

Validation status

Validated: end-to-end speculative decoding demonstrated. All the draft releases passed the following checks in the documented patched-runtime validation:

• GGUF structural inspection
• Tensor registration and loading
• P-EAGLE model construction
• EAGLE3 encoder graph construction and execution
• EAGLE3 decoder graph construction and execution
• Batched encoder/decoder execution
• Target hidden-state routing for layers [2, 12, 21]
• Real speculative draft generation
• Real draft acceptance/rejection cycles
• Target/speculative semantic equivalence

Many deterministic target-versus-speculative inference tests were completed using the same GPT-OSS-20B target.

Functional smoke test

With target GPT-OSS-20B-UD-Q6_K_XL.gguf and draft GPT-OSS-20B-P-EAGLE-Q6_K.gguf, the patched server answered the following prompt correctly and stopped normally:

> A train travels 180 miles in 3 hours. It then increases its speed by 20% and continues traveling for another 2 hours. How many miles does the train travel in total?

Expected and returned answer: 324 miles. The recorded response reported 197 completion tokens, finish_reason: stop, and draft statistics of 135 accepted out of 244 generated draft tokens (55.33% acceptance; reported mean draft length 3.21).

• With different n-max AND n-min I also recorded 146 accepted / 165 generated acceptance = 0.88485 (about 88.49%)

This is a functional smoke test, not a broad evaluation of every prompt or workload.

Performance notes

Performance depends on the target model, hardware, prompt, context, and speculative-window settings. The figures below are reported observations, not universal speedup claims.

Single-run throughput observation

This patched server build, produced the following single-run measurements on the tested workload:

| Measurement | Observed result |

|---|---:|

| Bare Unsloth Q6 target | 139.34 tok/s |

| P-EAGLE | 180.80 tok/s |

| Speculative settings | n-max = 4, n-min = 0.8 |

| Relative measured increase | Approximately 29.8% |

Important: Using the newly built llama.cpp server from the frozen upstream baseline, with the released P-EAGLE patch applied, one run measured 180.80 generated tokens/s on the same 197-token prompt/response. This was the highest result recorded in our testing to date on this system and configuration. These are single-run observations, not a statistically controlled comparison; repeat trials and compare medians before drawing broader performance conclusions.

Separate acceptance observation and tuning notes

A separate test observed:

143 accepted / 169 generated
acceptance = 0.84615 (about 84.62%)

This is a separate operating-point observation; it should not be assumed to correspond to the throughput measurement above or to produce the highest tokens-per-second result.

On my AMD system, testing showed that higher draft acceptance could come with somewhat lower throughput, while the generated answers remained on point in those tests. I saw good inference behavior with n-max values of 2 and 4. Your best setting may differ with different hardware and workloads.

The broader benefit of P-EAGLE is its ability to predict multiple future draft positions in parallel, allowing the target model to verify useful predictions in fewer speculative steps. The practical operating point depends on the hardware, model, prompt, and speculative-window settings; maximum acceptance does not necessarily mean maximum throughput.

P-EAGLE architecture and model contract

The converted GGUF follows this runtime-facing contract:

general.architecture           = eagle3
eagle3.block_count              = 4
eagle3.target_hidden_size        = 2880
eagle3.target_layers             = [2, 12, 21]
eagle3.embedding_length          = 2880
eagle3.feed_forward_length       = 14336
eagle3.attention.head_count      = 64
eagle3.attention.head_count_kv   = 8
eagle3.attention.key_length      = 64
eagle3.attention.value_length    = 64
eagle3.rope.dimension_count      = 64
eagle3.rope.freq_base            = 500000
eagle3.context_length            = 131072
eagle3.vocab_size                = 201088
eagle3.norm_before_fc            = false
eagle3.norm_before_residual      = false
ptd_token_id                     = 201020

The three target hidden states are concatenated before being supplied to the P-EAGLE encoder:

3 × 2880 = 8640

The learned mask_hidden tensor contains 8640 F32 elements (34,560 bytes). This is the encoder input width and an actual model tensor required by the P-EAGLE runtime.

Post-quantization audit

PASS  Q4_K_M architecture = eagle3
PASS  Q4_K_M ptd_token_id = 201020 UINT32
PASS  Q4_K_M tensor count = 44
PASS  Q4_K_M mask_hidden = 8640 F32 elements / 34560 bytes
PASS  Q4_K_M general.file_type = 15
PASS  Q4_K_M size = 1,464,936,512 bytes

PASS  Q6_K architecture = eagle3
PASS  Q6_K ptd_token_id = 201020 UINT32
PASS  Q6_K tensor count = 44
PASS  Q6_K mask_hidden = 8640 F32 elements / 34560 bytes
PASS  Q6_K general.file_type = 18
PASS  Q6_K size = 1,875,301,952 bytes

PASS :: Q4_K_M + Q6_K COMPLETE

The post-quantization audit confirms that both quantized variants retain the P-EAGLE architecture metadata, ptd_token_id, the 44-tensor structure, and the 8640-element F32 mask_hidden tensor.

Why stock llama.cpp does not work

Upstream llama.cpp supports standard EAGLE3, but the stock EAGLE3 implementation assumes a single decoder layer. Its implementation contains the one-layer assertion:

GGML_ASSERT(n_layer == 1); // eagle3 has only one decoder layer

The Amazon GPT-OSS P-EAGLE draft instead contains four decoder layers and additional P-EAGLE-specific state. It requires the learned mask_hidden tensor and ptd_token_id. The stock speculative runtime does not implement the P-EAGLE use of these components, so this draft requires the TULLUS runtime patch.

What the TULLUS runtime patch changes

The P-EAGLE runtime work has four main functional areas and modifies seven files in total.

1. Four-layer P-EAGLE tensor loader

File: src/models/eagle3.cpp

The stock single-layer decoder loader is adapted to the four-layer GPT-OSS P-EAGLE decoder contract.

Decoder block 0:

Q/K/V input width = 2 × 2880
attn_norm_2       = present

Decoder blocks 1–3:

Q/K/V input width = 2880
attn_norm_2       = absent

The resulting structure is:

block 0:
    concat(norm(token_embedding), norm(g_embedding))
    -> Q/K/V with 5760-wide input

block 1..3:
    previous decoder hidden state
    -> Q/K/V with 2880-wide input

2. Expose mask_hidden

Files: src/llama-ext.h, src/llama-model.cpp, and src/llama-model.h

A model-accessor API exposes the learned mask_hidden tensor to the speculative runtime. The accessor verifies that the tensor is F32 and exposes its element count and contents. For this model, the tensor contains 8640 F32 values.

3. Detect P-EAGLE and read ptd_token_id

Files: common/speculative.cpp, src/llama-arch.cpp, and src/llama-arch.h

The speculative runtime detects P-EAGLE from the presence of the learned mask_hidden tensor and reads and validates:

ptd_token_id = 201020

The value is supplied through GGUF metadata because the runtime-required value was not fully represented in the original GGUF conversion.

4. Implement the P-EAGLE parallel-drafting path

File: common/speculative.cpp

The runtime uses the learned mask_hidden to construct masked future positions. The flow is:

target model
    │
    ├── target hidden states [layers 2, 12, 21]
    │
    ▼
P-EAGLE encoder
    │
    └── g embedding, width 2880

For parallel drafting, the runtime constructs one decoder batch containing:

row 0:
    real target-derived token
    real target-derived g embedding

rows 1..K-1:
    ptd_token_id = 201020
    learned mask embedding

The learned mask_hidden is projected through the existing encoder once, and the resulting 2880-wide embedding is reused for masked future positions. The four-layer P-EAGLE decoder is then executed with one decoder call for the requested positions, and the resulting logits rows are sampled as the draft sequence.

This is the key difference from ordinary autoregressive EAGLE3 drafting: P-EAGLE predicts multiple future draft positions in parallel rather than generating them one at a time.

Frozen llama.cpp baseline and patch identity

The release patch was developed and validated against the following exact upstream revision:

• Repository: ggml-org/llama.cpp
• Frozen upstream commit: 8bb3cf0d216a5ae48360179f2931daf284be4d01
• Abbreviated commit: 8bb3cf0d2
• llama.cpp version: 0.6.0-dev
• Build: 11574

The seven modified files are:

common/speculative.cpp
src/llama-arch.cpp
src/llama-arch.h
src/llama-ext.h
src/llama-model.cpp
src/llama-model.h
src/models/eagle3.cpp

Patch filename inside the release materials:

TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2.patch

Patch SHA-256:

db84efbe6ee0d351425a2dc81469f7adaa74d745179b0448be32810de432bc6b

The quantization tools used for the released GGUFs were llama.cpp 0.6.0-dev, commit 8bb3cf0.

Reproduce the patched runtime

Back up your working tree and build environment before applying source changes. The instructions below describe the frozen release baseline; they are not instructions to patch an arbitrary current upstream checkout.

• this patch may work on upstream builds for a while, until they change something in the specific patched files. Be sure to verify all your sources and also live server patching was Not Tested nor is it Recommended.

1. Clone llama.cpp if needed, then check out the exact upstream revision:

```bash

git clone https://github.com/ggml-org/llama.cpp.git

cd llama.cpp

git checkout 8bb3cf0d216a5ae48360179f2931daf284be4d01

git rev-parse HEAD

```

Confirm that git rev-parse HEAD prints:

```text

8bb3cf0d216a5ae48360179f2931daf284be4d01

```

1. Extract one of the patch release archives and use the included patch file, TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2.patch. From the llama.cpp repository root, run:

```bash

git apply --check /path/to/TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2.patch

git apply /path/to/TULLUS-GPT-OSS-20B-P-EAGLE-upstream-8bb3cf0d2.patch

git diff --check

```

1. Build llama-server with the GPU backend appropriate for your system. The documented TULLUS validation build used the Linux x86_64 HIP backend. For the tested ROCm 7.2.4 / gfx1201 setup, consult the PATCH-REPRODUCTION.md or archive README included in the release materials for the CMake options.

1. Start the target and draft together using the quick-start command above. The documented configuration used --ctx-size 1024, --spec-type draft-eagle3, --spec-draft-n-max 4, --spec-draft-n-min 0.8, and --no-spec-draft-backend-sampling.

1. Confirm that the server logs identify the Amazon P-EAGLE state (mask_hidden=8640, ptd_token_id=201020), then verify readiness using the /health endpoint.

The essential requirement is that the executable contains the P-EAGLE runtime changes. The command-line options alone do not supply P-EAGLE support.

Upstream MoE-cache compatibility and observations

The frozen upstream revision retains the native llama.cpp MoE-cache implementation. The seven-file P-EAGLE patch does not itself add the MoE cache; it adapts the runtime for this P-EAGLE draft architecture.

I tested the patched server with and without the MoE cache, using both target and draft models. With a 2048 MiB CPU-cache budget and experts placed on the CPU, throughput on my PC fell to approximately 21–25 tokens/s. This was not an optimized cache-performance benchmark: CPU expert placement is a trade-off intended for workloads where models or expert weights exceed available GPU memory. The observation demonstrates that I exercised this combination, but it does not establish general cache performance.

The documented P-EAGLE smoke test does not quantify cache-hit rates, memory savings, or cache-specific performance. Those require separate workload-specific tests. The MoE cache is an upstream capability, not a feature implemented by the seven-file P-EAGLE patch.

Artifact relationship

amazon/GPT-OSS-20B-P-EAGLE
            │
            ▼
   Gold-standard BF16 GGUF
  3,613,773,664 bytes / 44 tensors
            │
            ▼
    Q6_K / Q4_K_M GGUFs
            validated
            │
            ▼
 llama.cpp EAGLE3 + P-EAGLE
       patched runtime
            │
            ▼
      GPT-OSS-20B target

The BF16 GGUF remains the reference artifact, while Q4_K_M and Q6_K are the validated quantized variants intended for practical inference.

Compatibility and limitations

Validation demonstrates functional llama.cpp EAGLE3 speculative decoding with the tested GPT-OSS-20B target, the TULLUS P-EAGLE GGUFs, and a patched llama.cpp runtime based on the frozen upstream revision documented here.

• These files are draft models only. They are not substitutes for the GPT-OSS-20B target model.
• A stock, unmodified llama.cpp runtime is not sufficient; the required P-EAGLE runtime changes must be present in the executable.
• The documented throughput numbers are individual observations from a specific setup, not statistically controlled or universal performance guarantees.
• Speculative-window settings should be tuned for the particular GPU/CPU, model, prompt, and workload.
• MoE-cache behavior and performance depend on the workload and cache configuration; this card does not claim a comprehensive cache benchmark.

For a reproducible starting point, use the published patch bundle and the exact upstream commit 8bb3cf0d216a5ae48360179f2931daf284be4d01.

---

License and Acknowledgements

The upstream repository identifies the model as licensed under Apache-2.0. Please consult the original Amazon P-EAGLE model repository and its dataset references for provenance and applicable terms.

TULLUS does not claim authorship of the upstream architecture, training data, or learned capabilities.

A huge thank-you to Amazon for P-EAGLE OpenAI for GPT-OSS-20B Unsloth for the UD-Q6_K_XL.gguf target artifact, and AMD for making the gfx1201 hardware platform part of this adventure.

And, of course, thanks to my amazing assistant, The Doc, for all the help along the way!

We had some UP's, we had some downs, and we had some ALL AROUNDS with this wonderfully fun Edge Case project. Through it all, we meticulously verified the conversion against the original checkpoint and produced a high-quality BF16 GGUF reference release, backed by tensor-equivalence and forward-logit checks.

The TULLUS Edge Case Equation

$$

\boxed{

\underbrace{\text{UP's}+\text{downs}+\text{ALL AROUNDS}}_{\text{The Journey}}

\;\xrightarrow{\text{Persistence + Verification}}\;

\text{THE GOODS 🦅}

}

$$

TULLUS BAKES 🍪

Fresh From The AI Ovens, It's ^^