---

license: apache-2.0

library_name: gguf

pipeline_tag: text-generation

base_model: EschaLabs/Qwen3.6-35B-A3B-Escha-W2

tags:

- gguf

- qwen3

- moe

- escha

- llama.cpp

language:

- en

---

Escha Qwen3.6-35B-A3B W2 GGUF

I ported EschaLabs/Qwen3.6-35B-A3B-Escha-W2 to llama.cpp. Weights are theirs. This is not a requant.

I decode their native 2-bit eschamoe code in-kernel (GGML_OP_ESCHA_MOE). Expert payloads are the same bytes as the safetensors. Non-expert tensors are fp16.

Stock llama.cpp cannot load this file. You need my fork, branch escha-w2:

https://github.com/Ajay9o9/llama.cpp-escha/tree/escha-w2

If you only download the GGUF, it will not run.

File

| | Source checkpoint | This GGUF |

|---|---:|---:|

| Size | 12.30 GB | 14.50 GB (+17.9%) |

| Expert code | native 2-bit / 3-bit | same bytes |

| Non-expert tensors | int8 | fp16 |

| Layers | 40 + MTP | 40, MTP dropped |

| Tensors | | 976 |

| VRAM, 32k context | | 14.5 GB of 24 |

File: Escha-Qwen3.6-35B-W2.gguf. 256 experts, top-8. gate/up K=2, down K=3. Extra size is fp16 on the non-expert tensors, not the expert code.

Quality vs Escha SGLang

Same GPU. I sent token ids, not text. Tokenizers agreed on 196,608 tokens of wikitext. Perplexity uses the same chunking on both sides (n_ctx 2048, 96 chunks, 98,208 scored tokens).

| Perplexity, wikitext-2 | |

|---|---:|

| Escha SGLang | 6.0988 |

| This GGUF | 6.0997 +/- 0.047 |

| Difference | +0.0009 (+0.015%) |

Gap is inside the llama.cpp run's own error bar, and smaller than SGLang's own rerun drift (+0.0016). One corpus. Not a task-accuracy claim.

| Teacher-forced, 97 positions | |

|---|---:|

| Top-1 token agreement | 93 / 97 = 95.9% |

| JS divergence over top-20 | mean 1.7e-4, max 1.0e-3 |

| \|delta logprob\| where they agree | mean 0.013, max 0.072 |

| True-token NLL | SGLang 1.39704, this GGUF 1.38663 |

Same prefix at every position, so a miss at i-1 does not poison i. Two SGLang runs against each other were 96 / 97.

| Greedy, 16 prompts, 64 tokens, temp 0 | |

|---|---:|

| Token-identical | 13 / 16 |

| Diverged | 3 / 16, at tokens 5, 26, 49 |

Two SGLang runs were also 13 / 16 identical. Greedy is a hard argmax. I would not hang a quality claim on this.

Op test vs a numpy dense-fold reference (CPU and CUDA scored separately): rel RMS 1.06e-6 at K=2, 9.26e-7 at K=3.

Speed vs Escha SGLang

Single stream, batch 1, one RTX 3090. Their runtime is still ~2.6x faster at generation. Context lengths were not matched, so the ratio is rough.

| | Escha SGLang | This GGUF |

|---|---:|---:|

| Generation (tok/s) | 110.4 | 42.6 |

| Prefill, 4,566 tokens (tok/s) | 2595 | 634 |

SGLang generation was 256 new tokens with ignore_eos. MTP is dropped here, so speculative decoding on their server is not in these numbers.

Build the fork

git clone -b escha-w2 https://github.com/Ajay9o9/llama.cpp-escha.git
cd llama.cpp-escha
cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
cmake --build build -j 12

86 is an RTX 3090. Change that for your GPU. -j 12 is a 12-core CPU.

Download the GGUF

hf download aj9o9/Qwen3.6-35B-A3B-Escha-W2-GGUF Escha-Qwen3.6-35B-W2.gguf

Direct file: https://huggingface.co/aj9o9/Qwen3.6-35B-A3B-Escha-W2-GGUF/resolve/main/Escha-Qwen3.6-35B-W2.gguf

Run it

./build/bin/llama-server \
  -m Escha-Qwen3.6-35B-W2.gguf \
  -ngl 99 --host 127.0.0.1 --port 8080 \
  -c 32768 -np 1 -t 12 --jinja

Or:

./build/bin/llama-cli \
  -m Escha-Qwen3.6-35B-W2.gguf \
  -ngl 99 --jinja -c 32768

It is a reasoning model. Give it room to think or you get an empty answer.

What this is not

• Not a Q4_K / Q8_0 requant of a dense reconstruction
• Not upstream llama.cpp
• Not bit-identical to Escha SGLang on every greedy prompt
• No MMLU / GSM8K / long-context eval

License

Apache-2.0, same as the Escha weights and the escha 1.0.2+qwen3moe wheel I used.

Source: EschaLabs/Qwen3.6-35B-A3B-Escha-W2

Runtime I compared against: escha 1.0.2+qwen3moe on SGLang