---

license: apache-2.0

base_model: yandex/AliceAI-T5-35B-A0.6B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• encoder-decoder
• mixture-of-experts
• quantized
• text-generation

---

AliceAI-T5-35B-A0.6B GGUF

Community GGUF quantizations of yandex/AliceAI-T5-35B-A0.6B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About AliceAI-T5-35B-A0.6B

AliceAI-T5-35B-A0.6B is a custom Transformers encoder-decoder mixture-of-experts

model from Yandex. The upstream configuration describes approximately 34.35B

unique parameters, 512 experts per MoE layer with top-8 routing, a 1,536-wide

hidden state, shared encoder/decoder embeddings, and RoPE with YaRN for a

context advertised up to 128K tokens. See the official model card

for the upstream benchmark claims and usage requirements.

This repository is a release draft. GGUF files will be listed here only after

the custom encoder-decoder converter, runtime graph, BF16 parity test and

generation smoke tests pass.

Current status: the upstream source is fully staged and SHA256-verified;

AliceAI-T5 MoE GGUF support is under implementation. No GGUF artifact has been

published yet.

Quantization scope

No training or fine-tuning is performed. The planned artifacts will be created

directly from the locked upstream BF16 checkpoint, using an AliceAI-specific

calibration/imatrix and the supported GGUF quantization ladder. No quantized

file will be used as the source for another quantized file.

Fidelity measurements

This section will be added after a fixed, held-out evaluation against the BF16

reference. It will report Mean KLD, Top-1 agreement, delta PPL and RMS delta-p

for every published file, together with the corpus, context, runtime and

machine-readable summary. These are next-token fidelity measurements; task

behavior may vary by workload.

Quick start

The final command will be added with the validated encoder-decoder runtime and

published filename. Do not use a decoder-only quickstart until the GGUF graph

has been verified to execute AliceAI's encoder, cross-attention and decoder

generation path.

Reproducibility and validation

The final release will lock upstream revision

a0d71f58c40d6affe461797b30b35ff47f52a5f2, source BF16 shard hashes, converter

and runtime revisions, the model-specific calibration/imatrix inputs, exact

quantization commands, artifact SHA256 values and validation results in

reproducibility/manifest.md.

License and attribution

The upstream model is released under the Apache License 2.0; the upstream

license will be included with the final package. These are community GGUF

quantizations, not an official Yandex release or endorsement.