---

license: apache-2.0

language:

- en

base_model: Qwen/Qwen3-30B-A3B-Instruct-2507

base_model_relation: quantized

pipeline_tag: text-generation

tags:

- gguf

- qwen3_moe

- mixture-of-experts

- llama.cpp

- conversational

- mobile

- on-device

- graft

- ultralabs

---

GRAFT-10B-A2B Mobile

GRAFT-10B-A2B Mobile is the phone-sized edition of

GRAFT-10B-A2B, a ~10B-parameter mixture-of-experts chat

model from UltraLabs. It packs the whole 10B model into a 3.2 GB file, so it runs entirely on a phone.

• Made by: UltraLabs (SmallAICreator)
• File: GRAFT-10B-A2B-Mobile.gguf (3.21 GB)
• Architecture: Qwen3-MoE, 48 layers, 40 experts, 4 active per token, hidden size 2048
• Chat template: ChatML with tool calling (embedded in the GGUF)
• Context: up to 32K (2K-4K recommended on phones)
• License: Apache 2.0

---

What makes it mobile

| | GRAFT-10B-A2B (Q4_K_M) | GRAFT-10B-A2B Mobile |

| --- | --- | --- |

| File size | 6.27 GB | 3.21 GB |

| Active experts per token | 5 | 4 |

| Expert weights | Q4_K / Q6_K | IQ2_XXS / IQ2_XS (importance-matrix calibrated) |

| Attention + embeddings | Q4_K / Q6_K | Q4_0 (fast on ARM CPUs) |

• Experts squeezed, the core kept sharp. The 40 experts are about 88% of the weights, but only a few run for any

token. They are stored at ~2 bits, calibrated with an importance matrix so the weights that matter most keep the most

precision. Attention and embeddings, which every token uses, stay at 4 bits.

• Q4_0 where speed matters. llama.cpp repacks Q4_0 into fast ARM-optimised layouts at load time.
• 4 experts per token instead of 5 for less work per token.

Quick start

On a phone

Use any llama.cpp-based app, for example L-AI or PocketPal.

Download GRAFT-10B-A2B-Mobile.gguf, load it, and:

• close other apps first: it needs about 3.5 GB of free RAM (phones with 8 GB+ RAM are the most comfortable)
• set the context to 2K-4K
• use 4 threads on phones with 2+2+4 core layouts

On a Pixel 6a (6 GB RAM) it runs at about 3 tokens/s.

llama.cpp

llama-cli -hf SmallAICreator/GRAFT-10B-A2B-Mobile-GGUF -cnv -c 4096 \
  --temp 0.7 --top-p 0.8 --top-k 20 --repeat-penalty 1.05

Recommended settings

| Setting | Value |

| --- | --- |

| Temperature | 0.7 |

| Top-p | 0.8 |

| Top-k | 20 |

| Repeat penalty | 1.05 |

Use a system prompt. Without one, the model sometimes introduces itself with its base model's name:

You are GRAFT-10B-A2B, a helpful AI assistant made by UltraLabs.

Want a fun chat partner? Add something like "Be lively, playful and witty."

🎣 Go Fish Showdown: GRAFT-10B vs GPT-OSS-20B

GRAFT-10B played a full game of Go Fish against GPT-OSS-20B and won 9 books to 4. (The game was played with the full-size

GRAFT-10B-A2B Q4_K_M; this Mobile file is the same model, compressed for phones.)

| | GRAFT-10B | GPT-OSS-20B |

| --- | --- | --- |

| Parameters | 10B | 20B |

| Reasoning | ❌ None, one pass per move | ✅ Reasoning enabled |

| Books won | 9 | 4 |

How it played out:

• The game was tied 4–4 at the midpoint. From there GRAFT won every remaining book.
• GRAFT started weak, asking for Jacks over and over. It then switched strategy mid-game and began asking for the ranks it already held three of, all without any chain-of-thought.
• GPT-OSS played a reasonable strategy but got stuck repeatedly asking for a 7 that was still in the pond.

Caveat: This was a single game, and GRAFT was dealt the stronger starting hand, so luck played a part. A larger batch of games with mirrored deals (each shuffle played twice with the hands swapped) is planned to separate skill from luck.

> Half the parameters. No reasoning. Still won. 🐟

Which file should I use?

• Phone or a PC with 8 GB of RAM or less: this Mobile file.
• PC with 16 GB of RAM: the full GRAFT-10B-A2B Q4_K_M

for the best quality.

Credits

• Base model: Qwen3-30B-A3B-Instruct-2507 by the

Qwen team (Apache 2.0).

• Shrinking with GRAFT, healing, chat-tuning and the mobile build: UltraLabs

(SmallAICreator).

• Runs on llama.cpp.

Also from UltraLabs: GRAFT-10B-A2B,

GRAFT-1B and the

L-AI Android app.