---

base_model: amd/Instella-MoE-16B-A3B-Think

datasets: [glaiveai/glaive-function-calling-v2, NousResearch/hermes-function-calling-v1]

license: other

license_name: researchrail

pipeline_tag: text-generation

library_name: gguf

base_model_relation: quantized

tags:

- gguf

- rocmfp4

- llama.cpp

- moe

- reasoning

- instella

- amd

- rocm

- strix-halo

- gfx1151

- tool-calling

- function-calling

- agentic

- lora

- ryzen-ai-max

---

Instella with Tool Calling — ROCmFP4 STRIX GGUF — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

AMD shipped a reasoning model that cannot call a tool. We taught it how.

This is amd/Instella-MoE-16B-A3B-Think

— AMD's fully-open MoE (16B total / ~2.8B active) — fine-tuned for **function calling and

agentic tool use, then quantized to ROCmFP4 for AMD Strix Halo (gfx1151)**.

To our knowledge this is the first tool-calling-capable Instella. AMD never trained or evaluated

the model for it: the card doesn't mention tools, and the eval suite (WinoGrande, HumanEval+, IFEval,

HELMET, RULER) contains no agentic or function-calling benchmark at all.

| | value |

|---|---|

| file | Instella-ToolCall-v1-STRIX-imat.gguf |

| size | 8,520,937,056 bytes (7.94 GiB) |

| sha256 | 4bb231bd1e135d7b45e9874372a9b439a82fcb19f97e5f1e124b97a852e2033d |

| quant type | 105 Q4_0_ROCMFP4_STRIX — Strix Halo attn-K/V quality recipe, imatrix-calibrated |

| decode | ~90 tok/s (3-run median, Ryzen AI MAX+ 395, -ngl 999 -dio) |

| base | amd/Instella-MoE-16B-A3B-Think |

---

What actually changed

| metric (held-out 200, stratified) | base | tuned |

|---|---:|---:|

| decision — call vs. don't call | 68.0% | 97.5% |

| function name correct | 58.6% | 98.2% |

| arguments correct | 49.5% | 89.2% |

| unterminated <think> | 121 / 200 | 0 |

Per capability the tuned model gets 19/19 parallel calls, 8/8 single calls, and makes only

2 false calls in 61 no-call cases.

🔍 The bug was hiding behind a good-looking number

The base model scores 85.2% on "correctly did not call a tool" — which looks like judgement.

It isn't. 52 of those 61 correct-looking rows had an unterminated <think> block — exactly 85.2%.

The model wasn't declining. It was thinking forever and never acting. Here it is reasoning perfectly

and then doing nothing:

> <think> We are given a request to book a flight… the available tools do not include any booking

> functionality. The only tool we have is generate_random_color, which is unrela…

and on a case where it should have called:

> <think> …we have a function search_recipes that takes ingredients… we can use it. However, note

> that t…

It concludes correctly, then never emits the call. The metric was rewarding a bug.

That single defect wears two faces:

• thinking on → never terminates, never acts
• thinking suppressed (needed to get the format right) → no reasoning phase, so it fires blindly:

"What is 2+2?" → get_weather{"city": "New York"}

The tuned model now declines for reasons:

> *"I'm sorry, but I'm unable to assist with that. My current capabilities are limited to calculating

> loan payments. I don't have the ability to book flights."*

---

How it was trained

LoRA, rank 64, on the Gated-MLA projections — q_proj, o_proj, kv_a_proj_with_mqa,

kv_b_proj — 25.7 M trainable params (0.16% of 15.9 B). 2 epochs, lr 1e-4 cosine, bf16,

gradient checkpointing, length-grouped batching, max-len 4096. Loss 0.56 → 0.13. ~6 h 40 m on a

single NVIDIA GB10.

Data — 13,171 examples, Apache-2.0 only, from

glaiveai/glaive-function-calling-v2

and NousResearch/hermes-function-calling-v1.

CC-BY-NC sources were deliberately excluded to keep the licence chain clean.

| capability | examples |

|---|---:|

| multi-turn (tool result → answer) | 7,789 |

| no_call — tools offered, prose is correct | 3,981 |

| parallel calls | 1,074 |

| single call | 328 |

Three decisions did most of the work:

1. Loss on assistant turns only. A {% generation %}-marked training template plus

return_assistant_tokens_mask means the model is never trained to reproduce the tool schemas sitting

in its own prompt. Verified by decoding the mask: 460 tokens in, 52 supervised.

1. Negative examples are the whole point. 30% of the corpus is *tools offered, correct answer is

prose*. Judgement is a learned behaviour — you cannot prompt it in.

1. Truncation was checked per capability, not globally. At max-len 1536 we were silently dropping

37.5% of parallel-call examples — the scarcest capability in the mix. 4096 keeps 100%.

⚠️ What we tried that made it worse

A second pass on a 50% negative mix improved every held-out metric — decision 97.5% → 98.0%,

no_call 96.7% → 98.4% — and regressed on an out-of-distribution probe, newly failing

"What is 2+2?", which v1 gets right.

More data of the same shape ≠ better generalisation. It learned Glaive's refusal phrasing harder

without learning the principle. We shipped v1. In-distribution eval alone would have shipped the

worse model — keep an adversarial probe that disagrees with your training distribution.

Known limitations (measured, not hedged)

On a 12-question adversarial sweep — general-knowledge questions with one irrelevant tool offered —

it makes 2 false calls:

• "What is 17*23?" → calls the tool
• "Translate 'hello' to Spanish." → calls the tool

Both resemble tool-able operations, and Glaive is dense with calculate_ / translate_ tools, so

it learned "this task type is tool-able" without fully learning "is this tool relevant."

Everything else answers correctly: 2+2, 100÷4, √144, leap year, Hamlet, boiling point, photosynthesis,

WW2, capital of France, largest planet.

It is also a 2.8 B-active model. It is a fast, decisive tool-caller — not a frontier coder.

---

⛔ Requirements — read before downloading

1. A ROCmFPX runtime. ROCmFP4 uses ggml tensor types 100–106; stock llama.cpp refuses the file:

gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 101. should be in [0, 43)

Build ROCmFPX — both llama-quantize and llama-server.

2. The instella architecture. Not in llama.cpp mainline. Instella declares

model_type: deepseek_v3 and reuses DeepSeek MLA shapes, but it is not a drop-in DeepSeek-V3 —

FarSkip-Collective needs dual residual streams, and a naive port **loads fine and emits

fluent-but-wrong text. Plus Gated MLA**, rope_interleave, qk_layernorm.

3. instella-tools-tuned.jinja (included) is mandatory for tool calling. Instella inherits

DeepSeek's chat template verbatim, which has no tools variable — it can format a tool call from

history but can never advertise available tools, so the model never learns what it may call. This is a

known, unresolved upstream DeepSeek issue

(DeepSeek-V3 #48). Our template keeps

Instella's trained turn markers exactly and adds a real <tools> block.

4. --reasoning on is required. --reasoning defaults to auto, which resolves to off for a

custom template and will silently disable thinking on a Think model.

Serving

llama-server -m Instella-ToolCall-v1-STRIX-imat.gguf \
  --host 0.0.0.0 --port 8087 \
  -ngl 999 -dio --no-warmup --jinja \
  --chat-template-file instella-tools-tuned.jinja \
  --reasoning on --reasoning-format deepseek \
  -c 32768 --parallel 1 --alias instella-toolcall-v1

Standard OpenAI tools / tool_choice; calls come back in message.tool_calls, reasoning in

message.reasoning_content.

Also included

• instella-tools-tuned.jinja — the serving template (required)
• adapter/ — the LoRA adapter (108 MB) if you'd rather merge it yourself
• llamacpp-instella-toolcall.patch — two upstreamable common/chat-diff-analyzer.cpp fixes:

registering the legacy-DeepSeek tool markers as preserved tokens (without which llama.cpp hard-400s

any request carrying tools), and a </tool_call> stop guard for untuned checkpoints. The first also

fixes DeepSeek-R1, R1-Distill and pre-DSML V3.

Credits

AMD for a genuinely open model — weights,

data mixtures and training code. Glaive

and Nous Research for

Apache-2.0 function-calling data. ROCmFPX for the ROCmFP4

quantizer.

Base weights remain under AMD's ResearchRAIL licence — research use only. The LoRA and template are

ours; the training data is Apache-2.0.

<!-- PEER-TABLE:START -->

Other public builds of this model

Compiled from Hugging Face repository metadata — file sizes, shipped files, quant variant as named by each repo. No third-party build was run or benchmarked here, so this table makes no speed or quality claim about any of them. It is here so you can see the size and format options at a glance and pick what fits your hardware.

| Repository | Largest model file | Variant | Ships | Downloads | Likes |

| --- | ---: | --- | --- | ---: | ---: |

| kingjones777/Instella-MoE-16B-A3B-Think-ROCmFP4-STRIX-GGUF | 7.94 GiB | STRIX | single model file | 12 | 0 |

| kingjones777/Instella-ToolCall-16B-A3B-ROCmFP4-STRIX-GGUF (this repo) | 7.94 GiB | STRIX | safetensors | 0 | 0 |

Base model: amd/Instella-MoE-16B-A3B-Think. Generated from Hub metadata; download counts move over time.

<!-- PEER-TABLE:END -->

<!-- CREDITS:START -->

Acknowledgements

This build would not exist without the work below. Please star and follow these

projects — the quantisation format used here is their engineering, not mine.

**ROCmFPX — maintained by

charlie12345 / caf**

The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork.

Every ROCmFP4 file in this repository was produced with its llama-quantize, and

runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney,

PlunderStruck and Aydan S., and acknowledges AMD for hardware support.

Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors

The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm

The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights,

licences and capabilities are theirs. This repository contributes quantisation and

measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->