---

base_model: F-A-I-L/mini-agent-v2

base_model_relation: quantized

library_name: gguf

pipeline_tag: text-generation

license: apache-2.0

language:

• en

tags:

• gguf
• quantized
• qwen2
• qwen2.5-coder
• lora
• qlora
• unsloth
• trl
• tool-calling
• function-calling
• agent
• bash
• shell
• code
• python
• edge
• on-device

datasets:

• Salesforce/xlam-function-calling-60k
• glaiveai/glaive-function-calling-v2
• Team-ACE/ToolACE
• Agent-Ark/Toucan-1.5M
• newfacade/LeetCodeDataset
• F-A-I-L/kodcode-verified-python-235k
• westenfelder/NL2SH-ALFA
• mecha-org/linux-command-dataset
• Frost2o24/bash-instruct-III-55k

---

<div align="center">

<img src="assets/frost-ai-lab.svg" alt="FROST AI lab" width="320"/>

mini-agent-v2-GGUF

**A 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct,

shipped as merged bf16 weights and six GGUF quants.**

bf16 weights · GGUF quants

checkpoint b15 (d1497d77e5) · evaluated 2026-10-04

</div>

---

Summary

mini-agent-v2 is a QLoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, trained jointly on

function calling, Bash and Python on a single 8 GB consumer GPU. It is small enough to run

on a laptop. It ships as merged bf16 weights and as GGUF builds from Q3_K_M to bf16.

It speaks the standard tool-call protocol: Qwen's own chat template with

<tool_call>{"name": ..., "arguments": {...}}</tool_call>. That is the format llama.cpp

(--jinja), vLLM, Ollama and OpenAI-style clients already parse. You don't need a custom

system prompt or parser: pass your tools in the normal tools field. On the Berkeley Function

Calling Leaderboard (BFCL) questions it answers 86.2% correctly, up from 52.7% for the

base model.

At a glance

| | |

|---|---|

| ✅ Use it for | native tool calls through any OpenAI-compatible server · Bash one-liners · small Python functions · local/offline agents |

| ⚠️ Don't use it for | long autonomous multi-step agent loops · shell commands you won't read · anything safety-critical |

| 📈 Best result | 86.2% on BFCL non-live in FC mode (base: 52.7%), level with Qwen3-1.7B and xLAM-2-3B |

| 📉 Biggest weakness | real-user BFCL live questions: 65.7%, so about 1 in 3 is still wrong |

| 🧰 Works with | llama-server --jinja, vLLM, transformers; tool calls checked on every GGUF |

| 💾 Smallest build | 824 MB (Q3_K_M GGUF) |

Specifications

| | |

|---|---|

| Base model | Qwen/Qwen2.5-Coder-1.5B-Instruct (1.54 B params, 28 layers, tied embeddings) |

| Method | QLoRA over unsloth/qwen2.5-coder-1.5b-instruct-bnb-4bit: LoRA rank 16, alpha 32, dropout 0, bias none |

| Trainable params | 18,467,840: 392 LoRA A/B tensors (28 layers × 7 projections) + 2 re-initialised tool-tag embedding rows |

| Target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |

| Training context | 2048 tokens, no packing |

| Inherited context | 32,768 tokens |

| Release precision | adapter merged into the bf16 base |

| License | Apache-2.0 |

---

Results

<img src="assets/headline.png" alt="Grouped bar chart comparing Qwen2.5-Coder-1.5B-Instruct against mini-agent-v2. BFCL standard 52.7% base vs 86.2%; BFCL live 56.9% vs 65.7%; NL2SH-ALFA bash 41.7% vs 50.3%; InterCode bash 30.5% vs 36.0%; MultiPL-E sh 32.8% vs 32.6%; HumanEval+/MBPP+ 59.2% vs 66.2%." width="100%"/>

These are the three tasks the model was trained for, compared with the base model under an

identical harness. Every answer is executed, not string-matched: Bash runs in Docker,

Python runs against unit tests, and tool calls go through BFCL's AST checker.

| Benchmark | n | Base Qwen2.5-Coder-1.5B-Instruct | mini-agent-v2 | Δ | Paired test |

|---|---:|---:|---:|---:|---|

| Tool calls: BFCL non-live, FC mode | 1000 | 52.7% | 86.2% | +33.5 pp | McNemar p < 0.001 |

| Tool calls: BFCL live, FC mode | 1335 | 56.9% | 65.7% | +8.8 pp | McNemar p < 0.001 |

| Bash: NL2SH-ALFA, executed | 300 | 41.7% | 50.3% | +8.6 pp | McNemar p < 0.001 |

| Bash: InterCode-Bash, executed | 200 | 30.5% | 36.0% | +5.5 pp | McNemar p = 0.080, n.s. |

| Bash: MultiPL-E sh, unit tests | 540 | 32.8% | 32.6% | −0.2 pp | McNemar p = 1.000, n.s. |

| Python: HumanEval+ / MBPP+, executed | 542 | 59.2% | 66.2% | +7.0 pp | McNemar p < 0.001 |

• Tool calling is the big gain. The base model's tool-tag embeddings are untrained, so it

barely emits its own <tool_call> format, and when asked for tools it mostly answers in

Markdown JSON. Fixing that, plus the tool data, takes it from 52.7% to 86.2%.

• Bash and Python improve without being traded away. NL2SH-ALFA gains 26 commands and

Python gains 38 problems, both significant. MultiPL-E sh (writing whole shell programs)

is unchanged.

• InterCode needs a caveat. 127 of its 200 tasks overlap NL2SH-ALFA's test split, and the

whole gain sits there. On the 73 independent tasks it is flat.

Compared with other small tool-calling models

<img src="assets/peers.png" alt="Bar charts of BFCL FC-mode accuracy. Standard: xLAM-2-3b 87.7%, Qwen3-1.7B 86.5%, mini-agent-v2 86.2%, Arch-Function-1.5B 82.1%, xLAM-2-1b 75.7%, Hammer2.1-1.5b 71.6%, base 52.7%. Live: mini-agent-v2 65.7%, Hammer2.1-1.5b 57.2%, base 56.9%, Arch-Function-1.5B 55.6%, xLAM-2-1b 52.2%." width="100%"/>

Every model was run through the same harness in bf16, using its own chat template and native

tool-call format.

| Model | BFCL non-live | BFCL live | NL2SH-ALFA | InterCode | MultiPL-E sh | HE+/MBPP+ |

|---|---:|---:|---:|---:|---:|---:|

| mini-agent-v2 (1.5B) | 86.2% | 65.7% | 50.3% | 36.0% | 32.6% | 66.2% |

| Qwen2.5-Coder-1.5B-Instruct (base) | 52.7% | 56.9% | 41.7% | 30.5% | 32.8% | 59.2% |

| Hammer2.1-1.5b | 71.6% | 57.2% | 41.3% | 28.5% | 34.3% | 62.0%* |

| xLAM-2-1b-fc-r | 75.7% | 52.2% | 26.0% | 21.0% | 22.4% | 54.6% |

| Arch-Function-1.5B | 82.1% | 55.6% | 34.3% | 23.0% | 24.1% | 55.4% |

| xLAM-2-3b-fc-r (3B) | 87.7% | | | | | ** |

| Qwen3-1.7B, no-think | 86.5% | | | | | ** |

Notes: * = the gap to mini-agent-v2 is statistically significant (both models answered the same

questions; paired exact McNemar test, p < 0.05), so it is unlikely to be luck. No * means the two are

effectively tied. ** = that model was not tested on this benchmark (not a score of zero).

---

Files

The weights are split across two repos: F-A-I-L/mini-agent-v2 holds the bf16

safetensors for transformers / vLLM, and F-A-I-L/mini-agent-v2-GGUF holds the GGUF

builds for llama.cpp and GGUF apps.

| File | Repo | Size | Bits per weight |

|---|---|---:|---:|

| model.safetensors + config.json + tokenizer files | mini-agent-v2 | 3.09 GB | 16 (bf16) |

| mini-agent-v2-Q3_K_M.gguf | mini-agent-v2-GGUF | 824 MB | 4.24 |

| mini-agent-v2-Q4_K_M.gguf | mini-agent-v2-GGUF | 986 MB | 5.08 |

| mini-agent-v2-Q5_K_M.gguf | mini-agent-v2-GGUF | 1.13 GB | 5.80 |

| mini-agent-v2-Q6_K.gguf | mini-agent-v2-GGUF | 1.27 GB | 6.56 |

| mini-agent-v2-Q8_0.gguf | mini-agent-v2-GGUF | 1.65 GB | 8.50 |

| mini-agent-v2-bf16.gguf | mini-agent-v2-GGUF | 3.09 GB | 16 (unquantized) |

GGUF metadata: architecture qwen2, 1,543,714,304 params, context 32768, EOS <|im_end|>,

with the stock Qwen2.5 chat template (tool support intact). All quants were made from the

bf16 GGUF with llama-quantize. Plan on the file size plus about 1 GB of memory.

Scores above are for the bf16 weights. Fewer bits per weight means a smaller file and lower

quality. Every GGUF was checked to produce a correct tool call, but the quants were not

re-benchmarked, so expect somewhat lower scores than these, more so the smaller the file.

---

How to prompt it

Tool calling: pass tools through the chat template's tools= field (or the OpenAI

tools field on a server). No system prompt is needed; the template inserts the tool list.

It replies with one or more <tool_call>{"name": ..., "arguments": {...}}</tool_call>

blocks, or with plain text when no tool fits.

Bash

You are a Bash assistant. Translate the request into a single Bash command.

For a full script: You are a Bash assistant. Write a complete Bash script for the request.

Python

You are a Python coding assistant. Write correct code. Respond with a single code block.

The benchmark numbers on this card were measured with greedy decoding (temperature 0).

Usage

llama.cpp

# downloads the chosen quant from the GGUF repo on first run
# (QUANT = Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0 or bf16)
llama-server -hf F-A-I-L/mini-agent-v2-GGUF:QUANT --jinja -c 4096 -ngl 99
# or, with a file you already downloaded
llama-server -m mini-agent-v2-QUANT.gguf --jinja -c 4096 -ngl 99

--jinja is required for tool calling. Without it, llama.cpp ignores the template's tool

support.

curl localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Berlin in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "parameters": {"type": "object", "properties": {
      "city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
    "required": ["city"]}}}]}'
# -> tool_calls: get_weather({"city": "Berlin", "unit": "celsius"})

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "F-A-I-L/mini-agent-v2"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")

tools = [{"type": "function", "function": {"name": "get_weather",
          "parameters": {"type": "object", "properties": {"city": {"type": "string"}},
                         "required": ["city"]}}}]
msgs = [{"role": "user", "content": "Weather in Paris?"}]
enc = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))
# <tool_call>
# {"name": "get_weather", "arguments": {"city": "Paris"}}
# </tool_call><|im_end|>

Any GGUF app

LM Studio, Jan and KoboldCpp load the .gguf directly. To use tools, turn on the app's

tool/function-calling support.

---

> 🖥️ Built on one 8 GB laptop GPU, looking for a bigger one

>

> Every run behind this model was trained and evaluated on a single RTX 4060 (8 GB). That is

> why it's 1.5B, trained on a 4-bit base with 2k-token examples. The same weights already

> score significantly higher on Bash and Python when run in full bf16 instead of 4-bit.

>

> If your team has an NVIDIA DGX Spark or an **AMD Ryzen AI Max+ ("Strix Halo") AI dev

> kit** to spare, it would go straight into full-precision training, longer context, 3B–7B

> bases and verified-reward RL, all released openly.

>

> Get in touch via Discussions.

> No obligation; the model stays free either way. Thank you!

---

Evaluation

How this was measured

All results come from a run on 2026-10-04 (external models 2026-10-05), scored with each

benchmark's own scorer. Where a task can be executed, it was executed.

• Decoding: greedy, batch size 8, 4096-token eval context, bf16 weights, the same harness

for every model.

• Tool calling: BFCL non-live (simple,

multiple, parallel, parallel-multiple) and live, in FC mode. Tools go through each

model's tools= chat template, the answer is decoded from its native tool-call output,

and it is checked with a reimplementation of BFCL's official AST checker. This is this

project's harness, not the public leaderboard's prompt mode, so the numbers are not

directly comparable with the leaderboard.

• Bash: NL2SH-ALFA test split,

executed through InterCode-ALFA in Docker and scored on filesystem/output equivalence.

InterCode-Bash runs in its own images.

MultiPL-E sh runs unit tests in a network-less container.

• Python: EvalPlus HumanEval+ and MBPP+ (base +

hardened "plus" tests), executed.

• Significance: paired exact McNemar test on per-question outcomes. Same problems, same

prompts, same decoding.

Tool calling, in detail

| Split | mini-agent-v2 | Base |

|---|---:|---:|

| non-live micro / macro | 862/1000 (86.2%) / 85.1% | 527/1000 (52.7%) / 43.7% |

| — simple / multiple | 362/400 · 181/200 | |

| — parallel / parallel-multiple | 162/200 · 157/200 | |

| live micro / macro | 877/1335 (65.7%) / 56.3% | 759/1335 (56.9%) / 29.0% |

| — live simple / live multiple | 160/258 · 697/1037 | |

| — live parallel / live parallel-multiple | 6/16 · 14/24 | |

The remaining errors are mostly argument values, not tool choice:

• Optional parameters left out or added when the user didn't ask for them
• User wording not mapped onto a schema enum value
• Date arithmetic ("next Tuesday")
• Very long tool lists

---

Development history

<img src="assets/lineage.png" alt="Line chart of BFCL FC-mode accuracy across runs b10-10pct to b16. Standard rises from 80.3% to 85.7% at b15; live from 59.7% to 65.8% at b15; b16 is slightly lower on both." width="100%"/>

mini-agent-v2 is run b15 of a series:

• b1–b9 trained BFCL's prompt format ([f(a=1)]). It scored well on the benchmark, but

no inference server parses it.

• From b10, every run is native-FC only. b10 also brought the tool-tag fix: in the base

model, the <tool_call> / </tool_call> embedding rows are byte-identical untrained

vectors with tied embeddings. LoRA doesn't touch embedding rows, so both were

re-initialised and trained as extra tokens.

• b11–b15: each run fixed one failure class found by row-level error analysis of the

previous run.

- b12 reverted a rule that taught dropping optional parameters and removed rows with

invented years.

- b14 added targeted slices.

- b15 removed the b14 slice that caused its non-live regression.

• b16 added longer context and a "fill discipline" filter. It came out null against b15,

so b15 is the release.

The chart shows each run's 4-bit evaluation, which is why b15 reads 85.7% / 65.8% there and

86.2% / 65.7% in bf16 above. Neighbouring runs are often within noise: b15 vs b14 was

p < 0.001 on non-live and p = 0.128 on live.

---

Training recipe

| | |

|---|---|

| Epochs | 1 |

| LR / schedule | 1e-4, cosine |

| Batch | 4 × grad-accum 2 (effective 8) |

| Sequence | max 2048, packing=False |

| Regularization | none (NEFTune off) |

| Loss masking | train_on_responses_only, anchored on the user header so tool results are never trained as model output |

| Seed | 3407 |

| Framework | Unsloth + TRL SFTTrainer, 1 × RTX 4060 8 GB |

Data: 24,745 rows after the length filter: tool 49%, Python 28%, Bash 23%.

• Tool (12,275, all native FC, rendered through the real chat template):

- xLAM calls, weighted toward parallel and multi-call shapes

- Toucan MCP calls from real servers

- 1,000 ToolACE rows

- 1,000 full glaive loops (call → tool result → answer)

- 500 "irrelevant" rows, where the tools are listed but the right answer is plain text

- Slices for array arguments, enum mapping, date phrasing and a generic shell tool

- No trained call contains a year the user didn't state.

• Python (7,000): LeetCode problems and stub completions, plus KodCode rows (CC-BY-NC-4.0, see License) whose answers

were re-executed against their unit tests.

• Bash (5,670): NL2SH-ALFA train, bash-instruct-III one-liners and scripts, and

linux-command-dataset.

• Leak protection: training prompts that match BFCL, NL2SH-ALFA test, HumanEval or MBPP+

were removed.

---

Known limitations

• About 1 in 3 real-user tool questions is wrong. See the error classes above.
• Not a multi-step agent. Training is mostly single-call, with 1,000 two-hop loops. On a

Docker agent probe it completes 7/54 tasks, and 8/45 on held-out task families.

• Bash output is a single command or a short script, unverified. Sandbox it.
• The quants were not re-benchmarked. Expect lower scores than the bf16 numbers, more so the smaller the file.
• English only. No safety tuning beyond the base model.

Intended use and limits

Good for: local and offline tool-calling back-ends behind any OpenAI-compatible server,

shell one-liner translation, small Python functions, and edge deployment.

Not for: production automation without validation, long autonomous agent loops, or

anything safety-critical. Never execute generated shell commands unreviewed.

---

Citation

@misc{mini_agent_v2,
  title  = {mini-agent-v2: a 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct},
  author = {Frost2o24},
  year   = {2026},
  url    = {https://huggingface.co/F-A-I-L/mini-agent-v2},
  note   = {Merged bf16 weights and GGUF builds; evaluated on BFCL (FC mode), NL2SH-ALFA, InterCode-Bash, MultiPL-E sh and EvalPlus}
}

License

The model weights are released under Apache-2.0, inherited from Qwen2.5-Coder-1.5B-Instruct.

> ⚠️ Training-data notice: about 5,100 of the 24,745 training rows (the KodCode Python

> slices: 4,300 general + 800 odd-function-name rows, via F-A-I-L/kodcode-verified-python-235k)

> come from KodCode-V1, which is licensed

> CC-BY-NC-4.0 (non-commercial). All other training sources are Apache-2.0, MIT or CC-BY-4.0.

> Whether a dataset's license carries over to a model trained on it is legally unsettled.

> If you plan to use this model commercially, review that dataset's terms and decide for

> yourself; this notice is not legal advice.

Training data licenses are those of the upstream sources listed in the metadata. xLAM

(CC-BY-4.0) is credited to Salesforce Research.