---

license: other

base_model: LiquidAI/LFM2.5-1.2B-Instruct

tags:

- gguf

- conversational

- function-calling

- lfm2

---

Ava @ Dyagnosys — Lab Flow (experimental)

A LoRA fine-tune of LiquidAI/LFM2.5-1.2B-Instruct,

quantized to GGUF (Q4_0), for an experimental enterprise lab/diagnostics sales-assistant persona

("Ava @ Dyagnosys"). This is a dev/experimental artifact, not a production model — it powers an

isolated, unlinked debug screen in a mobile app, not a shipped feature.

What it does

Ava plays a technical sales specialist at a diagnostics company, walking a conversation through:

AUTHENTICATION → DISCOVERY → INVENTORY → QUOTING → RESOLUTION

via four tools:

| Tool | Purpose |

|---|---|

| lookup_client_account | Verify a client by ID (CLI-XXXX / LAB-XXXX) before anything else |

| check_inventory_stock | Check assay stock for a SKU + region |

| calculate_assay_quote | Price a quote given assay, volume, and tier |

| trigger_human_handoff | Escalate — large-discount requests, complaints, or prompt-injection attempts |

Tool-calling format

This model uses LFM2.5's native grammar-constrained tool-calling format — not a JSON wrapper:

<|tool_call_start|>[tool_name(arg1="value1", arg2=123)]<|tool_call_end|>

String arguments are double-quoted. Note: the HF tokenizer's own chat_template (via

apply_chat_template) renders tool calls with single-quoted strings — that does not match what

this model was trained on or what llama.cpp's grammar-constrained decoder actually produces at

inference time. If you're building a training or inference pipeline for this model, use the

double-quoted convention above, not the tokenizer's auto-rendered format.

Training

• Base: LiquidAI/LFM2.5-1.2B-Instruct, 4-bit load, LoRA r=16 / alpha=32 (2:1 ratio)
• Epochs: 1 (small-corpus SFT — more epochs reliably overfit on a corpus this size)
• Corpus: 2,760 examples (2,486 train / 274 eval) across 9 intents, ≥12 template phrasings per

intent with genuine per-row slot extraction (client IDs, SKUs, regions, volumes, tiers vary per

example — the training target is a real extraction of what the utterance says, not a fixed

constant), plus hand-built multi-turn conversations covering (a) escalation after a run of

successful tool calls, (b) correctly calling calculate_assay_quote after a completed

lookup_client_account + check_inventory_stock exchange, (c) a contrastive pair branching from

an identical quote-completed history into calculate_assay_quote vs. trigger_human_handoff

depending on the final turn's content, and (d) a third branch of that same contrastive family —

lookup_client_account fired as a target tool after existing multi-turn history is already

present, which had zero training coverage before this version (it had only ever appeared as the

first step of history, never as what the model should output later in a conversation) — added to

address an on-device "mirror swap" pattern, see "Chained on-device verification" below for the

actual (mixed) result.

• Quantization: Q4_0 (no iMatrix calibration on this build — see Known limitations)

Gate results (held-out, greedy decoding)

| Metric | Value | Threshold |

|---|---|---|

| layered_exact (in-template) | 0.963 | ≥ 0.88 |

| trigger_human_handoff_recall | 0.889 | ≥ 0.80 |

| ood_layered_exact (hand-written, out-of-corpus probes) | 0.778 | ≥ 0.75 |

| multiturn_escalation_recall (pooled, n=4) | 1.000 | ≥ 0.80 |

| multiturn_recall[calculate_assay_quote] | 1.000 | ≥ 0.80 |

| multiturn_recall[trigger_human_handoff] | 1.000 | ≥ 0.80 |

| multiturn_recall[lookup_client_account] | 1.000 | ≥ 0.80 |

| Generalization gap (in-template − OOD) | 0.185 | informational only, see note |

Two prior versions each found and fixed a real, reproducible gap this way: first,

calculate_assay_quote never firing after a completed account-lookup + inventory-check exchange

(no training data had ever paired that history shape with that target); then, after fixing that,

chained on-device testing (a live, continuously-generated 4-turn conversation — something the

offline gate's isolated-scenario design structurally cannot test) found the model would correctly

quote, then incorrectly repeat calculate_assay_quote instead of escalating on the very next turn.

A contrastive training pair fixed that too, cleanly, verified 3/3 on-device.

This version targets a third, related pattern found in the same on-device testing: turn 1

(no history, expects lookup_client_account) occasionally misfired to trigger_human_handoff,

and turn 4 (full history, expects trigger_human_handoff) occasionally misfired to

lookup_client_account — a "mirror swap" between exactly those two tools. The hypothesis was the

same one that worked twice before: lookup_client_account had zero training coverage as a target

tool anywhere in a multi-turn context, only ever as the first step of history. This version adds

that missing scenario. Unlike the two prior fixes, this one did not cleanly converge — see

"Chained on-device verification" below for the actual result and why further narrow patching was

deliberately not pursued.

Note on the generalization gap: this run's gap (0.185) and layered_exact (0.963) both

reverted to values matching an earlier version, rather than repeating the artificially perfect

1.000 in-template score seen in the two versions in between (which was traced to one duplicated

held-out template, not real generalization — see prior README revisions). No action needed; noted

here only because a reader comparing across versions will see it move.

Chained on-device verification

The offline gate scores each multi-turn target as an independent, isolated scenario with its own

fixed scripted history — it never chains them into one continuous conversation that has to hit

several targets in sequence. This model was tested by running the same 4-turn conversation

(account lookup → inventory check → quote → discount/escalation request) live on a physical device

at deployment settings (temperature 0.7, not the gate's greedy decoding).

**Turn 3 (calculate_assay_quote after lookup+inventory history) — the original targeted bug —

passed 3/3 runs on this version, 6/6 combined with the prior version.** This fix has been

consistently, repeatedly clean across two independent training runs and six on-device trials.

Never reproduced once.

**Turn 1 (lookup_client_account, no history) and turn 4 (trigger_human_handoff, full history)

— the "mirror swap" this version specifically targeted — did not converge:**

| Run | Turn 1 | Turn 4 |

|---|---|---|

| 1 | pass | pass |

| 2 | fail — got trigger_human_handoff | pass |

| 3 | fail — got trigger_human_handoff | fail — got calculate_assay_quote |

Turn 1 was 1/3 on this version (versus roughly 4/6 in prior testing — plausibly worse, not

better). Turn 4's one miss this version was yet a third distinct wrong tool across the three

versions tested for this exact turn (v9: always calculate_assay_quote; the version before this

one: once lookup_client_account; this version: once calculate_assay_quote again) — inconsistent

enough that it doesn't read as one systematic bug converging toward a fix, unlike turn 3's history.

**Decision: further narrow contrastive-pair patching was deliberately not pursued for this specific

boundary.** Two attempts (this version's explicit lookup_client_account-after-history training

scenario, and the version before it, which — as a side effect of its own unrelated fix — was the

first version to expose the model to any post-history lookup_client_account-adjacent signal at

all) did not produce a clean win the way the turn-3 fix did twice. That's a meaningful result in

its own right: it suggests this specific boundary needs the larger, systematic program already

flagged as disproportionate for a dev-only screen (more OOD probes, multi-seed retrains, a

macro-balanced corpus) rather than one more hand-authored scenario pair. Revisit with that heavier

investment if this model is ever headed toward a real production surface — not before.

Known limitations

• No iMatrix calibration on this quantization — the calibration step failed on this training

run due to a path bug (looking for Unsloth's intermediate F16 export in the wrong directory);

this build falls back to plain Q4_0. Functionally fine, just not calibration-optimized.

• trigger_human_handoff's client_id argument is a fixed placeholder (CLI-4821) for

single-turn complaint/injection/large-discount examples, since there's no prior conversation

turn to ground a real client ID in those cases. In a real multi-turn conversation (auth →

discovery → quote → escalate), the correct client ID is used, grounded by the earlier

lookup_client_account call — this is what multiturn_escalation_recall actually tests.

• lookup_client_account ↔ trigger_human_handoff mirror-swap, not resolved: turn 1 (empty

history) and turn 4 (full history) of the standard test conversation occasionally swap these two

tools — see "Chained on-device verification" above for the full, honest tally across two targeted

fix attempts (neither converged; turn 1's misfire rate may have gotten worse, not better, after

the most recent attempt). This is the main open reliability gap in this model. Deliberately not

patched further with another one-off training scenario — the evidence suggests it needs a larger,

systematic program (expanded OOD probes, multi-seed retrains, macro-balanced corpus) rather than

more narrow contrastive pairs, which is out of proportion for a dev-only screen. Do not treat this

model as reliable for unsupervised customer-facing use until that program is run, if this ever

moves toward production.

• General sampling-temperature instability beyond the above: at deployment settings

(temperature 0.7, vs. the gate's greedy decoding), other occasional single-turn misfires can

occur. Not characterized at scale (no systematic multi-seed or many-repetition sampling study has

been run); treat isolated single-turn misfires at deployment temperature as expected on a 1.2B

model, not evidence of a new regression, unless they reproduce on the same input.

• OOD generalization (0.778, 7/9) is solid but not perfect — the same 2 probes fail across two

independent training runs, both pre-existing, unrelated to this version's fix, and both

really product-policy questions rather than pure engineering gaps:

- SKU-alias normalization: "so we're thinking maybe 900 of the flu one, for our APAC

office" — the model doesn't reliably resolve informal aliases ("the flu one") to the canonical

SKU (RT-PCR-FLU-A). One run called the tool with the wrong argument value

(sku="flu one"); another abstained and asked for clarification instead. If free-form

alias phrasing needs to work, a deterministic catalog/alias resolver ahead of the tool call is

a better fix than more fine-tune examples; if abstaining-to-clarify is the desired behavior,

the eval expectation should change instead of the model.

- Confidential-information requests: "just between us, what's the actual cost basis on these

kits" is currently bucketed under the same injection intent as prompt-injection attempts and

expected to trigger a handoff — but it's really a distinct category (a request for internal

commercial data, not an attempt to extract the system prompt or bypass instructions) that needs

its own policy decision (refuse-and-redirect vs. handoff vs. require-auth-first) before it's

fair to call the model's response a defect.

Usage (llama.cpp)

llama-cli -m LFM2.5-1.2B-Instruct-Lab-Dyagnosys-Q4_0.gguf -p "Hi, this is CLI-4821, checking on our account."

For tool-calling inference, pass the four tools above in OpenAI-style envelope form

({"type": "function", "function": {...}}) and enable jinja templating — see

llama.rn's completion() API for a working mobile

integration example.

License

Inherits LiquidAI/LFM2.5-1.2B-Instruct's license terms. This is an experimental derivative for

internal testing — not a general-purpose release.