---

license: apache-2.0

base_model: Qwen/Qwen3-VL-4B-Instruct

tags:

- emergency-triage

- structured-output

- json

- gguf

- on-device

- edge-ai

- vision-language

- multimodal

- extraction

- llama.cpp

language:

- en

pipeline_tag: image-text-to-text

---

CraneBear VLM 4B (GGUF) — transcript + optional photo → extraction JSON

A LoRA fine-tune of Qwen3-VL-4B-Instruct

that turns a panic emergency voice transcript, with an optional photo, into

the same 7-key extraction JSON the CraneBear text model emits, for a human

dispatcher. Ships as a Q4_K_M text GGUF plus an f16 multimodal projector, for

llama.cpp / llama-server / the iOS mtmd path.

> The model extracts. A human decides. It never triages, dispatches, or

> advises. Severity bands are evidence labels for a trained dispatcher.

This repo replaces ajvikram/emergency-vision-triage-2b-gguf, which was a

text-only decoder with no vision tower despite its name (documented in the

training write-up); that repo is deprecated.

Two checkpoints

| File | Use it for | 24-case held-out set (partner's, text-only) |

|---|---|---|

| vlm-v2-r2-q4_k_m.gguf | best text extraction (no photo) | 93.8% macro, 0% missed-IMMEDIATE (raw 92.7%) |

| vlm-v2-r3-q4_k_m.gguf + mmproj-vlm-v2-r3-f16.gguf | transcript + photo | 91.7% macro, 0% missed-IMMEDIATE (raw 88.5%) |

For reference, the partner's production text model (cranebear-extraction-4b,

round 3) scores 91.7% / 0% on the same set, on-contract. Both checkpoints here

were trained on the same 800-row round-3 text set; r3 adds 1,018

(transcript, photo) pairs.

Status: what is and is not established

Established, measured:

• Text contract: both checkpoints pass the partner's blocking gate (0%

missed-IMMEDIATE, asserted) and the ≥90% macro gate on their 24-case set.

• The vision path is real: r3 reads hazards from photos (7/7 on the

photo-adds slice of a 60-case photo set), ignores irrelevant photos in 14/18

cases, and over-triages far less than a text-only checkpoint shown the same

photos (false-IMMEDIATE 3/16 vs 16/31).

• Served through llama-server and through the partner's Swift/mtmd path,

the two stacks agree on these numbers.

Not established — do not treat as production-safe for photo input yet:

• The 60-case photo eval set carries proposed, unratified labels and only

34 sourced photos. On it, r3 still has 1 missed-IMMEDIATE, 1 photo-subtracted

severity, 2 photo-only HIGH confidences.

• Those last two classes go to zero for every checkpoint when the caller

applies the text-first merge (below). That merge is a caller-side guardrail;

it is not inside the weights.

• The photo training taught four rules (a photo may add a hazard; a photo never

subtracts; photo-only evidence never HIGH; an irrelevant photo changes

nothing). It did not teach a photo supplying a rule-1 sign (a person

under rubble) or a casualty count — no public image set covers that ethically.

• Never run on a phone. All numbers are llama.cpp on a Mac.

Serving contract (must match exactly)

• Prompt: the SYSTEM text in Modelfile.vlm-v2-r3 (byte-identical to the

CraneBear text model's round-3 prompt). With a photo, append the paragraph in

photo_addendum.py to the system prompt and place the image before

Transcript: in the user turn.

• No think block. Qwen3-VL-Instruct's template has none; rendering one is a

served/trained mismatch.

• Grammar: cranebear-extraction-presence.gbnf. The model emits a

presence-encoded object (no nulls); the consumer maps sentinels to nulls.

• Greedy (temperature 0). Photos downscaled to 512 px longest side

(~200 image tokens).

llama-server -m vlm-v2-r3-q4_k_m.gguf --mmproj mmproj-vlm-v2-r3-f16.gguf -c 4096 -ngl 99
# POST /v1/chat/completions with the system prompt (+ addendum), an image_url
# content part, "Transcript:\n<text>", and "grammar": <the gbnf>.

Recommended caller-side merge for photo input (what the partner's

Guardrails.mergePhoto does): run the text-only pass first, then the photo

pass; take severity = max on MINOR < DELAYED < IMMEDIATE (DECEASED only from

text), OR the hazard bits, keep a stated casualty count, and cap photo-only

severity confidence at MEDIUM. Measured: this removes every "photo subtracted"

and "photo-only HIGH" case for every checkpoint tested.

Training

• Base Qwen/Qwen3-VL-4B-Instruct (16-bit), LoRA r=16 on the language tower,

vision encoder frozen, early stopping on eval loss, best checkpoint loaded.

• r2: 800 text rows (partner-schema, presence-encoded), 4 epochs, ~3.4 h.
• r3: + 1,018 photo pairs from 880 Wikimedia Commons photos (CC0 / CC BY /

CC BY-SA / PD, recorded per file) with deterministic labels derived from

each photo's category and the four photo rules — no teacher model. 300

steps, ~6.9 h. mmproj is byte-identical to the base's (frozen tower).

• Hardware: NVIDIA DGX Spark (GB10). Reproduction scripts, data engines, eval

harnesses and the full round-by-round record (including a documented

negative round 4) live in the slm-factory repo.

Files

• vlm-v2-r2-q4_k_m.gguf, vlm-v2-r3-q4_k_m.gguf — text models (Q4_K_M, 2.5 GB)
• mmproj-vlm-v2-r3-f16.gguf — vision projector for r3 (839 MB)
• cranebear-extraction-presence.gbnf — grammar
• Modelfile.vlm-v2-r2, Modelfile.vlm-v2-r3 — Ollama Modelfiles (SYSTEM block = the prompt)
• photo_addendum.py — the photo paragraph, with its SHA
• evals/ — the raw eval printouts behind every number above
• SHA256SUMS