---

license: apache-2.0

base_model: HuggingFaceTB/SmolVLM2-2.2B-Instruct

tags:

- emergency-triage

- structured-output

- json

- gguf

- on-device

- edge-ai

- qlora

language:

- en

pipeline_tag: text-generation

---

Emergency Scene Triage 2B (GGUF)

A QLoRA fine-tune of SmolVLM2-2.2B-Instruct

that turns a text description of an emergency scene into a structured JSON

triage assessment. Quantized to Q4_K_M (1.1 GB) for on-device use.

> The model extracts. A human decides. This is decision-support tooling for a

> trained dispatcher, not an autonomous triage system.

---

⚠️ Read this first: it takes text, not images

Despite the vision-language base model, **this checkpoint does not accept image

input.** The fine-tune targets the LLM backbone only, through SmolVLM2's chat

template — no image ever passes through the vision encoder at training or

inference time.

It is the second stage of a two-stage pipeline:

photo → [any vision model / a human] → text scene description → THIS MODEL → structured JSON

If you pass it an image, it will not work. If you want scene-description →

structured-triage, you're in the right place.

Why split it this way: the description stage stays swappable (any VLM, or a

human on a radio), each stage is debuggable in isolation, and only the text

pathway needed fine-tuning.

---

Output schema

Ten fields, fixed enums:

{
  "scene_type": "VEHICLE_ACCIDENT | MEDICAL_EMERGENCY | FIRE | FLOOD | STRUCTURAL_COLLAPSE | HAZMAT | WILDERNESS_RESCUE | OTHER",
  "severity_estimate": "CRITICAL | HIGH | MEDIUM | LOW",
  "visible_injuries": ["..."],
  "hazards_present": ["..."],
  "people_count": { "visible": 0, "estimated_total": 0, "conscious": 0 },
  "scene_description": "...",
  "environment": { "lighting": "...", "weather": "...", "terrain": "..." },
  "recommended_resources": ["..."],
  "immediate_actions": ["..."],
  "accessibility": "..."
}

---

Usage

Ollama (recommended)

format: "json" is not optional — see Known limitations.

hf download ajvikram/emergency-vision-triage-2b-gguf vision-triage-2b-q4_k_m.gguf
hf download ajvikram/emergency-vision-triage-2b-gguf Modelfile.vision
ollama create emergency-vision -f Modelfile.vision

curl -s http://localhost:11434/api/generate -d '{
  "model": "emergency-vision",
  "prompt": "Analyze this emergency scene photograph:\n\nA red sedan is overturned on a rural mountain road. Two people are visible - one sitting on the roadside holding their arm, the other still inside the vehicle. Broken glass is scattered across the asphalt. It is raining and visibility is poor.",
  "format": "json",
  "stream": false
}'

Production wrapper (strongly recommended)

inference_vision.py in this repo is a ~100-line, zero-dependency wrapper that

handles the model's residual defects: constrained decoding, markdown stripping,

enum normalization, schema defaults, and one retry on parse failure.

python3 inference_vision.py "A two-story house is engulfed in flames. A family of four is on the front lawn; one child appears to have burns on their arms."

The difference is not cosmetic — see below.

---

Evaluation

Seven held-out emergency scenarios (vehicle accident, bridge collapse,

industrial fire, cardiac arrest, flood rescue, HAZMAT spill, wilderness rescue).

| Metric | Raw model | With wrapper |

|---|---|---|

| JSON parses (format: "json") | 7/7 | 7/7 |

| JSON parses (no format: "json") | 0/5 | n/a |

| Correct severity_estimate | 7/7 | 7/7 |

| Correct scene_type | 5/7 | 7/7 |

| All 10 fields present | 6/7 | 7/7 |

| Markdown-free strings | 5/7 | 7/7 |

| All checks pass | 4/7 | 7/7 |

Take the sample size seriously. Seven scenarios is enough to catch obvious

breakage and nothing more. These are not statistically meaningful accuracy

figures, and they are not a safety validation.

---

Known limitations

• Constrained decoding is required. Without Ollama's format: "json" (or an

equivalent GBNF grammar in llama.cpp), raw output failed to parse on 5/5 test

scenarios — markdown bullets break JSON syntax. This is the single most

important operational detail in this card.

• Markdown bleed-through. * bullets appear inside string values on roughly

40% of generations. This comes from the base model's pretraining, not the

training data (which was verified markdown-free). Five epochs of SFT did not

suppress it; a regex strip in the wrapper does.

• Enum truncation. Occasionally emits WILDERNESS instead of

WILDERNESS_RESCUE. The wrapper maps known aliases.

• accessibility is the weakest field — inconsistent shape run to run

(sometimes absent, sometimes an object with varying keys). The wrapper

defaults it.

• English only. Trained on synthetic data.
• Not validated against real dispatch data, real photographs, or any

professional triage standard.

On the GRPO experiments

Two rounds of GRPO (Group Relative Policy Optimization) with rule-based reward

functions were run on top of this SFT checkpoint to try to fix the markdown and

enum defects at the weights level. Neither beat this checkpoint on held-out

scenarios (4/7 → 3/7 → 2/7 strict pass), so the SFT model is what's published

here. The reward curves converged cleanly while real quality declined — a

reward-narrowing result worth knowing about if you plan to try the same thing.

---

Training

| | |

|---|---|

| Base | SmolVLM2-2.2B-Instruct (LLM backbone only) |

| Method | QLoRA, 4-bit NF4 |

| LoRA | r=32, alpha=64, dropout=0.05, all linear projections |

| Epochs | 5 |

| LR | 1e-4, warmup ratio 0.1 |

| Seq length | 2048 |

| Data | 3,000 synthetic examples (2,400 train / 300 val / 300 test) |

| Final train loss | 0.545 |

| Final eval loss | 0.612 |

| Hardware | NVIDIA GB10 (DGX Spark), ~3.5 h |

| Quantization | llama.cpp → F16 GGUF → Q4_K_M (1.1 GB) |

A note on chat templates

SmolVLM2's apply_chat_template requires content as a list of typed dicts:

{"role": "user", "content": [{"type": "text", "text": "..."}]}   # correct
{"role": "user", "content": "..."}                                # silently strips ALL content

Passing a plain string does not raise — it emits role markers with empty bodies,

and training runs on empty messages. The tell is a suspiciously fast run

converging to a suspiciously low loss. Assert your formatted text actually

contains your content before trusting any run.

The Ollama TEMPLATE in Modelfile.vision must also match the training

template byte-for-byte, leading <|im_start|> included.

---

Intended use / out of scope

Intended: research and prototyping of on-device emergency-triage tooling;

structured-output experiments on small VLM backbones; a starting checkpoint for

further fine-tuning.

Out of scope: live emergency dispatch, any deployment where output is acted

on without a qualified human in the loop, medical advice, and any use where

format: "json" and the post-processing wrapper are not in place.

License

Apache-2.0, inherited from SmolVLM2-2.2B-Instruct.