---

license: apache-2.0

base_model: Qwen/Qwen3-4B

tags:

- emergency-triage

- "911"

- slm

- qlora

- gguf

- on-device

- mobile

- edge-ai

language:

- en

pipeline_tag: text-generation

---

Emergency Triage 4B (GGUF)

A fine-tuned Qwen3-4B model specialized for 911/emergency incident triage in remote locations. Designed for on-device inference on phones, tablets, and edge hardware where connectivity is limited or unavailable.

What it does

Given a free-text emergency incident report, the model produces a structured JSON triage assessment:

{
  "severity": "CRITICAL",
  "incident_type": "MEDICAL",
  "summary": "Hiker fell 30 feet from cliff, unable to move legs...",
  "location": {
    "description": "Pacific Crest Trail, mile marker 1847",
    "accessibility": "TRAIL"
  },
  "resources_needed": ["Helicopter EMS", "Rope rescue team", "Trauma kit"],
  "priority_actions": ["Immobilize legs", "Apply thermal protection", "Request helicopter evacuation"],
  "requires_evacuation": true,
  "communication_status": "LIMITED"
}

---

How to run

1. Ollama (Mac / Linux / Windows)

The easiest way to run locally.

# Download the GGUF file
hf download ajvikram/emergency-triage-4b-gguf emergency-triage-4b-q4_k_m.gguf

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./emergency-triage-4b-q4_k_m.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

SYSTEM """You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment."""

PARAMETER temperature 0.3
PARAMETER num_ctx 2048
PARAMETER stop "<|im_end|>"
EOF

# Import into Ollama
ollama create emergency-triage -f Modelfile

# Run it
ollama run emergency-triage "A hiker has collapsed on a remote trail 8 miles from the nearest road. They are unresponsive, breathing shallow. Two other hikers are present. No cell service. Temperature is 95F with high humidity."

2. llama.cpp (Mac / Linux / Windows)

For direct GGUF inference without Ollama.

# Download the GGUF
hf download ajvikram/emergency-triage-4b-gguf emergency-triage-4b-q4_k_m.gguf

# Run inference (Metal GPU on Mac, CUDA on Linux)
llama-cli \
  -m emergency-triage-4b-q4_k_m.gguf \
  -p "<|im_start|>system
You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment.<|im_end|>
<|im_start|>user
Your emergency scenario here<|im_end|>
<|im_start|>assistant" \
  -n 512 --temp 0.3 -ngl 99 --single-turn

3. iPhone / iPad (iOS)

Using LLM Farm (free) or PocketPal AI:

1. Download emergency-triage-4b-q4_k_m.gguf (2.4 GB) to your device
2. Open the app and import the GGUF file
3. Set the system prompt to:

```

You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment.

```

1. Set temperature to 0.3 for consistent outputs
2. Type or speak your emergency scenario

Device requirements: iPhone 12+ or iPad Air 4+ (4GB+ RAM). The Q4_K_M quantization runs well on A14 chip and newer.

4. Android

Using ChatterUI or llama.cpp Android:

1. Download emergency-triage-4b-q4_k_m.gguf to your device
2. Open the app and load the model
3. Set system prompt as above
4. Set temperature to 0.3

Device requirements: 6GB+ RAM recommended. Works on Snapdragon 8 Gen 1+ or equivalent.

5. Python (transformers)

For the full-precision merged model:

from transformers import AutoModelForCausalLM, AutoTokenizer
import json

model = AutoModelForCausalLM.from_pretrained("ajvikram/emergency-triage-4b-gguf", subfolder="merged")
tokenizer = AutoTokenizer.from_pretrained("ajvikram/emergency-triage-4b-gguf", subfolder="merged")

messages = [
    {"role": "system", "content": "You are an emergency incident triage system. Analyze the incident and respond with a JSON triage assessment."},
    {"role": "user", "content": "Your scenario here"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.3)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

6. Edge Deployment (Raspberry Pi / Jetson)

The Q4_K_M GGUF works on edge devices with 4GB+ RAM:

# On Raspberry Pi 5 (8GB) or Jetson Nano
# Build llama.cpp for your platform
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build

# Run inference (CPU-only on Pi, CUDA on Jetson)
./build/bin/llama-cli -m emergency-triage-4b-q4_k_m.gguf \
  -p "your prompt" -n 512 --temp 0.3 --single-turn

Expected speeds: ~5-10 tok/s on Pi 5, ~30+ tok/s on Jetson Orin.

---

Training details

• Method: QLoRA (r=32, alpha=64) via Unsloth
• Data: 500 examples (52 hand-crafted seeds + 448 synthetic via knowledge distillation)
• Teacher: 120B parameter model for synthetic data generation and quality judging
• Epochs: 3
• Final eval loss: 0.52
• Hardware: NVIDIA RTX A6000 (48GB), ~8 minutes training time

Quantization

| Format | Size | BPW | Use case |

|--------|------|-----|----------|

| F16 (merged) | 7.6 GB | 16.0 | Server / fine-tuning base |

| Q4_K_M | 2.4 GB | 4.95 | Mobile / edge deployment |

Performance

| Platform | Speed | Notes |

|----------|-------|-------|

| Mac (Metal) | ~84 tok/s | M1/M2/M3 Apple Silicon |

| NVIDIA GPU | ~100+ tok/s | CUDA, depends on GPU |

| iPhone 15 Pro | ~15-20 tok/s | A17 Pro chip |

| Raspberry Pi 5 | ~5-10 tok/s | CPU only, 8GB model |

Output schema

{
  "severity": "CRITICAL | HIGH | MEDIUM | LOW",
  "incident_type": "MEDICAL | NATURAL_DISASTER | RESCUE | INFRASTRUCTURE | WILDLIFE | WEATHER | VEHICLE | OTHER",
  "summary": "1-2 sentence dispatch summary",
  "location": {
    "description": "location details",
    "accessibility": "ROAD | TRAIL | OFF_TRAIL | AIR_ONLY"
  },
  "resources_needed": ["specific resources"],
  "priority_actions": ["ordered by urgency"],
  "requires_evacuation": true/false,
  "communication_status": "FULL | LIMITED | NONE"
}

Limitations

• Phase 0 prototype trained on 500 examples — production would use 40,000+
• Schema adherence is good but not perfect with limited training data
• Optimized for English-language incident reports
• Designed for remote/rural emergency scenarios
• Not a replacement for trained dispatchers — intended as a decision support tool

License

Apache 2.0 (inherited from Qwen3-4B base model)