---

base_model: Joesh1/onca-3.0-12b

library_name: transformers

pipeline_tag: text-generation

license: apache-2.0

language:

• en

tags:

• gguf
• llama-cpp
• text-generation
• gemma4
• oncology
• clinical-trials
• pathology
• precision-oncology
• structured-generation

---

ONCA 3.0 (12B) GGUF

This repository provides GGUF quantizations for Joesh1/onca-3.0-12b.

ONCA 3.0 is a specialized 11.95B-parameter oncology research model derived from google/gemma-4-12B. It uses the text-only decoder within Gemma 4's unified architecture and is fine-tuned to generate compact, structured JSON outputs across six oncology tasks: pathological TNM classification (T, N, M staging), clinical trial triage, trial criterion reasoning, and CIViC molecular-evidence classification.

All GGUF files were converted from the original sharded safetensors weights using llama.cpp at native F16 precision, followed by quantization into standard k-quant variants.

> [!CAUTION]

> ONCA 3.0 is a research model, not a medical device, clinical decision system, or staging authority. It is intended strictly for research evaluation and structured information extraction workflows.

---

Available Files and Quantizations

| File | Quant Type | Size | Description / Recommendation |

| :--- | :--- | :--- | :--- |

| onca-3.0-12b-F16.gguf | F16 | 22.20 GB | Full precision base conversion. Highest fidelity reference weights. |

| onca-3.0-12b-Q8_0.gguf | Q8_0 | 11.80 GB | Near-lossless 8-bit quantization. Recommended for rigorous research evaluation. |

| onca-3.0-12b-Q6_K.gguf | Q6_K | 9.11 GB | Excellent retention of structured generation accuracy with reduced VRAM demands. |

| onca-3.0-12b-Q5_K_M.gguf | Q5_K_M | 7.96 GB | Strong balance of memory efficiency and schema adherence. |

| onca-3.0-12b-Q4_K_M.gguf | Q4_K_M | 6.87 GB | Fast 4-bit quant. Recommended default for local workstation inference and desktop GPUs. |

| onca-3.0-12b-Q3_K_M.gguf | Q3_K_M | 5.67 GB | Lower memory footprint when running on constrained devices. |

| onca-3.0-12b-Q2_K.gguf | Q2_K | 4.50 GB | Maximum compression for testing on memory-limited hardware. |

---

Technical Specifications

• Foundation Architecture: Gemma 4 Unified (Gemma4UnifiedForConditionalGeneration)
• Modality: Text only (multimodal vision/audio heads are not active in this checkpoint)
• Parameter Count: 11.95B parameters
• Context Window: Up to 262,144 tokens (evaluation sequences capped at 4,096 tokens)
• Base Precision: float16
• License: Apache-2.0

---

Supported Oncology Tasks and Schemas

Every query should specify its task in the prompt. The model outputs compact JSON without conversational filler or hidden chain-of-thought tokens:

1. Pathological T Staging (tnm_t):

```json

{"task":"tnm_t","label":"T2"}

```

1. Pathological N Staging (tnm_n):

```json

{"task":"tnm_n","label":"N1"}

```

1. Pathological M Staging (tnm_m):

```json

{"task":"tnm_m","label":"M0"}

```

1. Trial-Space Triage (trial_space):

```json

{"task":"trial_space","decision":"CONSIDER"}

```

1. Trial-Criterion Reasoning (trial_criterion):

```json

{"task":"trial_criterion","criterion_type":"inclusion","native_decision":"included","canonical_decision":"SATISFIED","evidence_sentence_ids":[2,4]}

```

1. CIViC Molecular Evidence (civic_evidence):

```json

{"task":"civic_evidence","evidence_type":"PREDICTIVE","evidence_level":"A","evidence_direction":"SUPPORTS","clinical_significance":"SENSITIVITY_RESPONSE"}

```

---

Prompt Format

This model uses Gemma 4 conversational turn markers:

<bos><start_of_turn>user
You are an oncology research assistant that outputs only valid JSON matching the requested schema.

Task: tnm_t
Cancer type: Pancreatic ductal adenocarcinoma
Pathology excerpt: Tumor measures 3.2 cm in greatest dimension, limited to the pancreas.<end_of_turn>
<start_of_turn>model
{"task":"tnm_t","label":"T2"}<end_of_turn>

---

Quickstart Guide

Option 1: llama.cpp CLI

Run local inference directly with llama-cli:

llama-cli -m onca-3.0-12b-Q4_K_M.gguf \
  -p "<bos><start_of_turn>user\nYou are an oncology research model. Return only JSON.\n\nTask: tnm_t\nCancer type: Lung adenocarcinoma\nReport: Invasive adenocarcinoma measuring 2.4 cm.<end_of_turn>\n<start_of_turn>model\n" \
  -n 128 \
  --temp 0.0 \
  -ngl 99

Option 2: llama.cpp Server

Launch an OpenAI-compatible HTTP server:

llama-server -m onca-3.0-12b-Q4_K_M.gguf \
  --port 8080 \
  -c 8192 \
  -ngl 99

Query the endpoint:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "onca-3.0-12b",
    "messages": [
      {"role": "user", "content": "Task: trial_space\nPatient: 62yo male with metastatic CRC.\nTrial space: First-line metastatic colorectal cancer immunotherapy."}
    ],
    "temperature": 0.0
  }'

Option 3: Ollama

Create a custom Ollama model using a Modelfile:

FROM ./onca-3.0-12b-Q4_K_M.gguf

TEMPLATE """<bos>{{- range .Messages }}
<start_of_turn>{{ .Role }}
{{ .Content }}<end_of_turn>
{{- end }}
<start_of_turn>model
"""

PARAMETER stop "<end_of_turn>"
PARAMETER temperature 0.0

Build and run:

ollama create onca-12b -f Modelfile
ollama run onca-12b "Task: tnm_m\nCancer type: Breast carcinoma\nReport: No evidence of distant metastasis."

Option 4: LM Studio

1. Place onca-3.0-12b-Q4_K_M.gguf inside your local LM Studio directory.
2. Select the model from the model dropdown.
3. Configure the chat preset for Gemma syntax (<start_of_turn>user / <start_of_turn>model).
4. Set temperature to 0.0 for deterministic structured JSON output.

---

Original Model and Citation

• Original Hugging Face Repository: Joesh1/onca-3.0-12b
• Base Foundation Model: google/gemma-4-12B
• Primary Research Focus: Structured clinical NLP, pathological staging extraction, and precision oncology trial screening.