Joesh1/onca-2.0-12b

ONCA 2.0

_{Strict automatic scores on the unchanged 1,309-example primary holdout; compare values within each task panel.}

Summary

ONCA 2.0 is an open oncology language model for trial screening, clinical reasoning, pathology extraction, and variant evidence interpretation. It builds on google/gemma-4-12B-it with continued supervised fine-tuning on a provenance-labeled oncology corpus while retaining the four-task ONCA 1.5 evaluation contract.

This repository contains the merged BF16 reference checkpoint exported on 2026-06-14. The model is specialized for pancreatic cancer and oncology-adjacent research workflows and performs best with tightly scoped prompts and explicit output formats.

At a Glance

Field	Value
Release	BF16 reference release
Base model	google/gemma-4-12B-it
Architecture	Gemma 4 unified 12B (Gemma4UnifiedForConditionalGeneration)
Context window	262,144 tokens
Training recipe	Continued SFT with merged adapter export
Domain focus	Pancreatic cancer and oncology research
Weights	Seven safetensors shards

Task	Headline metric	ONCA 2.0
Trial Screening	Accuracy	0.8240
Clinical Reasoning	Outcome-label accuracy	0.6761
Pathology Extraction	Overall field exact match	0.4634
Variant Evidence	Clinical-significance macro-F1	0.5427

ONCA 2.0 improves over ONCA 1.5 on trial screening, clinical reasoning, and variant evidence. Pathology extraction remains its primary weakness.

Quick Start

Use a recent Transformers release with Gemma 4 support.

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "Joesh1/onca-2.0-12B"

processor = AutoProcessor.from_pretrained(model_id)
tokenizer = processor.tokenizer
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    torch_dtype="auto",
    device_map="auto",
)

# Example: criterion-aware trial screening with structured output
messages = [{
    "role": "user",
    "content": (
        "Patient: metastatic pancreatic adenocarcinoma; ECOG 1; "
        "no prior metastatic-line therapy. Trial: metastatic PDAC, ECOG 0-1, "
        "no prior metastatic-line therapy. Return JSON with keys eligible, "
        "reason, and missing_information."
    ),
}]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=160, do_sample=False)
answer = tokenizer.decode(
    outputs[0, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
)
print(answer)

Use the included tokenizer and chat template. For structured workflows, request exact fields, provide all relevant criteria, ask for explicit uncertainty, and prefer deterministic decoding.

Training Scope

The active training corpus contains 25,302 examples. Validation, test, and primary-holdout sets retain the ONCA 1.5 four-task evaluation contract.

Task family	Train	Original	Generated	Val	Test	Holdout
Trial Screening	10,921	10,921	0	608	608	608
Clinical Reasoning	3,647	3,146	501	174	176	176
Pathology Extraction	4,559	333	4,226	410	400	400
Variant Evidence	6,175	2,191	3,984	116	125	125
Total	25,302	16,591	8,711	1,308	1,309	1,309

Related Releases

• onca-2.0-12B: BF16 reference release (this page).
• onca-2.0-12B-INT8: 8-bit BitsAndBytes release.
• onca-2.0-12B-INT4: 4-bit BitsAndBytes release.
• onca-2.0-12B-GGUF: llama.cpp-compatible GGUF collection.

Limitations

• This is a research model, not a clinical decision system.
• Outputs require review by qualified experts before real-world use.
• Structured or parser-valid output does not guarantee factual correctness.
• The model is specialized for pancreatic cancer and oncology-adjacent workflows rather than broad general medicine.
• Performance varies by task; pathology extraction remains the weakest evaluated area.

Citation

A formal ONCA 2.0 citation will be added with the accompanying manuscript. Until then, cite this model repository and the exact version used.

Acknowledgements

ONCA 2.0 continues the ONCA project lineage and builds on Google Gemma and the open-data contributors whose datasets supported training and evaluation.