CineMR

CineMR is a vision–language model for cardiac MRI visual question answering, built on Qwen3-VL-8B-Instruct. We first supervised-fine-tuned (SFT) on structured cardiac VQA, then applied GRPO (Group Relative Policy Optimization) with a domain-specific reward in EasyR1.

This Hub release is the merged full-weight GRPO export — a single model.safetensors (~16 GB) plus tokenizer and Qwen3VLProcessor configs, ready for inference with transformers.

Training data: ai-mind-lab/CineMR (ACDC, M&Ms, M&Ms-2 cardiac MRI VQA with optional tool-use supervision).

Authors

Kunyang Li^1,†, Hai Nguyen^1,2,†, Joshua Lowe^1,†, Chenguang Zhao³, Peace C. Madueme³, Mehdi Hedjazi Moghari⁴, Mubarak Shah^1,§, Pegah Khosravi^1,2,§, Yuzhang Shang^1,§

¹ Institute for Artificial Intelligence, University of Central Florida ² Department of Clinical Sciences, College of Medicine, University of Central Florida ³ Nemours Children's Health, Orlando, Florida ⁴ West Virginia University Medicine Children's Hospital, Morgantown, West Virginia

^† Co-first author  ·  ^§ Corresponding author

Model summary

Architecture	Qwen3VLForConditionalGeneration (qwen3_vl)
Parameters	~8.8B
Precision	bfloat16 (dtype in config.json)
Base model	Qwen/Qwen3-VL-8B-Instruct
SFT init	Merged SFT checkpoint on cardiac VQA
RL algorithm	GRPO (EasyR1), LoRA r=64 / α=128 on language layers (vision frozen during LoRA)
Transformers	Exported with transformers 5.8.x

Intended use

• Answer questions about cardiac cine / volumetric MRI when given frame images or short video clips.
• Supports the structured answer format used in CineMR training: final answers in \boxed{...} and optional <tool_call> blocks for measurement-style reasoning.

Not for clinical decision-making. This model is a research artifact; outputs must not be used for diagnosis or treatment without expert review and appropriate validation.

Contents

Artifact	Purpose
model.safetensors	Full merged weights (SFT + GRPO LoRA), single shard
config.json	Model architecture and dtype
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, …	Text tokenizer
preprocessor_config.json, video_preprocessor_config.json	Image / video preprocessing for Qwen3VLProcessor
chat_template.jinja	Chat formatting
generation_config.json	Default generation settings

Loading

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

repo_id = "ai-mind-lab/CineMR"  # or a local path to this directory

model = AutoModelForVision2Seq.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)

Example: single-image VQA

from PIL import Image

image = Image.open("path/to/frame.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "What is the left ventricular ejection fraction?"},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=2048,
        do_sample=True,
        temperature=0.7,
        repetition_penalty=1.15,
    )

print(processor.decode(out[0], skip_special_tokens=True))

Decoding note. Pure greedy decoding (do_sample=False, no repetition penalty) can drive this checkpoint into repetition collapse (a single reasoning sentence repeated until the token cap, with no \boxed{} answer or <tool_call> emitted). The evaluation numbers below were produced with do_sample=True, temperature=0.7, repetition_penalty=1.15, no_repeat_ngram_size=0, max_new_tokens=2048, and 4 sampled rollouts per prompt. Use a repetition penalty (≈1.1–1.2) for stable outputs.

Use the same trust_remote_code=True and bfloat16 settings as in training. For evaluation, match the CineMR prompt template and decoding settings used in your eval script.

Training procedure (summary)

1. SFT on CineMR JSONL (train split) starting from Qwen3-VL-8B-Instruct; weights merged to a full transformers checkpoint.
2. GRPO in EasyR1 with:
   • Reward: reward_cardiac_vqa.py (compute_score) — accuracy on \boxed{} answers plus format / tool-use terms.
   • Rollout: vLLM, n=2 samples per prompt, max response length 1024.
   • Actor LR 1e-5, KL coefficient 0.01, global batch size 4.
   • Image frames from preprocessed cine caches; pixel budget aligned with Qwen3-VL (min/max pixels in training config).

LoRA weights are merged into the base checkpoint for Hub deployment.

Evaluation

Evaluated on the CineMR held-out test split (n=3,320; distinct from the 1,191-sample validation split) with the training prompt template, 4 sampled rollouts per prompt (temperature=0.7, repetition_penalty=1.15, max_new_tokens=2048). pass@4 is the fraction of items with ≥1 correct rollout; mean rollout acc averages correctness over all 4 rollouts (this is the metric reported as "Acc." in the paper). All figures below are the mean ± SD over 3 independent evaluation runs of this exact checkpoint (same weights, stochastic decoding only — this captures rollout-sampling variance, not training-seed variance).

Metric	Value
Mean rollout accuracy	39.24% ± 0.09pp
pass@4 (any correct)	55.98%
Ground-truth satisfied (pass@1)	38.83%
ROUGE-L	0.620
BERTScore F1	0.974

Accuracy by reasoning layer (mean rollout acc): L1 10.87%, L2 73.22%, L3 66.90%, L4 55.84%, L5 11.46%, L6 37.25%. By clinical stage: L3–4 (clinical-criteria) 65.30% ± 0.66pp, L5–6 (full-diagnosis) 17.02% ± 1.41pp.

Tool use: tool-decision accuracy 89.61% (precision 100%), tool recall on required items 99.99%, redundant tool calls on optional items 0.00%, trace/JSON format validity 99.98%, tool-name set-match 88.28%, argument accuracy 88.28%, predicted names ⊆ expected 100%.

Limitations

• Trained on public cardiac MRI challenge-style corpora (ACDC, M&Ms, M&Ms-2); generalization to other scanners, sequences, or pathologies is not guaranteed.
• The evaluation above is held-out test-set accuracy (mean ± SD over 3 stochastic-decoding runs of this one checkpoint), which captures rollout-sampling variance but not training-seed variance — true multi-seed (independently trained) variance has not been measured.
• A non-VLM control that runs all six computational tools unconditionally and routes the outputs through the same clinical decision tree, with no VLM at all, currently exceeds this checkpoint's own diagnostic accuracy on the same test set. This checkpoint is a proof-of-concept that tool-integrated reasoning is necessary for this task, not evidence that it is the best way to obtain it — see the paper for the full discussion.
• Tool-use formatting in outputs may be inconsistent unless prompts and decoding match training.

License

This model inherits terms from Qwen3-VL (Apache 2.0) and your use of CineMR data and any dataset/challenge restrictions (ACDC, M&Ms, etc.). Use only for lawful research purposes.

Citation

If you use CineMR, please cite the base Qwen3-VL model and acknowledge the CineMR dataset and cardiac imaging sources:

@misc{cinemr_qwen3vl8b_grpo,
  title        = {CineMR: Augmenting Vision-Language Models with Tool-Integrated Reasoning for Quantitative Cardiac MRI Diagnosis},
  author       = {Li, Kunyang and Nguyen, Hai and Lowe, Joshua and Zhao, Chenguang and Madueme, Peace C. and Moghari, Mehdi Hedjazi and Shah, Mubarak and Khosravi, Pegah and Shang, Yuzhang},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/ai-mind-lab/CineMR}},
  note         = {GRPO checkpoint; dataset at huggingface.co/datasets/ai-mind-lab/CineMR},
}

@article{qwen3vl,
  title  = {Qwen3-VL Technical Report},
  author = {Qwen Team},
  year   = {2025},
}