CineMR

CineMR is a vision–language model for cardiac MRI visual question answering, built on Qwen3-VL-8B-Instruct. We first supervised-fine-tuned (SFT) on structured cardiac VQA, then applied GRPO (Group Relative Policy Optimization) with a domain-specific reward in EasyR1.

This Hub release is the merged full-weight GRPO export — a single model.safetensors (~16 GB) plus tokenizer and Qwen3VLProcessor configs, ready for inference with transformers.

Training data: ai-mind-lab/CineMR (ACDC, M&Ms, M&Ms-2 cardiac MRI VQA with optional tool-use supervision).

Authors

Kunyang Li^1,†, Hai Nguyen^1,2,†, Joshua Lowe^1,†, Chenguang Zhao³, Peace C. Madueme³, Mehdi Hedjazi Moghari⁴, Mubarak Shah^1,§, Pegah Khosravi^1,2,§, Yuzhang Shang^1,§

¹ Institute for Artificial Intelligence, University of Central Florida ² Department of Clinical Sciences, College of Medicine, University of Central Florida ³ Nemours Children's Health, Orlando, Florida ⁴ West Virginia University Medicine Children's Hospital, Morgantown, West Virginia

^† Co-first author  ·  ^§ Corresponding author

Model summary

Architecture	Qwen3VLForConditionalGeneration (qwen3_vl)
Parameters	~8.8B
Precision	bfloat16 (dtype in config.json)
Base model	Qwen/Qwen3-VL-8B-Instruct
SFT init	Merged SFT checkpoint on cardiac VQA
RL algorithm	GRPO (EasyR1), LoRA r=64 / α=128 on language layers (vision frozen during LoRA)
Transformers	Exported with transformers 5.8.x

Intended use

• Answer questions about cardiac cine / volumetric MRI when given frame images or short video clips.
• Supports the structured answer format used in CineMR training: final answers in \boxed{...} and optional <tool_call> blocks for measurement-style reasoning.

Not for clinical decision-making. This model is a research artifact; outputs must not be used for diagnosis or treatment without expert review and appropriate validation.

Contents

Artifact	Purpose
model.safetensors	Full merged weights (SFT + GRPO LoRA), single shard
config.json	Model architecture and dtype
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, …	Text tokenizer
preprocessor_config.json, video_preprocessor_config.json	Image / video preprocessing for Qwen3VLProcessor
chat_template.jinja	Chat formatting
generation_config.json	Default generation settings

Loading

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

repo_id = "ai-mind-lab/CineMR"  # or a local path to this directory

model = AutoModelForVision2Seq.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(repo_id, trust_remote_code=True)

Example: single-image VQA

from PIL import Image

image = Image.open("path/to/frame.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": "What is the left ventricular ejection fraction?"},
        ],
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=2048,
        do_sample=True,
        temperature=0.7,
        repetition_penalty=1.15,
    )

print(processor.decode(out[0], skip_special_tokens=True))

Use the same trust_remote_code=True and bfloat16 settings as in training. For evaluation, match the CineMR prompt template and decoding settings used in your eval script.

Decoding note. Pure greedy decoding (do_sample=False, no repetition penalty) can drive this checkpoint into repetition collapse (a single reasoning sentence repeated until the token cap, with no \boxed{} answer or <tool_call> emitted). The evaluation numbers below were produced with do_sample=True, temperature=0.7, repetition_penalty=1.15, no_repeat_ngram_size=0, max_new_tokens=2048, and 4 sampled rollouts per prompt. Use a repetition penalty (≈1.1–1.2) for stable outputs.

Training procedure (summary)

1. SFT on CineMR JSONL (train split) starting from Qwen3-VL-8B-Instruct; weights merged to a full transformers checkpoint.
2. GRPO in EasyR1 with:
   • Reward: reward_cardiac_vqa.py (compute_score) — accuracy on \boxed{} answers plus format / tool-use terms.
   • Rollout: vLLM, n=2 samples per prompt, max response length 1024.
   • Actor LR 1e-5, KL coefficient 0.01, global batch size 4.
   • Image frames from preprocessed cine caches; pixel budget aligned with Qwen3-VL (min/max pixels in training config).

LoRA weights are merged into the base checkpoint for Hub deployment.

Evaluation

Evaluated on the CineMR test split (1,191 samples) with the training prompt template, 4 sampled rollouts per prompt (temperature=0.7, repetition_penalty=1.15, max_new_tokens=2048). pass@k is the fraction of items with ≥1 correct rollout; mean rollout acc averages correctness over all rollouts.

Metric	Value
Mean rollout accuracy	0.378
pass@4 (any correct)	0.553
ROUGE-L	0.620
BERTScore F1	0.974
Ground-truth satisfied	0.370

Accuracy by reasoning layer (pass@4): L1 0.306, L2 0.771, L3 0.906, L4 0.614, L5 0.314, L6 0.563. By clinical stage: phenotype (L1–L4) pass@4 0.572, etiology (L5–L6) pass@4 0.353.

Tool use: tool-decision accuracy 0.891 (precision 0.999), tool recall on required items 99.8% (predicted names ⊆ expected 100%), trace/JSON format validity 99.4%, tool-name set-match 0.862, argument accuracy 0.879.

Numbers are from this GRPO checkpoint evaluated with the project eval_sft pipeline. Re-run on a held-out test split before drawing conclusions; the small GRPO validation set is used only for checkpoint tracking.

Limitations

• Trained on public cardiac MRI challenge-style corpora (ACDC, M&Ms, M&Ms-2); generalization to other scanners, sequences, or pathologies is not guaranteed.
• GRPO training used a small validation set for checkpoint tracking; prefer held-out test evaluation before drawing conclusions.
• Tool-use formatting in outputs may be inconsistent unless prompts and decoding match training.

License

This model inherits terms from Qwen3-VL (Apache 2.0) and your use of CineMR data and any dataset/challenge restrictions (ACDC, M&Ms, etc.). Use only for lawful research purposes.

Citation

If you use CineMR, please cite the base Qwen3-VL model and acknowledge the CineMR dataset and cardiac imaging sources:

@misc{cinemr_qwen3vl8b_grpo,
  title        = {CineMR: Augmenting Vision-Language Models with Tool-Integrated Reasoning for Quantitative Cardiac MRI Diagnosis},
  author       = {Li, Kunyang and Nguyen, Hai and Lowe, Joshua and Zhao, Chenguang and Madueme, Peace C. and Moghari, Mehdi Hedjazi and Shah, Mubarak and Khosravi, Pegah and Shang, Yuzhang},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/ai-mind-lab/CineMR}},
  note         = {GRPO checkpoint; dataset at huggingface.co/datasets/ai-mind-lab/CineMR},
}

@article{qwen3vl,
  title  = {Qwen3-VL Technical Report},
  author = {Qwen Team},
  year   = {2025},
}