| --- |
| license: apache-2.0 |
| base_model: |
| - Qwen/Qwen3-VL-4B-Instruct |
| - Qwen/Qwen3-VL-8B-Instruct |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| tags: |
| - medical |
| - vision-language |
| - reinforcement-learning |
| - qwen3-vl |
| datasets: |
| - MBZUAI/medix-rl-data |
| --- |
| |
| # MediX-R1 (reproduction), 4B and 8B |
|
|
| Two Qwen3-VL checkpoints trained with reinforcement learning following the MediX-R1 recipe. They are our reproduction of |
| MediX-R1, the "MediX-R1-4B" and "MediX-R1-8B" rows of the main results table in our paper. Each model is in its own folder. |
|
|
| | Folder | Base model | VQA | QA | Macro | Micro | |
| |---|---|---|---|---|---| |
| | [MediX-R1-4B](MediX-R1-4B) | Qwen/Qwen3-VL-4B-Instruct | 49.2 | 70.5 | 63.0 | 53.8 | |
| | [MediX-R1-8B](MediX-R1-8B) | Qwen/Qwen3-VL-8B-Instruct | 49.3 | 74.4 | 65.5 | 56.0 | |
|
|
| Accuracy (%) over 6 medical VQA and 11 medical text QA benchmarks, greedy decoding, answers scored by three LLM judges. |
| The README in each folder has the training recipe, the prompt format and the per-benchmark results. |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from huggingface_hub import snapshot_download |
| from transformers import AutoProcessor, Qwen3VLForConditionalGeneration |
| |
| name = "MediX-R1-8B" # or "MediX-R1-4B" |
| path = snapshot_download("Shayan0179/Medix-R1", allow_patterns=f"{name}/*") + f"/{name}" |
| model = Qwen3VLForConditionalGeneration.from_pretrained(path, dtype=torch.bfloat16, device_map="auto") |
| processor = AutoProcessor.from_pretrained(path) |
| |
| PROMPT = open(f"{path}/prompt.txt").read() # the training instructions, ending with "Question: \n" |
| messages = [{"role": "user", "content": [ |
| {"type": "text", "text": PROMPT}, |
| {"type": "image", "image": "chest_xray.png"}, |
| {"type": "text", "text": "Is there a pleural effusion?"}, |
| ]}] |
| inputs = processor.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, |
| return_dict=True, return_tensors="pt").to(model.device) |
| out = model.generate(**inputs, max_new_tokens=2048, do_sample=False) |
| print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0]) |
| ``` |
|
|
| The model answers with a modality tag, then its reasoning in `<thinking>...</thinking>`, then the final answer in |
| `<answer>...</answer>`. |
|
|
| ## License |
|
|
| Apache 2.0, following the base models. |
|
|
