| --- |
| license: apache-2.0 |
| language: |
| - en |
| base_model: |
| - Qwen/Qwen3-VL-4B-Instruct |
| pipeline_tag: image-text-to-text |
| library_name: transformers |
| tags: |
| - medical |
| - multimodal |
| - vision-language |
| - radiology |
| - pathology |
| - reasoning |
| - reinforcement-learning |
| --- |
| |
| # NetraV4 |
|
|
| NetraV4 is a 4B-parameter medical vision-language model that reasons step by step before answering. It reads |
| medical images (radiology, pathology, clinical photos, figures) together with text and is tuned for medical visual |
| question answering and medical exam-style reasoning. |
|
|
| It is built on [Qwen3-VL-4B](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct) and trained with reinforcement |
| learning on medical reasoning tasks. |
|
|
| > **Not a medical device.** NetraV4 is a research model. Its answers can be wrong, and it must not be used for |
| > diagnosis, treatment or any clinical decision without review by a qualified professional. |
|
|
| ## Highlights |
|
|
| - Ahead of published open models under 15B parameters on most of the benchmarks below, including the 3.5× |
| larger HealthGPT-14B, Lingshu-7B and MedGemma-4B. |
| - Above the published GPT-4.1 and Claude Sonnet 4 results on PMC-VQA, SLAKE (closed) and VQA-RAD (closed), and level |
| with GPT-4o on GMAI-MMBench. |
| - The newest frontier models (GPT-5, Gemini 3 Pro) remain far ahead on expert-level reasoning (MedXpertQA, MedQA). |
|
|
| ## Training |
|
|
| - **Base:** Qwen3-VL-4B (Apache-2.0), with medical domain tuning. |
|
|
| - **Data:** public medical VQA and medical reasoning datasets. The official test splits of every benchmark below were |
| held out from training; benchmark questions were checked against the training data for text and image overlap. |
|
|
| ## Results |
|
|
| Accuracy (%). NetraV4 was evaluated with greedy decoding, up to 4,096 new tokens, and its own prompt format (below). |
| The other columns are **published** results from each model's paper or report, obtained with their own prompts and |
| settings. They are an indicative comparison, not a same-setup head-to-head. |
|
|
| ### Medical vision |
|
|
| | Benchmark | NetraV4 (4B) | Frontier models (published) | Open models (published) | |
| |---|---|---|---| |
| | MedXpertQA-MM | 26.5 | GPT-5 71.2 · GPT-4.1 45.2 · Claude Sonnet 4 43.3 | Lingshu-7B 26.7 · HealthGPT-14B 24.7 · MedGemma-4B 22.3 · Qwen2.5-VL-7B 22.3 | |
| | GMAI-MMBench ¹ | 54.0 | GPT-4o 54.0 | — | |
| | VQA-RAD (closed) ² | 74.3 | GPT-5 70.9 · Claude Sonnet 4 67.6 · GPT-4.1 65.0 | MedGemma-4B 72.5 · Lingshu-7B 67.9 · HuatuoGPT-V-7B 67.0 · HealthGPT-14B 65.0 | |
| | SLAKE (closed) ² | 76.2 | GPT-4.1 72.2 · Claude Sonnet 4 70.6 | Lingshu-7B 83.1 · MedGemma-4B 76.4 · HealthGPT-14B 66.1 | |
| | PMC-VQA | 57.9 | GPT-4.1 55.2 · Claude Sonnet 4 54.4 | HealthGPT-14B 56.4 · Lingshu-7B 56.3 · MedGemma-4B 49.9 | |
| | MMMU-Med | 64.1 | GPT-4.1 75.2 · Claude Sonnet 4 74.6 | InternVL3-8B 59.2 · Lingshu-7B 54.0 · HealthGPT-14B 49.6 · MedGemma-4B 43.7 | |
| | VQA-RAD (open) ³ | 39.2 | — | — | |
| | SLAKE (open) ³ | 51.4 | — | — | |
|
|
| ### Medical text |
|
|
| | Benchmark | NetraV4 (4B) | Frontier models (published) | Open models (published) | |
| |---|---|---|---| |
| | MedQA (4-option) | 76.8 | Gemini 3 Pro 96.0 · GPT-5 95.8 · Claude Sonnet 4 92.1 · GPT-4.1 89.1 | MedGemma-27B 87.7 · HealthGPT-14B 66.2 · MedGemma-4B 64.4 · Lingshu-7B 63.3 | |
| | MedMCQA | 63.5 | Claude Sonnet 4 79.3 · GPT-4.1 77.7 | HealthGPT-14B 63.4 · Lingshu-7B 55.9 · MedGemma-4B 55.7 | |
| | PubMedQA | 77.6 | Claude Sonnet 4 78.6 · GPT-4.1 75.6 | Lingshu-7B 76.6 · MedGemma-4B 73.4 · HealthGPT-14B 68.0 | |
| | MMLU-Med | 83.8 | GPT-5 96.1 · Claude Sonnet 4 91.3 · GPT-4.1 89.6 | HealthGPT-14B 80.2 · Lingshu-7B 74.5 · MedGemma-4B 70.0 | |
| | MedXpertQA-Text | 17.3 | GPT-5 55.9 · Claude Sonnet 4 33.6 · GPT-4.1 30.9 | Lingshu-7B 16.5 · MedGemma-4B 14.2 · HealthGPT-14B 11.3 | |
|
|
| ¹ GMAI-MMBench: NetraV4 on the validation split (the test answers are not public); GPT-4o's published figure is on |
| the test split. |
| ² Closed (yes/no) questions only. Published VQA-RAD/SLAKE figures from the Lingshu report combine closed and open |
| questions. |
| ³ Open questions: mean token recall of the reference answer inside the model's `<answer>` tag; not comparable with |
| published accuracy figures. |
|
|
| **Sources of published results:** GPT-5: Wang et al. 2025, *Capabilities of GPT-5 on Multimodal Medical Reasoning* |
| (arXiv:2508.08224; MedXpertQA and MMLU-Med as the mean of the reported sub-scores). Gemini 3 Pro: Vals AI MedQA |
| leaderboard. GPT-4.1, Claude Sonnet 4, Lingshu, HealthGPT-14B, HuatuoGPT-V, InternVL3, Qwen2.5-VL: Lingshu technical |
| report (arXiv:2506.07044). MedGemma: MedGemma technical report (arXiv:2507.05201). GPT-4o on GMAI-MMBench: |
| GMAI-MMBench (NeurIPS 2024). |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from PIL import Image |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| |
| model_id = "dipankar500/NetraV4" |
| model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto") |
| processor = AutoProcessor.from_pretrained(model_id) |
| |
| question = "Is there evidence of pleural effusion?" |
| instruction = ("Think step-by-step and enclose your reasoning in <think>...</think> tags. " |
| "Then provide your answer in <answer>...</answer> tags.") |
| messages = [{"role": "user", "content": [ |
| {"type": "image", "image": Image.open("chest_xray.png")}, |
| {"type": "text", "text": question + "\n" + instruction}, |
| ]}] |
| prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) + "<think>\n" |
| inputs = processor(text=[prompt], images=[messages[0]["content"][0]["image"]], return_tensors="pt").to(model.device) |
| output = model.generate(**inputs, max_new_tokens=4096, do_sample=False) |
| print(processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)) |
| ``` |
|
|
| For multiple-choice questions, list the options after the question, one per line (`A. ...`, `B. ...`); the model |
| answers with the option inside `<answer>...</answer>`. The model also works with vLLM. |
|
|
| ## Limitations |
|
|
| - Expert-level clinical reasoning (MedXpertQA) remains far below frontier models. |
| - Long reasoning: some answers need more than 4,096 tokens and are cut off. |
| - Trained and evaluated mainly in English. |
| - May produce confident but incorrect statements; always verify against clinical sources. |
|
|
| ## License |
|
|
| Apache-2.0, following the base model Qwen3-VL-4B (Apache-2.0). |
|
|
