Add pipeline tag, library name and content from Github README

This PR adds the `pipeline_tag` and `library_name` to the model card metadata, improving discoverability and clarity. It also adds content from the Github README to provide more information about the model, including the architecture, installation instructions, getting started guide, and model zoo.

README.md

@@ -1,3 +1,158 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+pipeline_tag: image-feature-extraction
+library_name: transformers
+---
+
+# MedM-VL: What Makes a Good Medical LVLM?
+
+[![arXiv](https://img.shields.io/badge/Arxiv-2504.04323-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2504.04323) [![hf_space](https://img.shields.io/badge/🤗-%20Open%20In%20HF-blue.svg)](https://huggingface.co/collections/shiym2000/medm-vl-67f739e50d344d712eb7b010) [![License](https://img.shields.io/badge/License-Apache%202.0-yellow)](./LICENSE)
+
+![architecture](./assets/architecture.png)
+
+MedM-VL is a **modular**, LLaVA-based codebase for medical LVLMs, supporting flexible customization of encoders, connectors, and LLMs.
+
+MedM-VL focuses on **small-scale** medical LVLMs, designed for **direct deployment** in real-world medical scenarios or **efficient fine-tuning** on downstream tasks.
+
+## :newspaper: News
+
++ **[2025.04.10]**: The model weights (v1.0) have been uploaded to Hugging Face.
+  + [shiym2000/MedM-VL-2D-3B-en · Hugging Face](https://huggingface.co/shiym2000/MedM-VL-2D-3B-en)
+  + [shiym2000/MedM-VL-CT-Chest-3B-en · Hugging Face](https://huggingface.co/shiym2000/MedM-VL-CT-Chest-3B-en)
+  + [shiym2000/MedM-CLIP-CT · Hugging Face](https://huggingface.co/shiym2000/MedM-CLIP-CT)
++ **[2025.04.06]**: The technical report has been released on arXiv.
+  + [[2504.04323] MedM-VL: What Makes a Good Medical LVLM?](https://arxiv.org/abs/2504.04323)
++ **[2024.12.19]**: The complete code has been released on GitHub.
+
+## :sparkles: Features
+
+MedM-VL (v1.0: single image input, more details on Hugging Face)
++ [shiym2000/MedM-VL-2D-3B-en · Hugging Face](https://huggingface.co/shiym2000/MedM-VL-2D-3B-en): Trained on **2D** medical images and **English** medical texts.
++ [shiym2000/MedM-VL-CT-Chest-3B-en · Hugging Face](https://huggingface.co/shiym2000/MedM-VL-CT-Chest-3B-en): Trained on **3D** chest CT volumes and **English** medical texts.
+
+## :package: Installation
+
+``` bash
+# 1. clone and navigate
+git clone https://github.com/MSIIP/MedM-VL.git
+cd MedM-VL
+
+# 2. create a conda environment, activate it and install packages
+conda create -n medm python=3.10
+conda activate medm
+pip install -r requirements.txt
+pip install flash-attn --no-build-isolation
+```
+
+## :rocket: Getting Started
+
+If you are confused about some parameters during usage, please refer to [Parameter Interpretation](docs/param_interpretation.md).
+
+### 1. Train a general medical LVLM from scratch
+
+``` bash
+# For 2D medical LVLMs
+# 1. pre-train (annotation format: docs/example_2d_pretrain.json)
+bash scripts/train/MedM-VL-2D/pretrain_en.sh
+# 2. fine-tune (annotation format: docs/example_2d_finetune.json)
+bash scripts/train/MedM-VL-2D/finetune_en.sh
+
+# For 3D medical LVLMs
+# 1. pre-train (annotation format: docs/example_3d_pretrain.json)
+bash scripts/train/MedM-VL-CT-Chest/pretrain_en.sh
+# 2. fine-tune (annotation format: docs/example_3d_finetune.json)
+bash scripts/train/MedM-VL-CT-Chest/finetune_en.sh
+
+# In fact, there is no difference in the annotation file format between
+# pre-training and fine-tuning. The former is from image-text pairs
+# while the latter refers to instruction tuning data.
+```
+
+### 2. Fine-tune a specialized medical LVLM with pre-trained weights
+
+``` bash
+# For 2D medical LVLMs
+# 1. download weights from Hugging Face
+pip install -U huggingface_hub
+huggingface-cli download --resume-download shiym2000/MedM-VL-2D-3B-en --local-dir work_dirs/MedM-VL-2D-3B-en
+# 2. fine-tune using LoRA (annotation format: docs/example_2d_finetune.json)
+bash scripts/train/finetune_2d.sh
+
+# For 3D medical LVLMs
+# 1. download weights from Hugging Face
+pip install -U huggingface_hub
+huggingface-cli download --resume-download shiym2000/MedM-VL-CT-Chest-3B-en --local-dir work_dirs/MedM-VL-CT-Chest-3B-en
+# 2. fine-tune using LoRA (annotation format: docs/example_3d_finetune.json)
+bash scripts/train/finetune_3d.sh
+
+# You can choose full or LoRA fine-tuning based on available GPU memory.
+```
+
+### 3. Inference
+
+``` bash
+# For 2D medical LVLMs
+# inference (annotation format: docs/example_2d_inference.json)
+bash scripts/eval/inference_2d.sh
+
+# For 3D medical LVLMs
+# inference (annotation format: docs/example_3d_inference.json)
+bash scripts/eval/inference_3d.sh
+
+# Compared to `finetune.json``, `conversations` in `inference.json` lacks
+# the final response, which will be generated by the model.
+```
+
+### 4. Demo
+
+``` bash
+# Launch a Gradio demo locally.
+bash scripts/playground.sh
+```
+
+## :robot: Model Zoo
+
+<table>
+  <tr align="center">
+    <td><b>Encoder</b></td>
+    <td><b>Connector</b></td>
+    <td><b>LLM</b></td>
+  </tr>
+  <tr valign="top">
+    <td>
+      <li><a href="https://arxiv.org/abs/2103.00020"> CLIP (2021) </a></li>
+      <li><a href="https://arxiv.org/abs/2303.15343"> SigLIP (2023) </a></li>
+      <li><a href="https://arxiv.org/abs/2404.00578"> M3D-CLIP (2023) </a></li>
+      <li><a href="https://huggingface.co/collections/shiym2000/medm-clip-67f7afd8a3dbcff656466805"> MedM-CLIP <a></li>
+    </td>
+    <td>
+      <li> MLP </li>
+      <li> Spatial Pooling </li>
+      <li> Attention Pooling </li>
+    </td>
+    <td>
+      <li><a href="https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/"> Phi-2 (2023) </a></li>
+      <li><a href="https://arxiv.org/abs/2404.14219"> Phi-3 (2024) </a></li>
+      <li><a href="https://arxiv.org/abs/2412.15115"> Qwen2.5 (2024) </a></li>
+      <li><a href="https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/"> Llama-3.2 (2024) </a></li>
+    </td>
+  </tr>
+</table>
+
+## :book: Citation
+
+``` bibtex
+@article{shi2025medm,
+  title={MedM-VL: What Makes a Good Medical LVLM?},
+  author={Shi, Yiming and Yang, Shaoshuai and Zhu, Xun and Wang, Haoyu and Li, Miao and Wu, Ji},
+  journal={arXiv preprint arXiv:2504.04323},
+  year={2025}
+}
+```
+
+## :heart: Acknowledgements
+
+We would like to express our gratitude to the following resources:
++ [**TinyLLaVA_Factory**](https://github.com/TinyLLaVA/TinyLLaVA_Factory) - An open-source modular codebase for small-scale large multimodal models (LMMs).
+
+Code: https://github.com/MSIIP/MedM-VL