|
|
---
|
|
|
license: apache-2.0
|
|
|
datasets:
|
|
|
- leduckhai/S-Chain
|
|
|
language:
|
|
|
- en
|
|
|
---
|
|
|
|
|
|
<p align="center"> |
|
|
<img src="./SChain_icon.png" alt="S-Chain logo" width="70"> |
|
|
</p> |
|
|
|
|
|
<h1 align="center">S-Chain: Structured Visual Chain-of-Thought for Medicine</h1> |
|
|
|
|
|
|
|
|
|
|
|
[](https://arxiv.org/abs/2510.22728) |
|
|
[](https://huggingface.co/leduckhai/S-Chain) |
|
|
[](https://huggingface.co/datasets/leduckhai/S-Chain) |
|
|
[](https://github.com/leduckhai/S-Chain/blob/main/DATASET_LICENSE.md) |
|
|
[](https://s-chain.github.io/) |
|
|
[](https://github.com/leduckhai/S-Chain) |
|
|
|
|
|
--- |
|
|
|
|
|
⭐ **If you find this project helpful, please consider giving it a [star on GitHub](https://github.com/leduckhai/S-Chain)!** |
|
|
|
|
|
--- |
|
|
|
|
|
<p align="center"> |
|
|
<a href="https://github.com/leduckhai" target="_blank"><strong>Khai Le-Duc</strong></a><sup>* 1,2✉</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=_NIyeykAAAAJ&hl=en" target="_blank"><strong>Duy M. H. Nguyen</strong></a><sup>* 3,4,24✉</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=5CbQH_kAAAAJ&hl=en" target="_blank"><strong>Phuong T. H. Trinh</strong></a><sup>* 5</sup>, |
|
|
<strong>Tien-Phat Nguyen</strong><sup>* 6</sup>, |
|
|
Nghiem T. Diep<sup>** 3</sup>, |
|
|
An Ngo<sup>** 7</sup>, |
|
|
Tung Vu<sup>** 8</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=trFdwLkAAAAJ&hl=en" target="_blank"><strong>Trinh Vuong</strong></a><sup>9</sup>, |
|
|
Anh-Tien Nguyen<sup>10,11</sup>, |
|
|
Mau Nguyen<sup>12</sup>, |
|
|
Van Trung Hoang<sup>13</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=IMryD1YAAAAJ&hl=en" target="_blank"><strong>Khai-Nguyen Nguyen</strong></a><sup>14</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=ZAuQIqwAAAAJ&hl=en" target="_blank"><strong>Hy Nguyen</strong></a><sup>15</sup>, |
|
|
Chris Ngo<sup>2</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=k_4zYecAAAAJ&hl=en" target="_blank"><strong>Anji Liu</strong></a><sup>16</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=Xs7cKMwAAAAJ&hl=en" target="_blank"><strong>Nhat Ho</strong></a><sup>17</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=Khifj_MAAAAJ&hl=en" target="_blank"><strong>Anne-Christin Hauschild</strong></a><sup>11</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=SmqouhIAAAAJ&hl=en" target="_blank"><strong>Khanh Xuan Nguyen</strong></a><sup>18</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=UrTlMiwAAAAJ&hl=en" target="_blank"><strong>Thanh Nguyen-Tang</strong></a><sup>19</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=cnncomYAAAAJ&hl=en" target="_blank"><strong>Pengtao Xie</strong></a><sup>20,21</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=v7i6Uz4AAAAJ&hl=en" target="_blank"><strong>Daniel Sonntag</strong></a><sup>3,22</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=23ZXZvEAAAAJ&hl=en" target="_blank"><strong>James Zou</strong></a><sup>23</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=p5vLzq0AAAAJ&hl=en" target="_blank"><strong>Mathias Niepert</strong></a><sup>4,24</sup>, |
|
|
<a href="https://scholar.google.com/citations?user=EQw8d9AAAAAJ&hl=en" target="_blank"><strong>Anh Totti Nguyen</strong></a><sup>25✉</sup> |
|
|
</p> |
|
|
|
|
|
|
|
|
<p align="center"> |
|
|
<em>*Co-first authors; order randomized | **Co-second authors</em><br> |
|
|
<em>✉ Corresponding Authors</em> |
|
|
</p> |
|
|
<details> |
|
|
<summary><strong>🎓 Affiliations</strong> (click to expand)</summary> |
|
|
1. University of Toronto, Canada |
|
|
2. Knovel Engineering Lab, Singapore |
|
|
3. German Research Centre for Artificial Intelligence |
|
|
4. University of Stuttgart, Germany |
|
|
5. Chonnam National University, South Korea |
|
|
6. Singapore University of Technology and Design |
|
|
7. Bucknell University, USA |
|
|
8. Concordia University, Canada |
|
|
9. Korea University |
|
|
10. Justus Liebig University Giessen, Germany |
|
|
11. University Medical Center Göttingen, Germany |
|
|
12. Japan Advanced Institute of Science and Technology |
|
|
13. Hue University, Vietnam |
|
|
14. College of William & Mary, USA |
|
|
15. Deakin University, Australia |
|
|
16. National University of Singapore |
|
|
17. University of Texas at Austin, USA |
|
|
18. University of California, Berkeley, USA |
|
|
19. New Jersey Institute of Technology, USA |
|
|
20. University of California San Diego, USA |
|
|
21. MBZUAI, UAE |
|
|
22. Oldenburg University, Germany |
|
|
23. Stanford University, USA |
|
|
24. Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany |
|
|
25. Auburn University, USA |
|
|
</details> |
|
|
--- |
|
|
<p align="center"> |
|
|
✨ In honor of |
|
|
<a href="https://en.wikipedia.org/wiki/H%E1%BA%A3i_Th%C6%B0%E1%BB%A3ng_L%C3%A3n_%C3%94ng" target="_blank"><strong>Hải Thượng Lãn Ông (海上懶翁) – Lê Hữu Trác (黎友晫)</strong></a>, |
|
|
the father of Vietnamese traditional medicine ✨ |
|
|
</p> |
|
|
## 🔍 What is S-Chain? |
|
|
S-Chain is the first large-scale dataset of **Structured Visual Chain-of-Thought (SV-CoT)**: |
|
|
each reasoning step is explicitly linked to visual evidence via bounding boxes. |
|
|
This enables training and evaluating *grounded* medical VLM reasoning instead of |
|
|
hallucinated justifications. |
|
|
- **12,000 medical images** with expert bounding boxes. |
|
|
- **700k+ VQA / rationale pairs** across **16 languages**. |
|
|
- Each sample: image, question, answer, stepwise SV-CoT, and per-step visual regions. |
|
|
|
|
|
We show that supervising VLMs with SV-CoT: |
|
|
- Improves interpretability |
|
|
- Improves grounding fidelity (reasoning actually points to the right region) |
|
|
- Improves robustness across models and languages |
|
|
|
|
|
<p align="center"> |
|
|
<img src="main_pipeline.png" alt="Alt text" width="1400"/> |
|
|
</p> |
|
|
|
|
|
|
|
|
## 📣 News |
|
|
|
|
|
- **[Oct 2025]** Updated experiment scripts and checkpoints for ExGra-Med and LLaVA-Med. See the [readme](architectures/Exgra-Med-CoT/README.md) for detailed instructions. |
|
|
- **[Oct 2025]** Dataset and project site released. |
|
|
|
|
|
## Citation |
|
|
If you find this work useful, please cite our paper: [https://arxiv.org/abs/2510.22728](https://arxiv.org/abs/2510.22728) |
|
|
|
|
|
``` |
|
|
@article{leduc2025schain, |
|
|
title={S-Chain: Structured Visual Chain-of-Thought For Medicine}, |
|
|
author={Le-Duc, Khai and Trinh, Phuong T. H. and Nguyen, Duy M. H. and Nguyen, Tien-Phat and Diep, Nghiem T. and Ngo, An and Vu, Tung and Vuong, Trinh and Nguyen, Anh-Tien and Nguyen, Mau and Hoang, Van Trung and Nguyen, Khai-Nguyen and Nguyen, Hy and Ngo, Chris and Liu, Anji and Ho, Nhat and Hauschild, Anne-Christin and Nguyen, Khanh Xuan and Nguyen-Tang, Thanh and Xie, Pengtao and Sonntag, Daniel and Zou, James and Niepert, Mathias and Nguyen, Anh Totti}, |
|
|
journal={arXiv preprint}, |
|
|
eprint={2510.22728}, |
|
|
url={https://arxiv.org/abs/2510.22728}, |
|
|
year={2025} |
|
|
} |
|
|
``` |
|
|
|
|
|
## ⚖️ Important Notice on Dataset Usage |
|
|
|
|
|
The S-Chain dataset is provided solely for research and educational purposes. |
|
|
It may contain human or machine annotation errors, as well as potential biases or inconsistencies inherent to medical data. |
|
|
Users are expected to exercise appropriate caution in interpretation and ensure ethical and non-commercial use. |
