File size: 7,400 Bytes
2e6e0cb a45ed79 2e6e0cb a45ed79 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
---
license: apache-2.0
datasets:
- leduckhai/S-Chain
language:
- en
---
<p align="center">
<img src="./SChain_icon.png" alt="S-Chain logo" width="70">
</p>
<h1 align="center">S-Chain: Structured Visual Chain-of-Thought for Medicine</h1>
[](https://arxiv.org/abs/2510.22728)
[](https://huggingface.co/leduckhai/S-Chain)
[](https://huggingface.co/datasets/leduckhai/S-Chain)
[](https://github.com/leduckhai/S-Chain/blob/main/DATASET_LICENSE.md)
[](https://s-chain.github.io/)
[](https://github.com/leduckhai/S-Chain)
---
⭐ **If you find this project helpful, please consider giving it a [star on GitHub](https://github.com/leduckhai/S-Chain)!**
---
<p align="center">
<a href="https://github.com/leduckhai" target="_blank"><strong>Khai Le-Duc</strong></a><sup>* 1,2✉</sup>,
<a href="https://scholar.google.com/citations?user=_NIyeykAAAAJ&hl=en" target="_blank"><strong>Duy M. H. Nguyen</strong></a><sup>* 3,4,24✉</sup>,
<a href="https://scholar.google.com/citations?user=5CbQH_kAAAAJ&hl=en" target="_blank"><strong>Phuong T. H. Trinh</strong></a><sup>* 5</sup>,
<strong>Tien-Phat Nguyen</strong><sup>* 6</sup>,
Nghiem T. Diep<sup>** 3</sup>,
An Ngo<sup>** 7</sup>,
Tung Vu<sup>** 8</sup>,
<a href="https://scholar.google.com/citations?user=trFdwLkAAAAJ&hl=en" target="_blank"><strong>Trinh Vuong</strong></a><sup>9</sup>,
Anh-Tien Nguyen<sup>10,11</sup>,
Mau Nguyen<sup>12</sup>,
Van Trung Hoang<sup>13</sup>,
<a href="https://scholar.google.com/citations?user=IMryD1YAAAAJ&hl=en" target="_blank"><strong>Khai-Nguyen Nguyen</strong></a><sup>14</sup>,
<a href="https://scholar.google.com/citations?user=ZAuQIqwAAAAJ&hl=en" target="_blank"><strong>Hy Nguyen</strong></a><sup>15</sup>,
Chris Ngo<sup>2</sup>,
<a href="https://scholar.google.com/citations?user=k_4zYecAAAAJ&hl=en" target="_blank"><strong>Anji Liu</strong></a><sup>16</sup>,
<a href="https://scholar.google.com/citations?user=Xs7cKMwAAAAJ&hl=en" target="_blank"><strong>Nhat Ho</strong></a><sup>17</sup>,
<a href="https://scholar.google.com/citations?user=Khifj_MAAAAJ&hl=en" target="_blank"><strong>Anne-Christin Hauschild</strong></a><sup>11</sup>,
<a href="https://scholar.google.com/citations?user=SmqouhIAAAAJ&hl=en" target="_blank"><strong>Khanh Xuan Nguyen</strong></a><sup>18</sup>,
<a href="https://scholar.google.com/citations?user=UrTlMiwAAAAJ&hl=en" target="_blank"><strong>Thanh Nguyen-Tang</strong></a><sup>19</sup>,
<a href="https://scholar.google.com/citations?user=cnncomYAAAAJ&hl=en" target="_blank"><strong>Pengtao Xie</strong></a><sup>20,21</sup>,
<a href="https://scholar.google.com/citations?user=v7i6Uz4AAAAJ&hl=en" target="_blank"><strong>Daniel Sonntag</strong></a><sup>3,22</sup>,
<a href="https://scholar.google.com/citations?user=23ZXZvEAAAAJ&hl=en" target="_blank"><strong>James Zou</strong></a><sup>23</sup>,
<a href="https://scholar.google.com/citations?user=p5vLzq0AAAAJ&hl=en" target="_blank"><strong>Mathias Niepert</strong></a><sup>4,24</sup>,
<a href="https://scholar.google.com/citations?user=EQw8d9AAAAAJ&hl=en" target="_blank"><strong>Anh Totti Nguyen</strong></a><sup>25✉</sup>
</p>
<p align="center">
<em>*Co-first authors; order randomized | **Co-second authors</em><br>
<em>✉ Corresponding Authors</em>
</p>
<details>
<summary><strong>🎓 Affiliations</strong> (click to expand)</summary>
1. University of Toronto, Canada
2. Knovel Engineering Lab, Singapore
3. German Research Centre for Artificial Intelligence
4. University of Stuttgart, Germany
5. Chonnam National University, South Korea
6. Singapore University of Technology and Design
7. Bucknell University, USA
8. Concordia University, Canada
9. Korea University
10. Justus Liebig University Giessen, Germany
11. University Medical Center Göttingen, Germany
12. Japan Advanced Institute of Science and Technology
13. Hue University, Vietnam
14. College of William & Mary, USA
15. Deakin University, Australia
16. National University of Singapore
17. University of Texas at Austin, USA
18. University of California, Berkeley, USA
19. New Jersey Institute of Technology, USA
20. University of California San Diego, USA
21. MBZUAI, UAE
22. Oldenburg University, Germany
23. Stanford University, USA
24. Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany
25. Auburn University, USA
</details>
---
<p align="center">
✨ In honor of
<a href="https://en.wikipedia.org/wiki/H%E1%BA%A3i_Th%C6%B0%E1%BB%A3ng_L%C3%A3n_%C3%94ng" target="_blank"><strong>Hải Thượng Lãn Ông (海上懶翁) – Lê Hữu Trác (黎友晫)</strong></a>,
the father of Vietnamese traditional medicine ✨
</p>
## 🔍 What is S-Chain?
S-Chain is the first large-scale dataset of **Structured Visual Chain-of-Thought (SV-CoT)**:
each reasoning step is explicitly linked to visual evidence via bounding boxes.
This enables training and evaluating *grounded* medical VLM reasoning instead of
hallucinated justifications.
- **12,000 medical images** with expert bounding boxes.
- **700k+ VQA / rationale pairs** across **16 languages**.
- Each sample: image, question, answer, stepwise SV-CoT, and per-step visual regions.
We show that supervising VLMs with SV-CoT:
- Improves interpretability
- Improves grounding fidelity (reasoning actually points to the right region)
- Improves robustness across models and languages
<p align="center">
<img src="main_pipeline.png" alt="Alt text" width="1400"/>
</p>
## 📣 News
- **[Oct 2025]** Updated experiment scripts and checkpoints for ExGra-Med and LLaVA-Med. See the [readme](architectures/Exgra-Med-CoT/README.md) for detailed instructions.
- **[Oct 2025]** Dataset and project site released.
## Citation
If you find this work useful, please cite our paper: [https://arxiv.org/abs/2510.22728](https://arxiv.org/abs/2510.22728)
```
@article{leduc2025schain,
title={S-Chain: Structured Visual Chain-of-Thought For Medicine},
author={Le-Duc, Khai and Trinh, Phuong T. H. and Nguyen, Duy M. H. and Nguyen, Tien-Phat and Diep, Nghiem T. and Ngo, An and Vu, Tung and Vuong, Trinh and Nguyen, Anh-Tien and Nguyen, Mau and Hoang, Van Trung and Nguyen, Khai-Nguyen and Nguyen, Hy and Ngo, Chris and Liu, Anji and Ho, Nhat and Hauschild, Anne-Christin and Nguyen, Khanh Xuan and Nguyen-Tang, Thanh and Xie, Pengtao and Sonntag, Daniel and Zou, James and Niepert, Mathias and Nguyen, Anh Totti},
journal={arXiv preprint},
eprint={2510.22728},
url={https://arxiv.org/abs/2510.22728},
year={2025}
}
```
## ⚖️ Important Notice on Dataset Usage
The S-Chain dataset is provided solely for research and educational purposes.
It may contain human or machine annotation errors, as well as potential biases or inconsistencies inherent to medical data.
Users are expected to exercise appropriate caution in interpretation and ensure ethical and non-commercial use. |