serag-ai/Best-Fold-SPINE-Axial

SPINE: Segmentation-guided Processing and Integration of Multimodal Spinal MRI for Natural-Language Enhanced Report Generation

Abstract

SPINE is a segmentation-guided multimodal framework for spinal MRI report generation using 3D vision-language models. The framework integrates T1-weighted MRI, T2-weighted MRI, and anatomical segmentation masks to enhance spatial and contextual understanding of spinal structures. By incorporating anatomical priors through segmentation, SPINE improves the generation of clinically meaningful radiology reports.

Experiments were conducted on two public spinal MRI datasets comprising 515 axial and 190 sagittal cases. Three input configurations were evaluated, with the combination of T1, T2, and segmentation achieving the best performance on the axial dataset. For sagittal MRI, structured gradings were transformed into narrative reports using large language models, demonstrating that structured supervision improves report consistency and semantic accuracy.

---

Acknowledgements

This work builds upon:

• M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models
• LLaVA: Large Language and Vision Assistant

We thank the original authors for making their code and pretrained models publicly available.

---

Citation

If you use this work in your research, please cite:

@article{Helmy31122026,
  author = {Hoda Helmy and Abdullah Hosseini and Ahmed Ibrahim and Asfand Baig-Mirza and Ahmed-Ramadan Sadek and Ahmed Serag},
  title = {SPINE: Segmentation-guided Processing and Integration of Multimodal Spinal MRI for Natural-Language Enhanced Report Generation},
  journal = {Applied Artificial Intelligence},
  volume = {40},
  number = {1},
  pages = {2626117},
  year = {2026},
  publisher = {Taylor & Francis},
  doi = {10.1080/08839514.2026.2626117},
  url = {https://doi.org/10.1080/08839514.2026.2626117}
}