Add model card for RIS-FUSION and metadata
#1
by
nielsr
HF Staff
- opened
README.md
ADDED
|
@@ -0,0 +1,50 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
pipeline_tag: image-segmentation
|
| 3 |
+
datasets:
|
| 4 |
+
- ronniejiangC/MM-RIS
|
| 5 |
+
arxiv: 2509.12710
|
| 6 |
+
tags:
|
| 7 |
+
- referring-image-segmentation
|
| 8 |
+
- image-fusion
|
| 9 |
+
- multimodal
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
# RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation
|
| 13 |
+
|
| 14 |
+
This repository contains the model weights for **RIS-FUSION**, a cascaded framework presented in the paper [RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation](https://huggingface.co/papers/2509.12710).
|
| 15 |
+
|
| 16 |
+
RIS-FUSION unifies text-driven infrared and visible image fusion with referring image segmentation (RIS) through joint optimization. The framework addresses the lack of goal-aligned supervision in existing methods by observing that RIS and text-driven fusion share a common objective: highlighting the object referred to by the text. At its core is the *LangGatedFusion* module, which injects textual features into the fusion backbone to enhance semantic alignment.
|
| 17 |
+
|
| 18 |
+
## Resources
|
| 19 |
+
|
| 20 |
+
- **Paper**: [arXiv:2509.12710](https://huggingface.co/papers/2509.12710)
|
| 21 |
+
- **GitHub Repository**: [SijuMa2003/RIS-FUSION](https://github.com/SijuMa2003/RIS-FUSION)
|
| 22 |
+
- **Dataset (MM-RIS)**: [MM-RIS on Hugging Face](https://huggingface.co/datasets/ronniejiangC/MM-RIS)
|
| 23 |
+
|
| 24 |
+
## Sample Usage
|
| 25 |
+
|
| 26 |
+
To evaluate the model using the official implementation, you can use the following command provided in the GitHub repository:
|
| 27 |
+
|
| 28 |
+
```bash
|
| 29 |
+
python test.py \
|
| 30 |
+
--ckpt ./ckpts/risfusion/model_best_lavt.pth \
|
| 31 |
+
--test_parquet ./data/mm_ris_test.parquet \
|
| 32 |
+
--out_dir ./your_output_dir \
|
| 33 |
+
--bert_tokenizer ./bert/pretrained_weights/bert-base-uncased \
|
| 34 |
+
--ck_bert ./bert/pretrained_weights/bert-base-uncased
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
For detailed installation and training instructions, please refer to the [official GitHub repository](https://github.com/SijuMa2003/RIS-FUSION).
|
| 38 |
+
|
| 39 |
+
## Citation
|
| 40 |
+
|
| 41 |
+
If you find this work useful, please consider citing the paper:
|
| 42 |
+
|
| 43 |
+
```bibtex
|
| 44 |
+
@article{RIS-FUSION2025,
|
| 45 |
+
title = {RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation},
|
| 46 |
+
author = {Ma, Siju and Gong, Changsiyu and Fan, Xiaofeng and Ma, Yong and Jiang, Chengjie},
|
| 47 |
+
journal = {arXiv preprint arXiv:2509.12710},
|
| 48 |
+
year = {2025}
|
| 49 |
+
}
|
| 50 |
+
```
|