Add model card for RIS-FUSION and metadata

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +50 -0
README.md ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-segmentation
3
+ datasets:
4
+ - ronniejiangC/MM-RIS
5
+ arxiv: 2509.12710
6
+ tags:
7
+ - referring-image-segmentation
8
+ - image-fusion
9
+ - multimodal
10
+ ---
11
+
12
+ # RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation
13
+
14
+ This repository contains the model weights for **RIS-FUSION**, a cascaded framework presented in the paper [RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation](https://huggingface.co/papers/2509.12710).
15
+
16
+ RIS-FUSION unifies text-driven infrared and visible image fusion with referring image segmentation (RIS) through joint optimization. The framework addresses the lack of goal-aligned supervision in existing methods by observing that RIS and text-driven fusion share a common objective: highlighting the object referred to by the text. At its core is the *LangGatedFusion* module, which injects textual features into the fusion backbone to enhance semantic alignment.
17
+
18
+ ## Resources
19
+
20
+ - **Paper**: [arXiv:2509.12710](https://huggingface.co/papers/2509.12710)
21
+ - **GitHub Repository**: [SijuMa2003/RIS-FUSION](https://github.com/SijuMa2003/RIS-FUSION)
22
+ - **Dataset (MM-RIS)**: [MM-RIS on Hugging Face](https://huggingface.co/datasets/ronniejiangC/MM-RIS)
23
+
24
+ ## Sample Usage
25
+
26
+ To evaluate the model using the official implementation, you can use the following command provided in the GitHub repository:
27
+
28
+ ```bash
29
+ python test.py \
30
+ --ckpt ./ckpts/risfusion/model_best_lavt.pth \
31
+ --test_parquet ./data/mm_ris_test.parquet \
32
+ --out_dir ./your_output_dir \
33
+ --bert_tokenizer ./bert/pretrained_weights/bert-base-uncased \
34
+ --ck_bert ./bert/pretrained_weights/bert-base-uncased
35
+ ```
36
+
37
+ For detailed installation and training instructions, please refer to the [official GitHub repository](https://github.com/SijuMa2003/RIS-FUSION).
38
+
39
+ ## Citation
40
+
41
+ If you find this work useful, please consider citing the paper:
42
+
43
+ ```bibtex
44
+ @article{RIS-FUSION2025,
45
+ title = {RIS-FUSION: Rethinking Text-Driven Infrared and Visible Image Fusion from the Perspective of Referring Image Segmentation},
46
+ author = {Ma, Siju and Gong, Changsiyu and Fan, Xiaofeng and Ma, Yong and Jiang, Chengjie},
47
+ journal = {arXiv preprint arXiv:2509.12710},
48
+ year = {2025}
49
+ }
50
+ ```