Add metadata and improve model card structure
#1
by
nielsr HF Staff - opened
README.md
CHANGED
|
@@ -1,19 +1,40 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
, [Xinzhe Luo](https://scholar.google.com/citations?user=l-oyIaAAAAAJ&hl=en&oi=ao), [Declan P. O'Regan](https://scholar.google.com/citations?user=85u-LbAAAAAJ&hl=en&oi=ao), and [Chen Qin](https://scholar.google.com/citations?view_op=list_works&hl=en&hl=en&user=mTWrOqHOqjoC&pagesize=80&sortby=pubdate)**
|
| 11 |
+
|
| 12 |
+
[](#License)
|
| 13 |
+
[](https://arxiv.org/abs/2601.22853)
|
| 14 |
+
|
| 15 |
+
</div>
|
| 16 |
+
|
| 17 |
+

|
| 18 |
+
<p align="center">(a-b) Evidence of the discarding-imputation dilemma: (a-1) vs. (a-2) recovery-free methods (e.g., ModDrop) learn less discriminative features because they ignore highly task-relevant missing modalities {M,T}; (b) recovery-based methods (e.g., MoPoE) generate unreliable reconstructions, e.g., low-fidelity (orange) or misaligned (yellow). (c) Our DyMo, which addresses the dilemma by dynamically fusing task-relevant recovered modalities, improving accuracy by 1.61% on PolyMNIST, 1.68% on MST, and 3.88% on CelebA (Tab 1).</p>
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
This repository provides **pretrained model checkpoints** for the paper [Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification](https://huggingface.co/papers/2601.22853).
|
| 22 |
+
|
| 23 |
+
For the **training code, evaluation scripts, and usage instructions**, please refer to the official GitHub repository:
|
| 24 |
+
👉 [https://github.com/siyi-wind/DyMo](https://github.com/siyi-wind/DyMo).
|
| 25 |
+
|
| 26 |
+
## Abstract
|
| 27 |
+
Multimodal deep learning (MDL) has achieved remarkable success across various domains, yet its practical deployment is often hindered by incomplete multimodal data. Existing incomplete MDL methods either discard missing modalities, risking the loss of valuable task-relevant information, or recover them, potentially introducing irrelevant noise, leading to the discarding-imputation dilemma. To address this dilemma, we propose DyMo, a new inference-time dynamic modality selection framework that adaptively identifies and integrates reliable recovered modalities, fully exploring task-relevant information beyond the conventional discard-or-impute paradigm.
|
| 28 |
+
|
| 29 |
+
## Citation
|
| 30 |
+
|
| 31 |
+
If you find this work useful, please cite:
|
| 32 |
+
|
| 33 |
+
```bibtex
|
| 34 |
+
@inproceedings{du2026dymo,
|
| 35 |
+
title={Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification},
|
| 36 |
+
author={Du, Siyi and Luo, Xinzhe and O'Regan, Declan P. and Qin, Chen},
|
| 37 |
+
booktitle={International Conference on Learning Representations (ICLR) 2026},
|
| 38 |
+
year={2026}
|
| 39 |
+
}
|
| 40 |
+
```
|