|
|
--- |
|
|
license: apache-2.0 |
|
|
--- |
|
|
|
|
|
<div align="center"> |
|
|
|
|
|
<h1>Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization</h1> |
|
|
|
|
|
<div> |
|
|
<a href='https://sites.google.com/view/dong-hao/' target='_blank'>Hao Dong</a><sup>1</sup>  |
|
|
<a href='https://chatzi.ibk.ethz.ch/about-us/people/prof-dr-eleni-chatzi.html' target='_blank'>Eleni Chatzi</a><sup>1</sup>  |
|
|
<a href='https://people.epfl.ch/olga.fink?lang=en' target='_blank'>Olga Fink</a><sup>2</sup> |
|
|
</div> |
|
|
<div> |
|
|
<sup>1</sup>ETH Zurich, <sup>2</sup>EPFL |
|
|
</div> |
|
|
|
|
|
|
|
|
<div> |
|
|
<h4 align="center"> |
|
|
• <a href="https://openreview.net/forum?id=hj323oR3rw" target='_blank'>ICLR 2025</a> • |
|
|
</h4> |
|
|
</div> |
|
|
|
|
|
|
|
|
|
|
|
<div style="text-align:center"> |
|
|
<img src="moti.jpg" width="100%" height="100%"> |
|
|
</div> |
|
|
|
|
|
--- |
|
|
|
|
|
</div> |
|
|
|
|
|
|
|
|
Figure 1: (a) Tent minimizes the entropy of all samples, making it difficult to separate the prediction score distributions of known and unknown samples. (b) Our AEO amplifies entropy differences between known and unknown samples through adaptive optimization. (c) As a result, Tent negatively impacts MM-OSTTA performance while AEO significantly improves unknown class detection. |
|
|
|
|
|
## Code |
|
|
https://github.com/donghao51/AEO |
|
|
|
|
|
## Contact |
|
|
If you have any questions, please send an email to donghaospurs@gmail.com |
|
|
|
|
|
## Citation |
|
|
|
|
|
If you find our work useful in your research please consider citing our [paper](https://openreview.net/forum?id=hj323oR3rw): |
|
|
|
|
|
``` |
|
|
@inproceedings{dong2025aeo, |
|
|
title={Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization}, |
|
|
author={Dong, Hao and Chatzi, Eleni and Fink, Olga}, |
|
|
booktitle={The Thirteenth International Conference on Learning Representations}, |
|
|
year={2025} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Related Projects |
|
|
|
|
|
[SimMMDG](https://github.com/donghao51/SimMMDG): A Simple and Effective Framework for Multi-modal Domain Generalization |
|
|
|
|
|
[MultiOOD](https://github.com/donghao51/MultiOOD): Scaling Out-of-Distribution Detection for Multiple Modalities |
|
|
|
|
|
[MOOSA](https://github.com/donghao51/MOOSA): Towards Multimodal Open-Set Domain Generalization and Adaptation through Self-supervision |
|
|
|
|
|
|
