Spaces:
Sleeping
Sleeping
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | |
| <title>MRT-ICCV2023</title> | |
| <style> | |
| body { | |
| text-align: center; /* 将页面文本水平居中对齐 */ | |
| } | |
| header { | |
| background-color: #333; | |
| color: #fff; | |
| padding: 20px; | |
| } | |
| main { | |
| max-width: 800px; /* 控制内容的最大宽度 */ | |
| margin: 0 auto; /* 居中对齐内容 */ | |
| text-align: left; /* 恢复内容的左对齐 */ | |
| padding: 20px; | |
| } | |
| footer { | |
| background-color: #333; | |
| color: #fff; | |
| padding: 10px; | |
| } | |
| header a { | |
| color: #FFE4B5; | |
| } | |
| main a { | |
| color: #4169E1; | |
| } | |
| footer a { | |
| color: #FFE4B5; | |
| } | |
| img { | |
| max-width: 100%; /* 图片最大宽度为父元素宽度的100% */ | |
| height: auto; /* 让高度自动调整以保持原始宽高比 */ | |
| } | |
| .center { | |
| display: flex; | |
| justify-content: center; | |
| align-items: center; | |
| } | |
| </style> | |
| </head> | |
| <body> | |
| <header> | |
| <h1>Masked Retraining Teacher-Student Framework for Domain Adaptive Object Detection</h1> | |
| <p><strong>Accepted by ICCV 2023</strong></p> | |
| <p><strong>[<a href="https://openaccess.thecvf.com/content/ICCV2023/papers/Zhao_Masked_Retraining_Teacher-Student_Framework_for_Domain_Adaptive_Object_Detection_ICCV_2023_paper.pdf">pdf</a>] | |
| [<a href="https://openaccess.thecvf.com/content/ICCV2023/supplemental/Zhao_Masked_Retraining_Teacher-Student_ICCV_2023_supplemental.pdf">supp</a>] | |
| [<a href="https://github.com/JeremyZhao1998/MRT-release">code</a>]</strong></p> | |
| <p><a href="https://jeremyzhao1998.github.io/">Zijing Zhao</a><sup>1</sup>, Sitong Wei<sup>1</sup>, Qingchao Chen<sup>1</sup>, Dehui Li<sup>2</sup>, | |
| Yifan Yang<sup>2</sup>, <a href="http://www.wict.pku.edu.cn/mipl/pengyuxin">Yuxin Peng</a><sup>1</sup>, <a href="http://www.csyangliu.com/">Yang Liu</a><sup>1,†</sup></p> | |
| <p style="line-height: 0.2;">Peking University<sup>1</sup></p> | |
| <p style="line-height: 0.2;">Tencent Intelligent Mobility<sup>2</sup><p> | |
| </header> | |
| <main> | |
| <section> | |
| <h2>Abstract</h2> | |
| <p>Domain adaptive Object Detection (DAOD) leverages a labeled domain (source) to learn an object detector | |
| generalizing to a novel domain without annotation (target). Recent advances use a teacher-student | |
| framework, i.e., a student model is supervised by the pseudo labels from a teacher model. Though great | |
| success, they suffer from the limited number of pseudo boxes with incorrect predictions caused by the | |
| domain shift, misleading the student model to get sub-optimal results. To mitigate this problem, we | |
| propose Masked Retraining Teacher-student framework (MRT) which leverages masked autoencoder and | |
| selective retraining mechanism on detection transformer. Specifically, we present a customized design | |
| of masked autoencoder branch, masking the multi-scale feature maps of target images and reconstructing | |
| features by the encoder of the student model and an auxiliary decoder. This helps the student model | |
| capture target domain characteristics and become a more data-efficient learner to gain knowledge from | |
| the limited number of pseudo boxes. Furthermore, we adopt selective retraining mechanism, periodically | |
| re-initializing certain parts of the student parameters with masked autoencoder refined weights to allow | |
| the model to jump out of the local optimum biased to the incorrect pseudo labels. Experimental results | |
| on three DAOD benchmarks demonstrate the effectiveness of our method. Code can be found at | |
| <a href="https://github.com/JeremyZhao1998/MRT-release">MRT Codebase</a>.</p> | |
| </section> | |
| <section> | |
| <h2>Problem Definition</h2> | |
| <img src="https://github.com/JeremyZhao1998/MRT-release/blob/main/docs/task1.png?raw=true" alt="Task discription 1"> | |
| <img src="https://github.com/JeremyZhao1998/MRT-release/blob/main/docs/task2.png?raw=true" alt="Task discription 2"> | |
| <p>In real-world applications, there exists a distribution gap between training data (source domain) and the | |
| deployed environment (target domain). For example, the model trained on sunny weather may face a | |
| significant performance drop when applied to foggy weather scenes. Unsupervised Domain Adaptive Object | |
| Detection aims to generalize the model to target domain without additional annotations.</p> | |
| </section> | |
| <section> | |
| <h2>Method Overview</h2> | |
| <img src="https://github.com/JeremyZhao1998/MRT-release/blob/main/docs/method.png?raw=true" alt="MRT method overview"> | |
| <p>Overview Masked Retraining Teacher-student framework(MRT). The adaptive teacher-student baseline consists | |
| of a teacher model which takes weakly-augmented target images and produces pseudo labels, and a student | |
| model which takes strongly augmented source and target images, supervised by ground truth labels and | |
| pseudo labels respectively. Adversarial alignment are applied on backbone, encoder and decoder. Our | |
| proposed MAE branch masks feature maps of target images, and and reconstructs the feature by student | |
| encoder and an auxiliary decoder. Our proposed selective retraining mechanism periodically re-initialize | |
| certain parts of the student parameters as highlighted. The teacher model is updated only by EMA from | |
| the student model. Empirically, we use the teacher model at inference time.</p> | |
| </section> | |
| <section> | |
| <h2>Experimental Results</h2> | |
| <div class="center"> | |
| <img src="https://github.com/JeremyZhao1998/MRT-release/blob/main/docs/results1.png?raw=true" alt="Main results" width="60%" height="auto"> | |
| </div> | |
| <p>Our method achieves state-of-the-art performance on three benchmarks: | |
| cityscapes to foggy cityscapes(0.02) (city2foggy), | |
| sim10k to cityscapes(car) (sim2city), and | |
| cityscapes to bdd100k(daytime) (city2bdd). | |
| </p> | |
| <img src="https://github.com/JeremyZhao1998/MRT-release/blob/main/docs/results2.png?raw=true" alt="Visualization"> | |
| <p>Visulization of detection results demostrate the effectiveness of each module of our method.</p> | |
| </section> | |
| <section> | |
| <h2>Introduction Video</h2> | |
| <div class="center"> | |
| <iframe width="800" height="450" src="https://www.youtube.com/embed/GGhBn6akViU?si=9npiYvYJNRcQXT2C" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe> | |
| </div> | |
| </section> | |
| <section> | |
| <h2>Citation</h2> | |
| <p>If you use MRT in your research or wish to refer to the results published in the paper, please use the | |
| following BibTeX entry.</p> | |
| <pre> | |
| @inproceedings{zhao2023masked, | |
| title={Masked Retraining Teacher-Student Framework for Domain Adaptive Object Detection}, | |
| author={Zhao, Zijing and Wei, Sitong and Chen, Qingchao and Li, Dehui and Yang, Yifan and Peng, Yuxin and Liu, Yang}, | |
| booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, | |
| pages={19039--19049}, | |
| year={2023} | |
| } | |
| </pre> | |
| </section> | |
| </main> | |
| <footer> | |
| <p>© 2023 Page created by <a href="https://jeremyzhao1998.github.io/">Zijing Zhao</a></p> | |
| </footer> | |
| </body> | |
| </html> | |