| --- |
| license: apache-2.0 |
| extra_gated_fields: |
| Name: text |
| Institute: text |
| Institutional Email: text |
| I agree to use this model for non-commercial use ONLY: checkbox |
| --- |
| <p align="center"> |
| <h1 align="center">Towards Enhanced Image Inpainting:<br> |
| Mitigating Unwanted Object Insertion and Preserving Color Consistency</h1> |
| <center>Yikai Wang*, Chenjie Cao*, Junqiu Yu*, Ke Fan, Xiangyang Xue, Yanwei Fu†.<br> |
| Fudan University<br> |
| <b>CVPR 2025 <font color="#ed7748">(Highlight)</font></b> |
| </center> |
| <p align="center"> |
| <a href="https://arxiv.org/abs/2312.04831"><img alt='arXiv' src="https://img.shields.io/badge/arXiv-2312.04831-b31b1b.svg"></a> |
| <a href="https://yikai-wang.github.io/asuka/"><img alt='page' src="https://img.shields.io/badge/Project-Website-orange"></a> |
| |
| </p> |
| <br> |
| </p> |
| |
| ## Overview |
| This repo contains the proposed ASUKA model in our paper "[Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency](https://arxiv.org/abs/2312.04831)". |
| |
| > ASUKA solves two issues existed in current diffusion and rectified flow inpainting models: |
| <b>Unwanted object insertion</b>, where randomly elements that are not aligned with the unmasked region are generated; |
| <b>Color-inconsistenc</b>y, the color shift of the generated masked region, causing smear-like traces. |
| ASUKA proposes a post-training procedure for these models, significantly mitigates object hallucination and improves color consistency of inpainted results. |
| |
| We released ASUKA for [FLUX.1-Fill-dev](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev), denoted as ASUKA(FLUX.1-Fill). |
| The code and dataset can be found at [here](https://github.com/Yikai-Wang/asuka-misato). |
| We are actively working to improve both our model and evaluation dataset. |
| If you encounter failure cases with ASUKA (FLUX.1-Fill) or have challenging examples in image inpainting, we would love to hear from you. Please email them to yi-kai.wang@outlook.com. |
| We truly appreciate your contributions! |
| |
| ## Modifications to FLUX |
| - The text conditional input of CLIP and T5 is replaced by the MAE condition to mitigate object hallucination. |
| - The decoder is replaced by our conditional decoder to enhance color consistency. |
| |
| ## BibTeX |
| If you find our repo helpful, please consider cite our paper :) |
| ```bibtex |
| @inproceedings{wang2025towards, |
| title={Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency.}, |
| author={Wang, Yikai and Cao, Chenjie and Yu, Junqiu and Fan, Ke and Xue, Xiangyang and Fu, Yanwei}, |
| booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, |
| year={2025} |
| } |
| ``` |
