|
|
--- |
|
|
language: |
|
|
- en |
|
|
tags: |
|
|
- comfyui |
|
|
- inpainting |
|
|
- stable-diffusion |
|
|
- image-generation |
|
|
- computer-vision |
|
|
- diffusion |
|
|
- image-editing |
|
|
license: gpl-3.0 |
|
|
library_name: comfyui-extension |
|
|
pipeline_tag: image-to-image |
|
|
datasets: |
|
|
- stable-diffusion |
|
|
- stable-diffusion-xl |
|
|
- stable-diffusion-3 |
|
|
- qwen-image |
|
|
- hidream |
|
|
- flux |
|
|
metrics: |
|
|
- image-quality |
|
|
- inpainting-accuracy |
|
|
- consistency |
|
|
--- |
|
|
|
|
|
# LanPaint: Universal Inpainting Sampler |
|
|
|
|
|
## Model Description |
|
|
|
|
|
LanPaint is a universal inpainting sampler that works with any Stable Diffusion model without requiring specialized training. It introduces a "think mode" that allows models to process multiple iterations before denoising, resulting in superior inpainting quality. |
|
|
|
|
|
- **Developed by:** LanPaint Team |
|
|
- **Model type:** ComfyUI Extension (Sampler) |
|
|
- **License:** GNU General Public License v3.0 |
|
|
- **Repository:** [GitHub](https://github.com/scraed/LanPaint) |
|
|
- **Paper:** [LanPaint: Training-Free Diffusion Inpainting with Exact and Fast Conditional Inference](https://arxiv.org/abs/2502.03491) |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
LanPaint is designed for: |
|
|
- **Image Inpainting:** Fill in masked areas of images with contextually appropriate content |
|
|
- **Image Outpainting:** Extend images beyond their original boundaries |
|
|
- **Character Consistency:** Generate consistent character views and variations |
|
|
- **Universal Compatibility:** Work with any Stable Diffusion model (SD 1.5, XL, 3.5, Flux, HiDream, Qwen-Image) |
|
|
|
|
|
## How to Use |
|
|
|
|
|
### Installation |
|
|
1. Install ComfyUI (version > 0.3.11) |
|
|
2. Install ComfyUI-Manager |
|
|
3. Install LanPaint via ComfyUI-Manager or clone to `custom_nodes` folder |
|
|
4. Restart ComfyUI |
|
|
|
|
|
### Basic Usage |
|
|
- Use LanPaint nodes in the "sampling" category |
|
|
- Same workflow as standard ComfyUI KSampler |
|
|
- Supports any mask shape, size, or position |
|
|
- Configurable "thinking" steps for quality vs. speed trade-off |
|
|
|
|
|
### Example Workflows |
|
|
- Qwen Image inpainting/outpainting |
|
|
- HiDream character consistency |
|
|
- SD 3.5 high-quality inpainting |
|
|
- Flux model inpainting (with low guidance) |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- **Distillation Models:** Reduced performance on models like Flux.dev (use low guidance 1.0-2.0) |
|
|
- **Computation Time:** More thinking steps require more processing time |
|
|
- **Memory Usage:** May require significant GPU memory for high-resolution images |
|
|
|
|
|
## Training and Technical Details |
|
|
|
|
|
- **Training-free:** Works out-of-the-box with existing models |
|
|
- **No fine-tuning required:** Universal compatibility across model architectures |
|
|
- **Exact conditional inference:** Mathematically sound approach to inpainting |
|
|
- **Fast implementation:** Optimized for real-world usage |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
LanPaint has been benchmarked against traditional inpainting methods: |
|
|
- VAE Encode for Inpainting |
|
|
- Set Latent Noise Mask |
|
|
- Standard KSampler approaches |
|
|
|
|
|
Results show significant improvements in: |
|
|
- Content consistency |
|
|
- Mask boundary handling |
|
|
- Overall image quality |
|
|
- Character preservation |
|
|
|
|
|
## Citation |
|
|
|
|
|
```bibtex |
|
|
@article{lanpaint2024, |
|
|
title={LanPaint: Training-Free Diffusion Inpainting with Exact and Fast Conditional Inference}, |
|
|
author={LanPaint Team}, |
|
|
journal={arXiv preprint arXiv:2502.03491}, |
|
|
year={2024} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Additional Resources |
|
|
|
|
|
- [ComfyUI Documentation](https://docs.comfy.org/) |
|
|
- [LanPaintBench Repository](https://github.com/scraed/LanPaintBench) |
|
|
- [Blog Post](https://scraed.github.io/scraedBlog/) |
|
|
- [Community Examples](https://github.com/scraed/LanPaint/tree/master/examples) |
|
|
|
|
|
## License |
|
|
|
|
|
This project is licensed under the GNU General Public License v3.0 - see the [LICENSE](LICENSE) file for details. |
|
|
|