Instructions to use ilkerzgi/Overlay-Kontext-Dev-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ilkerzgi/Overlay-Kontext-Dev-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-Kontext-dev", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ilkerzgi/Overlay-Kontext-Dev-LoRA") prompt = "Place it" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Document academic use and citation
#8
by gokaygokay - opened
README.md
CHANGED
|
@@ -1,97 +1,135 @@
|
|
| 1 |
-
---
|
| 2 |
-
tags:
|
| 3 |
-
- flux
|
| 4 |
-
- overlay
|
| 5 |
-
- image-overlay
|
| 6 |
-
- lora
|
| 7 |
-
- fal
|
| 8 |
-
- diffusers
|
| 9 |
-
- ilkerzgi
|
| 10 |
-
- gokaygokay
|
| 11 |
-
language: en
|
| 12 |
-
base_model: black-forest-labs/FLUX.1-Kontext-dev
|
| 13 |
-
instance_prompt: place it
|
| 14 |
-
license_name: flux1-dev-non-commercial-license
|
| 15 |
-
license_link: >-
|
| 16 |
-
https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md
|
| 17 |
-
license: other
|
| 18 |
-
widget:
|
| 19 |
-
- text: Place it
|
| 20 |
-
output:
|
| 21 |
-
url: out1.png
|
| 22 |
-
- src: in1.png
|
| 23 |
-
prompt: Place it
|
| 24 |
-
output:
|
| 25 |
-
url: out1.png
|
| 26 |
-
- text: Place it
|
| 27 |
-
output:
|
| 28 |
-
url: out2.png
|
| 29 |
-
- src: in2.png
|
| 30 |
-
prompt: Place it
|
| 31 |
-
output:
|
| 32 |
-
url: out2.png
|
| 33 |
-
- text: Place it
|
| 34 |
-
output:
|
| 35 |
-
url: out3.png
|
| 36 |
-
- src: ins3.jpg
|
| 37 |
-
prompt: Place it
|
| 38 |
-
output:
|
| 39 |
-
url: out3.png
|
| 40 |
-
pipeline_tag: image-to-image
|
| 41 |
-
---
|
| 42 |
-
|
| 43 |
-
# Kontext Dev LoRA - Image Overlay
|
| 44 |
-
|
| 45 |
-
<Gallery />
|
| 46 |
-
|
| 47 |
-
This is a LoRA (Low-Rank Adaptation) model that has been trained for image overlay tasks, allowing you to seamlessly blend and place elements into existing images.
|
| 48 |
-
|
| 49 |
-
## Table of Contents
|
| 50 |
-
- [Model Details](#model-details)
|
| 51 |
-
- [Uses](#uses)
|
| 52 |
- [Training](#training)
|
| 53 |
- [Usage Tips](#usage-tips)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
-
##
|
| 56 |
-
|
| 57 |
-
**Model Description:** This model is a LoRA fine-tune of `black-forest-labs/FLUX.1-Kontext-dev` specifically designed for image overlay workflows, enabling natural integration of new elements into existing scenes.
|
| 58 |
-
|
| 59 |
-
* **Trained by:** [ilkerzgi](https://huggingface.co/ilkerzgi) and [gokaygokay](https://huggingface.co/gokaygokay)
|
| 60 |
-
* **Model Type:** LoRA for Image Overlay
|
| 61 |
-
* **Language(s):** English
|
| 62 |
-
* **License:** [flux1-dev-non-commercial-license](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)
|
| 63 |
-
* **Parent Model:** [black-forest-labs/FLUX.1-Kontext-dev](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
|
| 64 |
-
* **Resources for more information:**
|
| 65 |
-
* [Download the weights](https://huggingface.co/ilkerzgi/kontext-dev-overlay-lora/blob/main/WVVtJFD90b8SsU6EzeGkO_adapter_model_comfy_converted.safetensors)
|
| 66 |
-
* [Try the model directly on fal.ai](https://infinite-kanvas.vercel.app)
|
| 67 |
-
|
| 68 |
-
## Uses
|
| 69 |
|
| 70 |
-
|
| 71 |
-
This model is designed for image overlay tasks, allowing you to naturally integrate new objects, elements, or modifications into existing images. The model excels at understanding context and placement, creating seamless compositions.
|
| 72 |
|
| 73 |
-
|
|
|
|
| 74 |
|
| 75 |
-
|
| 76 |
-
- **LoRA Scale:** 1.0 or 0.9 for optimal results
|
| 77 |
-
- **Image Format:** Square format images work best
|
| 78 |
-
- **Overlay Area:** Clear definition of where elements should be placed
|
| 79 |
-
- **Context Understanding:** Works well with natural scene composition
|
| 80 |
|
| 81 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 82 |
|
| 83 |
-
|
| 84 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
-
|
| 89 |
|
| 90 |
-
-
|
| 91 |
-
- Set **LoRA scale to 1.0 or 0.9** for optimal output quality
|
| 92 |
-
- Use the simple prompt **"place it"** as the trigger phrase
|
| 93 |
-
- Provide clear context about what you want to overlay and where
|
| 94 |
-
- The model understands spatial relationships and lighting conditions for natural integration
|
| 95 |
-
- Works well with both object placement and scene modifications
|
| 96 |
-
|
| 97 |
-
---
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- flux
|
| 4 |
+
- overlay
|
| 5 |
+
- image-overlay
|
| 6 |
+
- lora
|
| 7 |
+
- fal
|
| 8 |
+
- diffusers
|
| 9 |
+
- ilkerzgi
|
| 10 |
+
- gokaygokay
|
| 11 |
+
language: en
|
| 12 |
+
base_model: black-forest-labs/FLUX.1-Kontext-dev
|
| 13 |
+
instance_prompt: place it
|
| 14 |
+
license_name: flux1-dev-non-commercial-license
|
| 15 |
+
license_link: >-
|
| 16 |
+
https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md
|
| 17 |
+
license: other
|
| 18 |
+
widget:
|
| 19 |
+
- text: Place it
|
| 20 |
+
output:
|
| 21 |
+
url: out1.png
|
| 22 |
+
- src: in1.png
|
| 23 |
+
prompt: Place it
|
| 24 |
+
output:
|
| 25 |
+
url: out1.png
|
| 26 |
+
- text: Place it
|
| 27 |
+
output:
|
| 28 |
+
url: out2.png
|
| 29 |
+
- src: in2.png
|
| 30 |
+
prompt: Place it
|
| 31 |
+
output:
|
| 32 |
+
url: out2.png
|
| 33 |
+
- text: Place it
|
| 34 |
+
output:
|
| 35 |
+
url: out3.png
|
| 36 |
+
- src: ins3.jpg
|
| 37 |
+
prompt: Place it
|
| 38 |
+
output:
|
| 39 |
+
url: out3.png
|
| 40 |
+
pipeline_tag: image-to-image
|
| 41 |
+
---
|
| 42 |
+
|
| 43 |
+
# Kontext Dev LoRA - Image Overlay
|
| 44 |
+
|
| 45 |
+
<Gallery />
|
| 46 |
+
|
| 47 |
+
This is a LoRA (Low-Rank Adaptation) model that has been trained for image overlay tasks, allowing you to seamlessly blend and place elements into existing images.
|
| 48 |
+
|
| 49 |
+
## Table of Contents
|
| 50 |
+
- [Model Details](#model-details)
|
| 51 |
+
- [Uses](#uses)
|
| 52 |
- [Training](#training)
|
| 53 |
- [Usage Tips](#usage-tips)
|
| 54 |
+
- [Academic Use and Citation](#academic-use-and-citation)
|
| 55 |
+
|
| 56 |
+
## Model Details
|
| 57 |
+
|
| 58 |
+
**Model Description:** This model is a LoRA fine-tune of `black-forest-labs/FLUX.1-Kontext-dev` specifically designed for image overlay workflows, enabling natural integration of new elements into existing scenes.
|
| 59 |
+
|
| 60 |
+
* **Trained by:** [ilkerzgi](https://huggingface.co/ilkerzgi) and [gokaygokay](https://huggingface.co/gokaygokay)
|
| 61 |
+
* **Model Type:** LoRA for Image Overlay
|
| 62 |
+
* **Language(s):** English
|
| 63 |
+
* **License:** [flux1-dev-non-commercial-license](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev/blob/main/LICENSE.md)
|
| 64 |
+
* **Parent Model:** [black-forest-labs/FLUX.1-Kontext-dev](https://huggingface.co/black-forest-labs/FLUX.1-Kontext-dev)
|
| 65 |
+
* **Resources for more information:**
|
| 66 |
+
* [Download the weights](https://huggingface.co/ilkerzgi/kontext-dev-overlay-lora/blob/main/WVVtJFD90b8SsU6EzeGkO_adapter_model_comfy_converted.safetensors)
|
| 67 |
+
* [Try the model directly on fal.ai](https://infinite-kanvas.vercel.app)
|
| 68 |
+
|
| 69 |
+
## Uses
|
| 70 |
+
|
| 71 |
+
#### Direct Use
|
| 72 |
+
This model is designed for image overlay tasks, allowing you to naturally integrate new objects, elements, or modifications into existing images. The model excels at understanding context and placement, creating seamless compositions.
|
| 73 |
+
|
| 74 |
+
The model works best when using the trigger phrase: **"place it"**.
|
| 75 |
+
|
| 76 |
+
#### Recommended Settings
|
| 77 |
+
- **LoRA Scale:** 1.0 or 0.9 for optimal results
|
| 78 |
+
- **Image Format:** Square format images work best
|
| 79 |
+
- **Overlay Area:** Clear definition of where elements should be placed
|
| 80 |
+
- **Context Understanding:** Works well with natural scene composition
|
| 81 |
+
|
| 82 |
+
## Training
|
| 83 |
+
|
| 84 |
+
#### Training Procedure
|
| 85 |
+
Training was performed using the [FLUX Kontext Trainer on fal.ai](https://fal.ai/models/fal-ai/flux-kontext-trainer/playground).
|
| 86 |
+
|
| 87 |
+
The base model for training was `black-forest-labs/FLUX.1-Kontext-dev`, with specialized focus on image overlay scenarios and contextual element placement.
|
| 88 |
+
|
| 89 |
+
## Usage Tips
|
| 90 |
+
|
| 91 |
+
- Use **square format images** for best results
|
| 92 |
+
- Set **LoRA scale to 1.0 or 0.9** for optimal output quality
|
| 93 |
+
- Use the simple prompt **"place it"** as the trigger phrase
|
| 94 |
+
- Provide clear context about what you want to overlay and where
|
| 95 |
+
- The model understands spatial relationships and lighting conditions for natural integration
|
| 96 |
+
- Works well with both object placement and scene modifications
|
| 97 |
|
| 98 |
+
## Academic Use and Citation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 99 |
|
| 100 |
+
This model is cited and evaluated as a baseline in:
|
|
|
|
| 101 |
|
| 102 |
+
- [*LayerComposer: Multi-Human Personalized Generation via Layered Canvas*](https://arxiv.org/abs/2510.20820) (2025 preprint)
|
| 103 |
+
- [*Canvas-to-Image: Compositional Image Generation with Multimodal Controls*](https://doi.org/10.1145/3799902.3811219) (ACM SIGGRAPH 2026)
|
| 104 |
|
| 105 |
+
If this model is useful in your research, please cite the model and the relevant paper:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
|
| 107 |
+
```bibtex
|
| 108 |
+
@software{ilkerzgi_aydogan2025overlay_kontext,
|
| 109 |
+
author = {{ilkerzgi} and Aydoğan, Gökay},
|
| 110 |
+
title = {Overlay Kontext Dev LoRA},
|
| 111 |
+
year = {2025},
|
| 112 |
+
url = {https://huggingface.co/ilkerzgi/Overlay-Kontext-Dev-LoRA},
|
| 113 |
+
note = {Hugging Face model}
|
| 114 |
+
}
|
| 115 |
|
| 116 |
+
@article{qian2025layercomposer,
|
| 117 |
+
author = {Qian, Guocheng Gordon and Zhang, Ruihang and Chen, Tsai-Shien and Dalva, Yusuf and Goyal, Anujraaj Argo and Menapace, Willi and Skorokhodov, Ivan and Dong, Meng and Sahni, Arpit and Ostashev, Daniil and Hu, Ju and Tulyakov, Sergey and Wang, Kuan-Chieh Jackson},
|
| 118 |
+
title = {LayerComposer: Multi-Human Personalized Generation via Layered Canvas},
|
| 119 |
+
journal = {arXiv preprint arXiv:2510.20820},
|
| 120 |
+
year = {2025},
|
| 121 |
+
doi = {10.48550/arXiv.2510.20820}
|
| 122 |
+
}
|
| 123 |
|
| 124 |
+
@inproceedings{dalva2026canvas,
|
| 125 |
+
author = {Dalva, Yusuf and Qian, Guocheng Gordon and Goldenberg, Maya and Chen, Tsai-Shien and Aberman, Kfir and Tulyakov, Sergey and Yanardag, Pinar and Wang, Kuan-Chieh Jackson},
|
| 126 |
+
title = {Canvas-to-Image: Compositional Image Generation with Multimodal Controls},
|
| 127 |
+
booktitle = {ACM SIGGRAPH 2026 Conference Papers},
|
| 128 |
+
year = {2026},
|
| 129 |
+
doi = {10.1145/3799902.3811219}
|
| 130 |
+
}
|
| 131 |
+
```
|
| 132 |
|
| 133 |
+
Co-trained by [ilkerzgi](https://huggingface.co/ilkerzgi) and [Gökay Aydoğan](https://orcid.org/0000-0002-2343-9433).
|
| 134 |
|
| 135 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|