++: Instruction-Based Image Creation and Editing via Context-Aware Content Filling
+ +
++: Instruction-Based Image Creation and Editing
+ 
+ Chaojie Mao
+ ·
+ Jingfeng Zhang
+ ·
+ Yulin Pan
+ ·
+ Zeyinzi Jiang
+ ·
+ Zhen Han
+
+ ·
+ Yu Liu
+ ·
+ Jingren Zhou
+
+ Tongyi Lab, Alibaba Group
+
+ 
+ |
+
| Tuning Method | +Input | +Output | +Instruction | +Models | +
| LoRA + ACE Data |
+  |
+  |
+ "Maintain the facial features. A girl is wearing a neat police uniform and sporting a badge. She is smiling with a friendly and confident demeanor. The background is blurred, featuring a cartoon logo." | +  |
+
| Tuning Method | +Input | +Output | +Instruction | +Models | +
| LoRA + ACE Data |
+  |
+  |
+ "Display the logo in a minimalist style printed in white on a matte black ceramic coffee mug, alongside a steaming cup of coffee on a cozy cafe table." | + |
+
| Tuning Method | +Input | +Output | +Instruction | +Models | +
| LoRA + ACE Data |
+   |
+  |
+ "By referencing the mask, restore a partial image from the doodle {image} that aligns with the textual explanation: "1 white old owl"." | + |
+
| ACE++ Model | +Input Reference Image | +Input Edit Image | +Input Edit Mask | +Output | +Instruction | +Function | +
|---|---|---|---|---|---|---|
| Portrait LoRA(recommended) / FFT model | + |
+ + | + |  |
+ "Maintain the facial features, A girl is wearing a neat police uniform and sporting a badge. She is smiling with a friendly and confident demeanor. The background is blurred, featuring a cartoon logo." | +"Character ID Consistency Generation" | +
| Subject LoRA(recommended) / FFT model | + |
+ + | + |  |
+ "Display the logo in a minimalist style printed in white on a matte black ceramic coffee mug, alongside a steaming cup of coffee on a cozy cafe table." | +"Subject Consistency Generation" | +
| Subject LoRA(recommended) / FFT model | + |
+  |
+  |
+  |
+ "The item is put on the table." | +"Subject Consistency Editing" | +
| Subject LoRA(recommended) / FFT model | + |
+  |
+  |
+  |
+ "The logo is printed on the headphones." | +"Subject Consistency Editing" | +
| Subject LoRA(recommended) / FFT model | + |
+  |
+  |
+  |
+ "The woman dresses this skirt." | +"Try On" | +
| Portrait LoRA(recommended) / FFT model | + |
+ |
+  |
+  |
+ "{image}, the man faces the camera." | +"Face swap" | +
| FFT model | ++ |  |
+  |
+  |
+ "{image} features a close-up of a young, furry tiger cub on a rock. The tiger, which appears to be quite young, has distinctive orange, black, and white striped fur, typical of tigers. The cub's eyes have a bright and curious expression, and its ears are perked up, indicating alertness. The cub seems to be in the act of climbing or resting on the rock. The background is a blurred grassland with trees, but the focus is on the cub, which is vividly colored while the rest of the image is in grayscale, drawing attention to the tiger's details. The photo captures a moment in the wild, depicting the charming and tenacious nature of this young tiger, as well as its typical interaction with the environment." | +"Super-resolution" | +
| FFT model | ++ | |
+  |
+  |
+ "a blue hand" | +"Regional Editing" | +
| FFT model | ++ | |
+  |
+  |
+ "Mechanical hands like a robot" | +"Regional Editing" | +
| Local Editing LoRA/FFT model | ++ |  |
+  |
+  |
+ "{image} Beautiful female portrait, Robot with smooth White transparent carbon shell, rococo detailing, Natural lighting, Highly detailed, Cinematic, 4K." | +"Recolorizing" | +
| Local Editing LoRA/FFT model | ++ |  |
+ |
+  |
+ "{image} Beautiful female portrait, Robot with smooth White transparent carbon shell, rococo detailing, Natural lighting, Highly detailed, Cinematic, 4K." | +"Depth Guided Generation" | +
| Local Editing LoRA/FFT model | ++ |  |
+ |
+  |
+ "{image} Beautiful female portrait, Robot with smooth White transparent carbon shell, rococo detailing, Natural lighting, Highly detailed, Cinematic, 4K." | +"Contour Guided Generation" | +
| Workflow | +Author | +Example | +
| 【飞翔鲸】王炸！FLUX+ACE++换头 | +飞翔鲸 | + |
+
| 最新ACE++ Redux 万物迁移 AI模特换装 UP 楚门的AI世界 | +楚门的AI世界 | + |
+
| Ace Plus+Fill+Redux稳定无抽卡换装工作流 | +T8star-Aix | + |
+
| Ace Plus+Fill+Redux稳定少抽卡标志工作流 | +T8star-Aix | + |
+
| Ace Plus+Fill+Redux稳定无抽卡换脸工作流 | +T8star-Aix | + |
+
| ace++ face swap in different styles | +jax | + |
+
| fllux ace++ subject without reference image | +leeguandong | + |
+
| Scepter-ACE++ More convenient replacement of everything | +HaoBeen | + |
+
| Workflow | +Description | +Other dependency models | +Setting | +
| ACE_Plus_LoRA_workflow_reference_generation.json | +Reference image generation capability for portrait or subject. | +Potrait or subject LoRA Model + FLUX.1-Fill-dev | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_LoRA_workflow_redux_reference_generation.json | +Reference image generation capability for portrait or subject used in conjunction with Redux. | +Potrait or subject LoRA Model + FLUX.1-Fill-dev + FLUX.1-Redux | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_LoRA_workflow_reference_editing.json | +Reference image editing capability such as logo paste, face swap. | +Potrait or subject LoRA Model + FLUX.1-Fill-dev | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_LoRA_workflow_redux_reference_editing.json | +Reference image editing capability such as logo paste, face swap used in conjunction with Redux. | +Potrait or subject LoRA Model + FLUX.1-Fill-dev + FLUX.1-Redux | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_LoRA_workflow_localcontrol_generation.json | +Controllable image-to-image translation capability. To preprocess depth and contour information from images, + we use externally-provided models that are typically downloaded from the ModelScope Hub. Because download success + can vary depending on the user's environment, we offer alternatives: users can either leverage existing community + nodes (depth extration node or contour extraction node) for this task (then choosing the 'no_preprocess' option), + or users can pre-download the required models + contour and + depth + and adjust + the configuration file 'workflow/ComfyUI-ACE_Plus/config/ace_plus_fft_processor.yaml' to + specify the models' local paths. | +Local editing LoRA Model + FLUX.1-Fill-dev + Preprocessing model (depth or contour) | +Task_type: contour_repainting/depth_repainting/recolorizing (you need to install dependencies like scepter) | +
| ACE_Plus_FFT_workflow_referenceediting_generation.json | +Reference image editing capability | +FFT model | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_FFT_workflow_no_preprocess.json | +Use the preprocessed images, such as depth and contour, as input, or the super-resolution. | +FFT model | +Task_type: no_preprocess (you don't need to install dependencies like scepter) | +
| ACE_Plus_FFT_workflow_controlpreprocess.json | +Controllable image-to-image translation capability. To preprocess depth and contour information from images, + we use externally-provided models that are typically downloaded from the ModelScope Hub. Because download success + can vary depending on the user's environment, we offer alternatives: users can either leverage existing community + nodes (depth extration node or contour extraction node) for this task (then choosing the 'no_preprocess' option), + or users can pre-download the required models + contour and + depth + and adjust + the configuration file 'workflow/ComfyUI-ACE_Plus/config/ace_plus_fft_processor.yaml' to + specify the models' local paths. | +FFT model | +Task_type: contour_repainting/depth_repainting/recolorizing (you need to install dependencies like scepter) | +
| ACE_Plus_FFT_workflow_reference_generation.json | +Reference image generation capability for portrait or subject. | +FFT model | +Task_type: repainting (you don't need to install dependencies like scepter) | +
| ACE_Plus_FFT_workflow_referenceediting_generation.json | +Reference image editing capability | +FFT model | +Task_type: repainting (you don't need to install dependencies like scepter) | +
+
+
+## ⚙️️ Installation
+Download the code using the following command:
+```bash
+git clone https://github.com/ali-vilab/ACE_plus.git
+```
+
+Install the necessary packages with `pip`:
+```bash
+cd ACE_plus
+pip install -r repo_requirements.txt
+```
+ACE++ depends on FLUX.1-Fill-dev as its base model, which you can download from [](https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev).
+In order to run the inference code or Gradio demo normally, we have defined the relevant environment variables to specify the location of the model.
+For model preparation, we provide three methods for downloading the model. The summary of relevant settings is as follows.
+
+| Model Downloading Method | Clone to Local Path | Automatic Downloading during Runtime