| --- |
| license: other |
| license_name: flux-1-dev-non-commercial-license |
| license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md |
|
|
| language: |
| - en |
| library_name: diffusers |
| pipeline_tag: text-to-image |
|
|
| tags: |
| - Text-to-Image |
| - ControlNet |
| - Diffusers |
| - Flux.1-dev |
| - image-generation |
| - Stable Diffusion |
| base_model: black-forest-labs/FLUX.1-dev |
| --- |
| |
| # FLUX.1-dev-ControlNet-Depth |
|
|
| This repository contains a Depth ControlNet for FLUX.1-dev model jointly trained by researchers from [InstantX Team](https://huggingface.co/InstantX) and [Shakker Labs](https://huggingface.co/Shakker-Labs). |
|
|
| # Model Cards |
| - The model consists of 4 FluxTransformerBlock and 1 FluxSingleTransformerBlock. |
| - This checkpoint is trained on both real and generated image datasets. with 16\*A800 for 50K steps. The batch size 16\*4=64 with resolution=1024. The learning rate is set to 5e-6. |
| - The recommended controlnet_conditioning_scale is 0.3-0.7. |
|
|
| # Showcases |
|
|
| <div class="container"> |
| <img src="./assets/teaser.png" width="1024"/> |
| </div> |
|
|
|
|
| # Inference |
| ```python |
| import torch |
| from diffusers.utils import load_image |
| from diffusers import FluxControlNetPipeline, FluxControlNetModel |
| |
| controlnet_model = "black-forest-labs/FLUX.1-dev" |
| base_model = "Shakker-Labs/FLUX.1-dev-ControlNet-Depth" |
| |
| controlnet = FluxControlNetModel.from_pretrained(controlnet_model, torch_dtype=torch.bfloat16) |
| pipe = FluxControlNetPipeline.from_pretrained( |
| base_model, controlnet=controlnet, torch_dtype=torch.bfloat16 |
| ) |
| pipe.to("cuda") |
| |
| control_image = load_image("https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Depth/resolve/main/assets/cond1.png") |
| prompt = "an old man with white hair" |
| |
| image = pipe(prompt, |
| control_image=control_image, |
| controlnet_conditioning_scale=0.5, |
| width=control_image.size[0], |
| height=control_image.size[1], |
| num_inference_steps=24, |
| guidance_scale=3.5, |
| ).images[0] |
| ``` |
|
|
| For multi-ControlNets support, please refer to [Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro](https://huggingface.co/Shakker-Labs/FLUX.1-dev-ControlNet-Union-Pro). |
|
|
| # Acknowledgements |
| This project is released by [Shakker AI](https://www.shakker.ai/). All copyright reserved. |
|
|
