Instructions to use thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora with Diffusers:
pip install -U diffusers transformers accelerate
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline controlnet = ControlNetModel.from_pretrained("thedeoxen/refcontrol-FLUX.2-klein-9B-reference-pose-lora") pipe = StableDiffusionControlNetPipeline.from_pretrained( "black-forest-labs/FLUX.2-klein-base-9B", controlnet=controlnet ) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Did you train this to be used for 1 mega pixel?
Resolutions like 2 mega pixels produce weird anatomy problems.
Flux Klein 9B can go to 4 mega pixels easily.
Trying to figure out what is the problem.
What strength of this lora to use and what effects it has?
What generation resolutions x/y you suggest should be best to avoid problems?
hey I used 1 mega pixel when did training so it possible can affect to results.
you can use about 0.7-1 lora strength.
Trained and tested with scale to 1 megapixel.
Possible train with higher resolution can help but it will require more VRAM from me.
I investigated the topic further.
I have good results with higher resolutions, but it depends on reference aspect ratio compared to the pose image aspect ratio (depth for example).
Also similar initial poses of depth image and image reference produce better results.
It is good also to manipulate CFG = 3.0 - 5.5
I will investigate more and come back her.
You have done a great job - basically added controlnet to Flux 9B Klein. Thank you.