Instructions to use thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora with Diffusers:
pip install -U diffusers transformers accelerate
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline controlnet = ControlNetModel.from_pretrained("thedeoxen/refcontrol-FLUX.2-klein-9B-reference-canny-lora") pipe = StableDiffusionControlNetPipeline.from_pretrained( "black-forest-labs/FLUX.2-klein-base-9B", controlnet=controlnet ) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Regarding the LORA training process
May I ask how you trained the FLUX.2 Klein model? Are there any training details or procedures you could share? Which model architectures were used in the LORA training?
I noticed that in your previous FLUX.1 Kontext model, the training data consisted of the original image on the left and the control condition on the right, which were concatenated as input and output. Is this also the case for FLUX.2 Klein, or are img1 and img2 fed separately as the original image and the control, respectively?
I look forward to your reply. Thank you very much.
hey @ezramo for klen training it was just different input images first image was canny, second image is reference.
There is no need to concatinating images as I did for kontext (because kontext didn't support multiple images input)
I used ai-toolkit with pretty default parameters for training.
hey @ezramo for klen training it was just different input images first image was canny, second image is reference.
There is no need to concatinating images as I did for kontext (because kontext didn't support multiple images input)
I used ai-toolkit with pretty default parameters for training.
I see. Thank you.
hey @ezramo for klen training it was just different input images first image was canny, second image is reference.
There is no need to concatinating images as I did for kontext (because kontext didn't support multiple images input)
I used ai-toolkit with pretty default parameters for training.
I would also like to ask whether the resolution of the original images and control images in the training data is uniformly 1024×1024.
Images have different aspect ratio/sizes, I didn't change them to square.
For train I used different resolutions like 512, 768, 1024
