Instructions to use XLabs-AI/flux-controlnet-collections with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use XLabs-AI/flux-controlnet-collections with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("XLabs-AI/flux-controlnet-collections", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps
- Draw Things
- DiffusionBee
Not working in ComfyUI
#3
by Betadyne - opened
Can we use it for both?
I tried v2 in ComfyUI with your workflow (I use schnell-fp8 with embedded vae)
And it throw error:
Error occurred when executing LoadFluxControlNet:
Error(s) in loading state_dict for ControlNetFlux:
Unexpected key(s) in state_dict: "guidance_in.in_layer.bias", "guidance_in.in_layer.weight", "guidance_in.out_layer.bias", "guidance_in.out_layer.weight".
Betadyne changed discussion title from flux-canny-depth_v2.safetensors ??? to Not working in ComfyUI
you need to use dev version
Any plan to make it work with schnell-fp8?
For this better wait for tier-dev. Even the Xlab-Lora loader give error. ..
I tried another loader form rgthree with shnell and .. like a breeze...