data/model_data_json/ByteDance_Hyper-SD.json

{
    "model_id": "ByteDance/Hyper-SD",
    "downloads": 113274,
    "tags": [
        "diffusers",
        "lora",
        "text-to-image",
        "stable-diffusion",
        "flux",
        "arxiv:2404.13686",
        "base_model:black-forest-labs/FLUX.1-dev",
        "base_model:adapter:black-forest-labs/FLUX.1-dev",
        "region:us"
    ],
    "description": "--- library_name: diffusers inference: false tags: - lora - text-to-image - stable-diffusion - flux base_model: black-forest-labs/FLUX.1-dev --- # Hyper-SD Official Repository of the paper: *Hyper-SD*. Project Page: ## News🔥🔥🔥 * Aug.26, 2024. 💥💥💥 Our 8-steps and 16-steps **FLUX.1-dev-related LoRAs** are available now! We recommend LoRA scales around 0.125 that is adaptive with training and guidance scale could be kept on 3.5. Lower step LoRAs would be coming soon. 💥💥💥 * Aug.19, 2024. SD3-related CFG LoRAs are available now! We recommend setting guidance scale to 3.0/5.0/7.0 at 4/8/16-steps. Don't forget to fuse lora with a relatively small scale (e.g. 0.125 that is adaptive with training) before inference with diffusers. Note that 8-steps and 16-steps LoRA can also inference on a little bit smaller steps like 6-steps and 12-steps, respectively. Hope to hear your feedback, FLUX-related models will be coming next week. * May.13, 2024. The 12-Steps CFG-Preserved Hyper-SDXL-12steps-CFG-LoRA and Hyper-SD15-12steps-CFG-LoRA is also available now(support 5~8 guidance scales), this could be more practical with better trade-off between performance and speed. Enjoy! * Apr.30, 2024. Our 8-Steps CFG-Preserved Hyper-SDXL-8steps-CFG-LoRA and Hyper-SD15-8steps-CFG-LoRA is available now(support 5~8 guidance scales), we strongly recommend making the 8-step CFGLora a standard configuration for all SDXL and SD15 models!!! * Apr.28, 2024. ComfyUI workflows on 1-Step Unified LoRA 🥰 with TCDScheduler to inference on different steps are released! Remember to install ⭕️ ComfyUI-TCD in your folder!!! You're encouraged to adjust the eta parameter to get better results 🌟! * Apr.26, 2024. Thanks to @Pete for contributing to our scribble demo with larger canvas right now 👏. * Apr.24, 2024. The ComfyUI workflow and checkpoint on 1-Step SDXL UNet ✨ is also available! Don't forget ⭕️ to install the custom scheduler in your folder!!! * Apr.23, 2024. ComfyUI workflows on N-Steps LoRAs are released! Worth a try for creators 💥! * Apr.23, 2024. Our technical report 📚 is uploaded to arXiv! Many implementation details are provided and we welcome more discussions👏. * Apr.21, 2024. Hyper-SD ⚡️ is highly compatible and work well with different base models and controlnets. To clarify, we also append the usage example of controlnet here. * Apr.20, 2024. Our checkpoints and two demos 🤗 (i.e. SD15-Scribble and SDXL-T2I) are publicly available on HuggingFace Repo. ## Try our Hugging Face demos: Hyper-SD Scribble demo host on 🤗 scribble Hyper-SDXL One-step Text-to-Image demo host on 🤗 T2I ## Introduction Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques. In this repository, we release the models distilled from FLUX.1-dev, SD3-Medium, SDXL Base 1.0 and Stable-Diffusion v1-5。 ## Checkpoints * : Lora checkpoint, for FLUX.1-dev-related models. * : Lora checkpoint, for SD3-related models. * : Lora checkpoint, for SDXL-related models. * : Lora checkpoint, for SD1.5-related models. * : Unet checkpoint distilled from SDXL-Base. ## Text-to-Image Usage ### FLUX.1-dev-related models ### SD3-related models ### SDXL-related models #### 2-Steps, 4-Steps, 8-steps LoRA Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting. #### Unified LoRA (support 1 to 8 steps inference) You can flexibly adjust the number of inference steps and eta value to achieve best performance. #### 1-step SDXL Unet Only for the single step inference. ### SD1.5-related models #### 2-Steps, 4-Steps, 8-steps LoRA Take the 2-steps LoRA as an example, you can also use other LoRAs for the corresponding inference steps setting. #### Unified LoRA (support 1 to 8 steps inference) You can flexibly adjust the number of inference steps and eta value to achieve best performance. ## ControlNet Usage ### SDXL-related models #### 2-Steps, 4-Steps, 8-steps LoRA Take Canny Controlnet and 2-steps inference as an example: #### Unified LoRA (support 1 to 8 steps inference) Take Canny Controlnet as an example: ### SD1.5-related models #### 2-Steps, 4-Steps, 8-steps LoRA Take Canny Controlnet and 2-steps inference as an example: #### Unified LoRA (support 1 to 8 steps inference) Take Canny Controlnet as an example: ## Comfyui Usage * : text-to-image workflow * : text-to-image workflow * : text-to-image workflow * **REQUIREMENT / INSTALL** for 1-Step SDXL UNet: Please install our scheduler folder into your to enable sampling from 800 timestep instead of 999. * i.e. making sure the folder exist. * For more details, please refer to our technical report. * : text-to-image workflow * : text-to-image workflow * **REQUIREMENT / INSTALL** for 1-Step Unified LoRAs: Please install the ComfyUI-TCD into your to enable TCDScheduler with support of different inference steps (1~8) using single checkpoint. * i.e. making sure the folder exist. * You're encouraged to adjust the eta parameter in TCDScheduler to get better results. ## Citation",
    "model_explanation_gemini": "Hyper-SD accelerates text-to-image generation using LoRA-based distilled models for Stable Diffusion variants, enabling high-quality outputs with fewer inference steps."
}