| <!--Copyright 2024 The HuggingFace Team. All rights reserved. |
|
|
| Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with |
| the License. You may obtain a copy of the License at |
|
|
| http://www.apache.org/licenses/LICENSE-2.0 |
|
|
| Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on |
| an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the |
| specific language governing permissions and limitations under the License. |
| --> |
|
|
| # Schedulers |
|
|
| 🤗 Diffusers provides many scheduler functions for the diffusion process. A scheduler takes a model's output (the sample which the diffusion process is iterating on) and a timestep to return a denoised sample. The timestep is important because it dictates where in the diffusion process the step is; data is generated by iterating forward *n* timesteps and inference occurs by propagating backward through the timesteps. Based on the timestep, a scheduler may be *discrete* in which case the timestep is an `int` or *continuous* in which case the timestep is a `float`. |
|
|
| Depending on the context, a scheduler defines how to iteratively add noise to an image or how to update a sample based on a model's output: |
|
|
| - during *training*, a scheduler adds noise (there are different algorithms for how to add noise) to a sample to train a diffusion model |
| - during *inference*, a scheduler defines how to update a sample based on a pretrained model's output |
|
|
| Many schedulers are implemented from the [k-diffusion](https://github.com/crowsonkb/k-diffusion) library by [Katherine Crowson](https://github.com/crowsonkb/), and they're also widely used in A1111. To help you map the schedulers from k-diffusion and A1111 to the schedulers in 🤗 Diffusers, take a look at the table below: |
|
|
| | A1111/k-diffusion | 🤗 Diffusers | Usage | |
| |---------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------| |
| | DPM++ 2M | [`DPMSolverMultistepScheduler`] | | |
| | DPM++ 2M Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` | |
| | DPM++ 2M SDE | [`DPMSolverMultistepScheduler`] | init with `algorithm_type="sde-dpmsolver++"` | |
| | DPM++ 2M SDE Karras | [`DPMSolverMultistepScheduler`] | init with `use_karras_sigmas=True` and `algorithm_type="sde-dpmsolver++"` | |
| | DPM++ 2S a | N/A | very similar to `DPMSolverSinglestepScheduler` | |
| | DPM++ 2S a Karras | N/A | very similar to `DPMSolverSinglestepScheduler(use_karras_sigmas=True, ...)` | |
| | DPM++ SDE | [`DPMSolverSinglestepScheduler`] | | |
| | DPM++ SDE Karras | [`DPMSolverSinglestepScheduler`] | init with `use_karras_sigmas=True` | |
| | DPM2 | [`KDPM2DiscreteScheduler`] | | |
| | DPM2 Karras | [`KDPM2DiscreteScheduler`] | init with `use_karras_sigmas=True` | |
| | DPM2 a | [`KDPM2AncestralDiscreteScheduler`] | | |
| | DPM2 a Karras | [`KDPM2AncestralDiscreteScheduler`] | init with `use_karras_sigmas=True` | |
| | DPM adaptive | N/A | | |
| | DPM fast | N/A | | |
| | Euler | [`EulerDiscreteScheduler`] | | |
| | Euler a | [`EulerAncestralDiscreteScheduler`] | | |
| | Heun | [`HeunDiscreteScheduler`] | | |
| | LMS | [`LMSDiscreteScheduler`] | | |
| | LMS Karras | [`LMSDiscreteScheduler`] | init with `use_karras_sigmas=True` | |
| | N/A | [`DEISMultistepScheduler`] | | |
| | N/A | [`UniPCMultistepScheduler`] | | |
|
|
| ## Noise schedules and schedule types |
| | A1111/k-diffusion | 🤗 Diffusers | |
| |--------------------------|----------------------------------------------------------------------------| |
| | Karras | init with `use_karras_sigmas=True` | |
| | sgm_uniform | init with `timestep_spacing="trailing"` | |
| | simple | init with `timestep_spacing="trailing"` | |
| | exponential | init with `timestep_spacing="linspace"`, `use_exponential_sigmas=True` | |
|
|
| All schedulers are built from the base [`SchedulerMixin`] class which implements low level utilities shared by all schedulers. |
|
|
| ## SchedulerMixin |
| [[autodoc]] SchedulerMixin |
|
|
| ## SchedulerOutput |
| [[autodoc]] schedulers.scheduling_utils.SchedulerOutput |
| |
| ## KarrasDiffusionSchedulers |
| |
| [`KarrasDiffusionSchedulers`] are a broad generalization of schedulers in 🤗 Diffusers. The schedulers in this class are distinguished at a high level by their noise sampling strategy, the type of network and scaling, the training strategy, and how the loss is weighed. |
| |
| The different schedulers in this class, depending on the ordinary differential equations (ODE) solver type, fall into the above taxonomy and provide a good abstraction for the design of the main schedulers implemented in 🤗 Diffusers. The schedulers in this class are given [here](https://github.com/huggingface/diffusers/blob/a69754bb879ed55b9b6dc9dd0b3cf4fa4124c765/src/diffusers/schedulers/scheduling_utils.py#L32). |
| |
| ## PushToHubMixin |
| |
| [[autodoc]] utils.PushToHubMixin |
| |
