diffusers/pr_12652/en/api/schedulers/overview.md

# Schedulers

🤗 Diffusers provides many scheduler functions for the diffusion process. A scheduler takes a model's output (the sample which the diffusion process is iterating on) and a timestep to return a denoised sample. The timestep is important because it dictates where in the diffusion process the step is; data is generated by iterating forward *n* timesteps and inference occurs by propagating backward through the timesteps. Based on the timestep, a scheduler may be *discrete* in which case the timestep is an `int` or *continuous* in which case the timestep is a `float`.

Depending on the context, a scheduler defines how to iteratively add noise to an image or how to update a sample based on a model's output:

- during *training*, a scheduler adds noise (there are different algorithms for how to add noise) to a sample to train a diffusion model
- during *inference*, a scheduler defines how to update a sample based on a pretrained model's output

Many schedulers are implemented from the [k-diffusion](https://github.com/crowsonkb/k-diffusion) library by [Katherine Crowson](https://github.com/crowsonkb/), and they're also widely used in A1111. To help you map the schedulers from k-diffusion and A1111 to the schedulers in 🤗 Diffusers, take a look at the table below:

| A1111/k-diffusion    | 🤗 Diffusers                         | Usage                                                                                                         |
|---------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------|
| DPM++ 2M            | [DPMSolverMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler)     |                                                                                                               |
| DPM++ 2M Karras     | [DPMSolverMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler)     | init with `use_karras_sigmas=True`                                                                            |
| DPM++ 2M SDE        | [DPMSolverMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler)     | init with `algorithm_type="sde-dpmsolver++"`                                                                  |
| DPM++ 2M SDE Karras | [DPMSolverMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/multistep_dpm_solver#diffusers.DPMSolverMultistepScheduler)     | init with `use_karras_sigmas=True` and `algorithm_type="sde-dpmsolver++"`                                     |
| DPM++ 2S a          | N/A                                 | very similar to  `DPMSolverSinglestepScheduler`                         |
| DPM++ 2S a Karras   | N/A                                 | very similar to  `DPMSolverSinglestepScheduler(use_karras_sigmas=True, ...)` |
| DPM++ SDE           | [DPMSolverSinglestepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/singlestep_dpm_solver#diffusers.DPMSolverSinglestepScheduler)    |                                                                                                               |
| DPM++ SDE Karras    | [DPMSolverSinglestepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/singlestep_dpm_solver#diffusers.DPMSolverSinglestepScheduler)    | init with `use_karras_sigmas=True`                                                                            |
| DPM2                | [KDPM2DiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/dpm_discrete#diffusers.KDPM2DiscreteScheduler)          |                                                                                                               |
| DPM2 Karras         | [KDPM2DiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/dpm_discrete#diffusers.KDPM2DiscreteScheduler)          | init with `use_karras_sigmas=True`                                                                            |
| DPM2 a              | [KDPM2AncestralDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/dpm_discrete_ancestral#diffusers.KDPM2AncestralDiscreteScheduler) |                                                                                                               |
| DPM2 a Karras       | [KDPM2AncestralDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/dpm_discrete_ancestral#diffusers.KDPM2AncestralDiscreteScheduler) | init with `use_karras_sigmas=True`                                                                            |
| DPM adaptive        | N/A                                 |                                                                                                               |
| DPM fast            | N/A                                 |                                                                                                               |
| Euler               | [EulerDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/euler#diffusers.EulerDiscreteScheduler)          |                                                                                                               |
| Euler a             | [EulerAncestralDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/euler_ancestral#diffusers.EulerAncestralDiscreteScheduler) |                                                                                                               |
| Heun                | [HeunDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/heun#diffusers.HeunDiscreteScheduler)           |                                                                                                               |
| LMS                 | [LMSDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler)            |                                                                                                               |
| LMS Karras          | [LMSDiscreteScheduler](/docs/diffusers/pr_12652/en/api/schedulers/lms_discrete#diffusers.LMSDiscreteScheduler)            | init with `use_karras_sigmas=True`                                                                            |
| N/A                 | [DEISMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/deis#diffusers.DEISMultistepScheduler)          |                                                                                                               |
| N/A                 | [UniPCMultistepScheduler](/docs/diffusers/pr_12652/en/api/schedulers/unipc#diffusers.UniPCMultistepScheduler)         |                                                                                                               |

## Noise schedules and schedule types
| A1111/k-diffusion        | 🤗 Diffusers                                                               |
|--------------------------|----------------------------------------------------------------------------|
| Karras                   | init with `use_karras_sigmas=True`                                         |
| sgm_uniform              | init with `timestep_spacing="trailing"`                                    |
| simple                   | init with `timestep_spacing="trailing"`                                    |
| exponential              | init with `timestep_spacing="linspace"`, `use_exponential_sigmas=True`     |
| beta                     | init with `timestep_spacing="linspace"`, `use_beta_sigmas=True`            |

All schedulers are built from the base [SchedulerMixin](/docs/diffusers/pr_12652/en/api/schedulers/overview#diffusers.SchedulerMixin) class which implements low level utilities shared by all schedulers.

## SchedulerMixin[[diffusers.SchedulerMixin]]
#### diffusers.SchedulerMixin[[diffusers.SchedulerMixin]]

[Source](https://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/schedulers/scheduling_utils.py#L74)

Base class for all schedulers.

[SchedulerMixin](/docs/diffusers/pr_12652/en/api/schedulers/overview#diffusers.SchedulerMixin) contains common functions shared by all schedulers such as general loading and saving
functionalities.

[ConfigMixin](/docs/diffusers/pr_12652/en/api/configuration#diffusers.ConfigMixin) takes care of storing the configuration attributes (like `num_train_timesteps`) that are passed to
the scheduler's `__init__` function, and the attributes can be accessed by `scheduler.config.num_train_timesteps`.

Class attributes:
- **_compatibles** (`list[str]`) -- A list of scheduler classes that are compatible with the parent scheduler
  class. Use [from_config()](/docs/diffusers/pr_12652/en/api/configuration#diffusers.ConfigMixin.from_config) to load a different compatible scheduler class (should be overridden
  by parent class).

from_pretraineddiffusers.SchedulerMixin.from_pretrainedhttps://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/schedulers/scheduling_utils.py#L94[{"name": "pretrained_model_name_or_path", "val": ": str | os.PathLike | None = None"}, {"name": "subfolder", "val": ": str | None = None"}, {"name": "return_unused_kwargs", "val": " = False"}, {"name": "**kwargs", "val": ""}]- **pretrained_model_name_or_path** (`str` or `os.PathLike`, *optional*) --
  Can be either:

  - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on
    the Hub.
  - A path to a *directory* (for example `./my_model_directory`) containing the scheduler
    configuration saved with [save_pretrained()](/docs/diffusers/pr_12652/en/api/schedulers/overview#diffusers.SchedulerMixin.save_pretrained).
- **subfolder** (`str`, *optional*) --
  The subfolder location of a model file within a larger model repository on the Hub or locally.
- **return_unused_kwargs** (`bool`, *optional*, defaults to `False`) --
  Whether kwargs that are not consumed by the Python class should be returned or not.
- **cache_dir** (`str | os.PathLike`, *optional*) --
  Path to a directory where a downloaded pretrained model configuration is cached if the standard cache
  is not used.
- **force_download** (`bool`, *optional*, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.

- **proxies** (`dict[str, str]`, *optional*) --
  A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128',
  'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.
- **output_loading_info(`bool`,** *optional*, defaults to `False`) --
  Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages.
- **local_files_only(`bool`,** *optional*, defaults to `False`) --
  Whether to only load local model weights and configuration files or not. If set to `True`, the model
  won't be downloaded from the Hub.
- **token** (`str` or *bool*, *optional*) --
  The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from
  `diffusers-cli login` (stored in `~/.huggingface`) is used.
- **revision** (`str`, *optional*, defaults to `"main"`) --
  The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier
  allowed by Git.0

Instantiate a scheduler from a pre-defined JSON configuration file in a local directory or Hub repository.

> [!TIP] > To use private or [gated models](https://huggingface.co/docs/hub/models-gated#gated-models), log-in
with `hf > auth login`. You can also activate the special >
["offline-mode"](https://huggingface.co/diffusers/installation.html#offline-mode) to use this method in a >
firewalled environment.

**Parameters:**

pretrained_model_name_or_path (`str` or `os.PathLike`, *optional*) : Can be either:  - A string, the *model id* (for example `google/ddpm-celebahq-256`) of a pretrained model hosted on the Hub. - A path to a *directory* (for example `./my_model_directory`) containing the scheduler configuration saved with [save_pretrained()](/docs/diffusers/pr_12652/en/api/schedulers/overview#diffusers.SchedulerMixin.save_pretrained).

subfolder (`str`, *optional*) : The subfolder location of a model file within a larger model repository on the Hub or locally.

return_unused_kwargs (`bool`, *optional*, defaults to `False`) : Whether kwargs that are not consumed by the Python class should be returned or not.

cache_dir (`str | os.PathLike`, *optional*) : Path to a directory where a downloaded pretrained model configuration is cached if the standard cache is not used.

force_download (`bool`, *optional*, defaults to `False`) : Whether or not to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. 

proxies (`dict[str, str]`, *optional*) : A dictionary of proxy servers to use by protocol or endpoint, for example, `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request.

output_loading_info(`bool`, *optional*, defaults to `False`) : Whether or not to also return a dictionary containing missing keys, unexpected keys and error messages.

local_files_only(`bool`, *optional*, defaults to `False`) : Whether to only load local model weights and configuration files or not. If set to `True`, the model won't be downloaded from the Hub.

token (`str` or *bool*, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True`, the token generated from `diffusers-cli login` (stored in `~/.huggingface`) is used.

revision (`str`, *optional*, defaults to `"main"`) : The specific model version to use. It can be a branch name, a tag name, a commit id, or any identifier allowed by Git.
#### save_pretrained[[diffusers.SchedulerMixin.save_pretrained]]

[Source](https://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/schedulers/scheduling_utils.py#L155)

Save a scheduler configuration object to a directory so that it can be reloaded using the
[from_pretrained()](/docs/diffusers/pr_12652/en/api/schedulers/overview#diffusers.SchedulerMixin.from_pretrained) class method.

**Parameters:**

save_directory (`str` or `os.PathLike`) : Directory where the configuration JSON file will be saved (will be created if it does not exist).

push_to_hub (`bool`, *optional*, defaults to `False`) : Whether or not to push your model to the Hugging Face Hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace).

kwargs (`dict[str, Any]`, *optional*) : Additional keyword arguments passed along to the [push_to_hub()](/docs/diffusers/pr_12652/en/api/pipelines/overview#diffusers.utils.PushToHubMixin.push_to_hub) method.

## SchedulerOutput[[diffusers.schedulers.scheduling_utils.SchedulerOutput]]
#### diffusers.schedulers.scheduling_utils.SchedulerOutput[[diffusers.schedulers.scheduling_utils.SchedulerOutput]]

[Source](https://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/schedulers/scheduling_utils.py#L61)

Base class for the output of a scheduler's `step` function.

**Parameters:**

prev_sample (`torch.Tensor` of shape `(batch_size, num_channels, height, width)` for images) : Computed sample `(x_{t-1})` of previous timestep. `prev_sample` should be used as next model input in the denoising loop.

## KarrasDiffusionSchedulers

`KarrasDiffusionSchedulers` are a broad generalization of schedulers in 🤗 Diffusers. The schedulers in this class are distinguished at a high level by their noise sampling strategy, the type of network and scaling, the training strategy, and how the loss is weighed.

The different schedulers in this class, depending on the ordinary differential equations (ODE) solver type, fall into the above taxonomy and provide a good abstraction for the design of the main schedulers implemented in 🤗 Diffusers. The schedulers in this class are given [here](https://github.com/huggingface/diffusers/blob/a69754bb879ed55b9b6dc9dd0b3cf4fa4124c765/src/diffusers/schedulers/scheduling_utils.py#L32).

## PushToHubMixin[[diffusers.utils.PushToHubMixin]]

#### diffusers.utils.PushToHubMixin[[diffusers.utils.PushToHubMixin]]

[Source](https://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/utils/hub_utils.py#L483)

A Mixin to push a model, scheduler, or pipeline to the Hugging Face Hub.

push_to_hubdiffusers.utils.PushToHubMixin.push_to_hubhttps://github.com/huggingface/diffusers/blob/vr_12652/src/diffusers/utils/hub_utils.py#L518[{"name": "repo_id", "val": ": str"}, {"name": "commit_message", "val": ": str | None = None"}, {"name": "private", "val": ": bool | None = None"}, {"name": "token", "val": ": str | None = None"}, {"name": "create_pr", "val": ": bool = False"}, {"name": "safe_serialization", "val": ": bool = True"}, {"name": "variant", "val": ": str | None = None"}, {"name": "subfolder", "val": ": str | None = None"}]- **repo_id** (`str`) --
  The name of the repository you want to push your model, scheduler, or pipeline files to. It should
  contain your organization name when pushing to an organization. `repo_id` can also be a path to a local
  directory.
- **commit_message** (`str`, *optional*) --
  Message to commit while pushing. Default to `"Upload {object}"`.
- **private** (`bool`, *optional*) --
  Whether to make the repo private. If `None` (default), the repo will be public unless the
  organization's default is private. This value is ignored if the repo already exists.
- **token** (`str`, *optional*) --
  The token to use as HTTP bearer authorization for remote files. The token generated when running `hf
  auth login` (stored in `~/.huggingface`).
- **create_pr** (`bool`, *optional*, defaults to `False`) --
  Whether or not to create a PR with the uploaded files or directly commit.
- **safe_serialization** (`bool`, *optional*, defaults to `True`) --
  Whether or not to convert the model weights to the `safetensors` format.
- **variant** (`str`, *optional*) --
  If specified, weights are saved in the format `pytorch_model..bin`.0

Upload model, scheduler, or pipeline files to the 🤗 Hugging Face Hub.

Examples:

```python
from diffusers import UNet2DConditionModel

unet = UNet2DConditionModel.from_pretrained("stabilityai/stable-diffusion-2", subfolder="unet")

# Push the `unet` to your namespace with the name "my-finetuned-unet".
unet.push_to_hub("my-finetuned-unet")

# Push the `unet` to an organization with the name "my-finetuned-unet".
unet.push_to_hub("your-org/my-finetuned-unet")
```

**Parameters:**

repo_id (`str`) : The name of the repository you want to push your model, scheduler, or pipeline files to. It should contain your organization name when pushing to an organization. `repo_id` can also be a path to a local directory.

commit_message (`str`, *optional*) : Message to commit while pushing. Default to `"Upload {object}"`.

private (`bool`, *optional*) : Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.

token (`str`, *optional*) : The token to use as HTTP bearer authorization for remote files. The token generated when running `hf auth login` (stored in `~/.huggingface`).

create_pr (`bool`, *optional*, defaults to `False`) : Whether or not to create a PR with the uploaded files or directly commit.

safe_serialization (`bool`, *optional*, defaults to `True`) : Whether or not to convert the model weights to the `safetensors` format.

variant (`str`, *optional*) : If specified, weights are saved in the format `pytorch_model..bin`.