diffusers/pr_13881/en/api/normalization.md

# Normalization layers

Customized normalization layers for supporting various models in 🤗 Diffusers.

## AdaLayerNorm[[diffusers.models.normalization.AdaLayerNorm]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`, *optional*) -- The size of the embeddings dictionary.
- **output_dim** (`int`, *optional*) --
- **norm_elementwise_affine** (`bool`, defaults to `False) --
- **norm_eps** (`bool`, defaults to `False`) --
- **chunk_dim** (`int`, defaults to `0`) --

Norm layer modified to incorporate timestep embeddings.

## AdaLayerNormZero[[diffusers.models.normalization.AdaLayerNormZero]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`) -- The size of the embeddings dictionary.

Norm layer adaptive layer norm zero (adaLN-Zero).

## AdaLayerNormSingle[[diffusers.models.normalization.AdaLayerNormSingle]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **use_additional_conditions** (`bool`) -- To use additional conditions for normalization or not.

Norm layer adaptive layer norm single (adaLN-single).

As proposed in PixArt-Alpha (see: https://huggingface.co/papers/2310.00426; Section 2.3).

## AdaGroupNorm[[diffusers.models.normalization.AdaGroupNorm]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`) -- The size of the embeddings dictionary.
- **num_groups** (`int`) -- The number of groups to separate the channels into.
- **act_fn** (`str`, *optional*, defaults to `None`) -- The activation function to use.
- **eps** (`float`, *optional*, defaults to `1e-5`) -- The epsilon value to use for numerical stability.

GroupNorm layer modified to incorporate timestep embeddings.

## AdaLayerNormContinuous[[diffusers.models.normalization.AdaLayerNormContinuous]]

- **embedding_dim** (`int`) -- Embedding dimension to use during projection.
- **conditioning_embedding_dim** (`int`) -- Dimension of the input condition.
- **elementwise_affine** (`bool`, defaults to `True`) --
  Boolean flag to denote if affine transformation should be applied.
- **eps** (`float`, defaults to 1e-5) -- Epsilon factor.
- **bias** (`bias`, defaults to `True`) -- Boolean flag to denote if bias should be use.
- **norm_type** (`str`, defaults to `"layer_norm"`) --
  Normalization layer to use. Values supported: "layer_norm", "rms_norm".

Adaptive normalization layer with a norm layer (layer_norm or rms_norm).

## RMSNorm[[diffusers.models.normalization.RMSNorm]]

- **dim** (`int`) -- Number of dimensions to use for `weights`. Only effective when `elementwise_affine` is True.
- **eps** (`float`) -- Small value to use when calculating the reciprocal of the square-root.
- **elementwise_affine** (`bool`, defaults to `True`) --
  Boolean flag to denote if affine transformation should be applied.
- **bias** (`bool`, defaults to False) -- If also training the `bias` param.

RMS Norm as introduced in https://huggingface.co/papers/1910.07467 by Zhang et al.

## GlobalResponseNorm[[diffusers.models.normalization.GlobalResponseNorm]]

- **dim** (`int`) -- Number of dimensions to use for the `gamma` and `beta`.

Global response normalization as introduced in ConvNeXt-v2 (https://huggingface.co/papers/2301.00808).

## LuminaLayerNormContinuous[[diffusers.models.normalization.LuminaLayerNormContinuous]]

## SD35AdaLayerNormZeroX[[diffusers.models.normalization.SD35AdaLayerNormZeroX]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`) -- The size of the embeddings dictionary.

Norm layer adaptive layer norm zero (AdaLN-Zero).

## AdaLayerNormZeroSingle[[diffusers.models.normalization.AdaLayerNormZeroSingle]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`) -- The size of the embeddings dictionary.

Norm layer adaptive layer norm zero (adaLN-Zero).

## LuminaRMSNormZero[[diffusers.models.normalization.LuminaRMSNormZero]]

- **embedding_dim** (`int`) -- The size of each embedding vector.

Norm layer adaptive RMS normalization zero.

## LpNorm[[diffusers.models.normalization.LpNorm]]

## CogView3PlusAdaLayerNormZeroTextImage[[diffusers.models.normalization.CogView3PlusAdaLayerNormZeroTextImage]]

- **embedding_dim** (`int`) -- The size of each embedding vector.
- **num_embeddings** (`int`) -- The size of the embeddings dictionary.

Norm layer adaptive layer norm zero (adaLN-Zero).

## CogVideoXLayerNormZero[[diffusers.models.normalization.CogVideoXLayerNormZero]]

## MochiRMSNormZero[[diffusers.models.transformers.transformer_mochi.MochiRMSNormZero]]

- **embedding_dim** (`int`) -- The size of each embedding vector.

Adaptive RMS Norm used in Mochi.

## MochiRMSNorm[[diffusers.models.normalization.MochiRMSNorm]]