diffusers/pr_12595/en/api/models/allegro_transformer3d.md

AllegroTransformer3DModel

A Diffusion Transformer model for 3D data from Allegro was introduced in Allegro: Open the Black Box of Commercial-Level Video Generation Model by RhymesAI.

The model can be loaded with the following code snippet.

from diffusers import AllegroTransformer3DModel

transformer = AllegroTransformer3DModel.from_pretrained("rhymes-ai/Allegro", subfolder="transformer", torch_dtype=torch.bfloat16).to("cuda")

AllegroTransformer3DModel[[diffusers.AllegroTransformer3DModel]]

class diffusers.AllegroTransformer3DModeldiffusers.AllegroTransformer3DModelhttps://github.com/huggingface/diffusers/blob/vr_12595/src/diffusers/models/transformers/transformer_allegro.py#L176[{"name": "patch_size", "val": ": int = 2"}, {"name": "patch_size_t", "val": ": int = 1"}, {"name": "num_attention_heads", "val": ": int = 24"}, {"name": "attention_head_dim", "val": ": int = 96"}, {"name": "in_channels", "val": ": int = 4"}, {"name": "out_channels", "val": ": int = 4"}, {"name": "num_layers", "val": ": int = 32"}, {"name": "dropout", "val": ": float = 0.0"}, {"name": "cross_attention_dim", "val": ": int = 2304"}, {"name": "attention_bias", "val": ": bool = True"}, {"name": "sample_height", "val": ": int = 90"}, {"name": "sample_width", "val": ": int = 160"}, {"name": "sample_frames", "val": ": int = 22"}, {"name": "activation_fn", "val": ": str = 'gelu-approximate'"}, {"name": "norm_elementwise_affine", "val": ": bool = False"}, {"name": "norm_eps", "val": ": float = 1e-06"}, {"name": "caption_channels", "val": ": int = 4096"}, {"name": "interpolation_scale_h", "val": ": float = 2.0"}, {"name": "interpolation_scale_w", "val": ": float = 2.0"}, {"name": "interpolation_scale_t", "val": ": float = 2.2"}]

Transformer2DModelOutput[[diffusers.models.modeling_outputs.Transformer2DModelOutput]]

class diffusers.models.modeling_outputs.Transformer2DModelOutputdiffusers.models.modeling_outputs.Transformer2DModelOutputhttps://github.com/huggingface/diffusers/blob/vr_12595/src/diffusers/models/modeling_outputs.py#L21[{"name": "sample", "val": ": torch.Tensor"}]- sample (torch.Tensor of shape (batch_size, num_channels, height, width) or (batch size, num_vector_embeds - 1, num_latent_pixels) if Transformer2DModel is discrete) -- The hidden states output conditioned on the encoder_hidden_states input. If discrete, returns probability distributions for the unnoised latent pixels.0

The output of Transformer2DModel.