Sample-Aligned Decoders for Latent Generative Models

Yu Lei1,2, Yi Li2, Hongcheng Gao3, Yipeng Wei2, Liang Li1, Li Su2, Qingming Huang2

1Institute of Computing Technology, Chinese Academy of Sciences
2University of Chinese Academy of Sciences   3Tsinghua University

Paper · Project page · Code

These are drop-in decoders for latent generative models. Each file holds only the decoder (decoder.*, post_quant_conv.*) of the original autoencoder. It was fine-tuned with SAD on latents from one generator. The encoder, the latent space and the generator stay unchanged, and so does the inference cost.

Usage

pip install git+https://github.com/torpedo2003/SAD
from diffusers import DiTPipeline
from sad import load_sad_decoder

pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256").to("cuda")
load_sad_decoder(pipe.vae, "dit-xl2-256/sad-gan")   # swaps the decoder in place
images = pipe(class_labels=[279, 388], num_inference_steps=250, guidance_scale=1.5).images

You can also load the weights without our package:

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
pipe.vae.load_state_dict(load_file(hf_hub_download("torpedo2003/SAD", "dit-xl2-256/sad-gan.safetensors")), strict=False)

The VA-VAE and E2E-VAE decoders are not diffusers models. Load them into the autoencoder classes of the LightningDiT and REPA-E code bases, using the same load_sad_decoder call.

Files

Each decoder is trained on latents from its own generator and sampler, so use the one that matches your pipeline.

File Generator Tokenizer Sampler
dit-xl2-256/sad-gan.safetensors DiT-XL/2 SD-VAE (ft-EMA) official 250-step DDPM, cfg 1.5
dit-xl2-256/sad-mmd.safetensors DiT-XL/2 SD-VAE (ft-EMA) official 250-step DDPM, cfg 1.5
dit-xl2-256-dpm50/sad-gan.safetensors DiT-XL/2 SD-VAE (ft-EMA) 50-step DPM-Solver++, cfg 1.5
dit-xl2-256-dpm50/sad-mmd.safetensors DiT-XL/2 SD-VAE (ft-EMA) 50-step DPM-Solver++, cfg 1.5
sit-xl2-repa-256/sad-gan.safetensors SiT-XL/2 + REPA SD-VAE (ft-EMA) SDE 250 steps
lightningdit-xl-vavae-256/sad-gan.safetensors LightningDiT-XL VA-VAE official recipe
repa-e-e2evae-256/sad-gan.safetensors REPA-E (SiT-XL/1) E2E-VAE SDE 250 steps
pixart-alpha-256/sad-gan.safetensors PixArt-α 256px SD-VAE (ft-EMA) 20 steps, guidance 4.5

Every decoder was trained for 2,000 steps, about one GPU-hour. Training used the GAN discrepancy in DINOv2 space (λ = 0.05 on ImageNet, 0.01 for PixArt-α) or MMD (λ = 1), plus the L1 encoder anchor. To train a decoder for your own generator, see the training guide.

Citation

@article{lei2026sad,
  title   = {Sample-Aligned Decoders for Latent Generative Models},
  author  = {Lei, Yu and Li, Yi and Gao, Hongcheng and Wei, Yipeng and Li, Liang and Su, Li and Huang, Qingming},
  journal = {arXiv preprint arXiv:2611.22458},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for torpedo2003/SAD