Add pipeline tag and improve usage documentation

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +10 -1
README.md CHANGED
@@ -1,5 +1,6 @@
1
  ---
2
  license: mit
 
3
  tags:
4
  - diffusion
5
  - image-generation
@@ -19,7 +20,7 @@ tags:
19
 
20
  ## πŸ“„ Abstract
21
 
22
- We propose LayerSync, a domain-agnostic approach for improving the generation quality and the training efficiency of diffusion models. Prior studies have highlighted the connection between the quality of generation and the representations learned by diffusion models, showing that external guidance on model intermediate representations accelerates training. We reconceptualize this paradigm by regularizing diffusion models with their own intermediate representations. Building on the observation that representation quality varies across diffusion model layers, we show that the most semantically rich representations can act as an intrinsic guidance for weaker ones, reducing the need for external supervision. Our approach, LayerSync, is a self-sufficient, plug-and-play regularizer term with no overhead on diffusion model training and generalizes beyond the visual domain to other modalities. LayerSync requires no pretrained models nor additional data. We extensively evaluate the method on image generation and demonstrate its applicability to other domains such as audio, video, and human motion generation. LayerSync consistently enhances the generation quality and the training efficiency. For example, it speeds up the training of flow-based transformer by over 8.75x on the ImageNet dataset and improves the generation quality by 23.6%.
23
 
24
  ---
25
 
@@ -36,6 +37,14 @@ This repository contains pretrained model checkpoints:
36
 
37
  Download the checkpoints and use them with the [LayerSync codebase](https://github.com/vita-epfl/LayerSync).
38
 
 
 
 
 
 
 
 
 
39
  ---
40
 
41
  ## πŸ“š Citation
 
1
  ---
2
  license: mit
3
+ pipeline_tag: unconditional-image-generation
4
  tags:
5
  - diffusion
6
  - image-generation
 
20
 
21
  ## πŸ“„ Abstract
22
 
23
+ We propose LayerSync, a domain-agnostic approach for improving the generation quality and the training efficiency of diffusion models. Prior studies have highlighted the connection between the quality of generation and the representations learned by diffusion models, showing that external guidance on model intermediate representations accelerates training. We reconceptualize this paradigm by regularizing diffusion models with their own intermediate representations. Our approach, LayerSync, is a self-sufficient, plug-and-play regularizer term with no overhead on diffusion model training and generalizes beyond the visual domain to other modalities.
24
 
25
  ---
26
 
 
37
 
38
  Download the checkpoints and use them with the [LayerSync codebase](https://github.com/vita-epfl/LayerSync).
39
 
40
+ To generate images using the provided scripts:
41
+
42
+ ```bash
43
+ torchrun --nnodes=1 --nproc_per_node=N sample_ddp.py ODE \
44
+ --model SiT-XL/2 \
45
+ --num-fid-samples 50000
46
+ ```
47
+
48
  ---
49
 
50
  ## πŸ“š Citation