File size: 2,382 Bytes
73b7aad | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | ---
license: apache-2.0
base_model: Efficient-Large-Model/Sana_600M_512px_diffusers
library_name: peft
tags:
- diffusion
- distillation
- sana
- lora
- text-to-image
---
# Diffusion Distillation Workshop: 4-step Sana-600M students
LoRA adapters that distill [Sana-600M-512px](https://huggingface.co/Efficient-Large-Model/Sana_600M_512px_diffusers) from 40 NFE (20 steps with CFG 4.5) into **4-NFE students** using four methods. They were trained with the workshop notebook on one H100. Data: [`leffff/diffusion-distillation-workshop`](https://huggingface.co/datasets/leffff/diffusion-distillation-workshop).
| file | method | sampler (4 NFE) | training | CLIP ↑ | diversity | latent MSE to teacher |
|---|---|---|---|---|---|---|
| teacher | — | 20 Euler steps + CFG (40 NFE) | — | 32.29 | 0.075 | 0 |
| teacher | — | 4 Euler steps + CFG (8 NFE) | — | 30.73 | 0.117 | 0.54 |
| `adapter_cm.pt` | multistep (phased) consistency distillation | Euler | 5 min, 933 it × 16 | 32.67 | 0.088 | 0.41 |
| `adapter_ladd.pt` | LADD (adversarial, teacher-feature discriminator) | x0 + renoise | 5 min, 853 it × 16 | 31.95 | 0.101 | 1.42 |
| `adapter_dmd.pt` | DMD2 (reverse KL, fake score, backward simulation) | x0 + renoise | 30 min, 3101 it × 16, LR 1e-5 | 31.25 | 0.091 | 1.01 |
| `adapter_pdd.pt` | Parallel Decoding Distillation (N=32, L=8, midpoint) | 4 block-Euler steps | 5 min, 866 it × 16 | 32.88 | 0.093 | 0.42 |
Metrics: 6 eval prompts × 4 seeds, CLIP ViT-B/32. Trajectory methods (CM, PDD) keep the teacher's noise→image map (low MSE); distribution methods (LADD, DMD) do not.
Note on DMD: the same config with LR 5e-5 (3100 it × 16) reached CLIP 32.37. That adapter is not included here.
## Format
Each `.pt` file is a dict produced by the notebook's `save_adapter`: a peft LoRA state dict (`get_peft_model_state_dict`; r=32 for CM/LADD/PDD, r=128 for DMD; targets `to_q, to_k, to_v, to_out.0, conv_inverted, conv_point, proj_out.base`). `adapter_pdd.pt` also contains `grid.w` / `grid.b`: PDD's 32 output heads.
## Usage (in the workshop notebook)
```python
from huggingface_hub import hf_hub_download
import shutil
for m in ["cm", "ladd", "dmd", "pdd"]:
shutil.copy(hf_hub_download("leffff/diffusion-distillation-workshop-adapters", f"adapter_{m}.pt"), f"{CACHE_DIR}/adapter_{m}.pt")
load_adapter(m)
```
|