| --- |
| license: apache-2.0 |
| base_model: Efficient-Large-Model/Sana_600M_512px_diffusers |
| library_name: peft |
| tags: |
| - diffusion |
| - distillation |
| - sana |
| - lora |
| - text-to-image |
| --- |
| |
| # Diffusion Distillation Workshop: 4-step Sana-600M students |
|
|
| LoRA adapters that distill [Sana-600M-512px](https://huggingface.co/Efficient-Large-Model/Sana_600M_512px_diffusers) from 40 NFE (20 steps with CFG 4.5) into **4-NFE students** using four methods. They were trained with the workshop notebook on one H100. Data: [`leffff/diffusion-distillation-workshop`](https://huggingface.co/datasets/leffff/diffusion-distillation-workshop). |
|
|
| | file | method | sampler (4 NFE) | training | CLIP ↑ | diversity | latent MSE to teacher | |
| |---|---|---|---|---|---|---| |
| | teacher | — | 20 Euler steps + CFG (40 NFE) | — | 32.29 | 0.075 | 0 | |
| | teacher | — | 4 Euler steps + CFG (8 NFE) | — | 30.73 | 0.117 | 0.54 | |
| | `adapter_cm.pt` | multistep (phased) consistency distillation | Euler | 5 min, 933 it × 16 | 32.67 | 0.088 | 0.41 | |
| | `adapter_ladd.pt` | LADD (adversarial, teacher-feature discriminator) | x0 + renoise | 5 min, 853 it × 16 | 31.95 | 0.101 | 1.42 | |
| | `adapter_dmd.pt` | DMD2 (reverse KL, fake score, backward simulation) | x0 + renoise | 30 min, 3101 it × 16, LR 1e-5 | 31.25 | 0.091 | 1.01 | |
| | `adapter_pdd.pt` | Parallel Decoding Distillation (N=32, L=8, midpoint) | 4 block-Euler steps | 5 min, 866 it × 16 | 32.88 | 0.093 | 0.42 | |
|
|
| Metrics: 6 eval prompts × 4 seeds, CLIP ViT-B/32. Trajectory methods (CM, PDD) keep the teacher's noise→image map (low MSE); distribution methods (LADD, DMD) do not. |
|
|
| Note on DMD: the same config with LR 5e-5 (3100 it × 16) reached CLIP 32.37. That adapter is not included here. |
|
|
|  |
|
|
| ## Format |
|
|
| Each `.pt` file is a dict produced by the notebook's `save_adapter`: a peft LoRA state dict (`get_peft_model_state_dict`; r=32 for CM/LADD/PDD, r=128 for DMD; targets `to_q, to_k, to_v, to_out.0, conv_inverted, conv_point, proj_out.base`). `adapter_pdd.pt` also contains `grid.w` / `grid.b`: PDD's 32 output heads. |
|
|
| ## Usage (in the workshop notebook) |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| import shutil |
| for m in ["cm", "ladd", "dmd", "pdd"]: |
| shutil.copy(hf_hub_download("leffff/diffusion-distillation-workshop-adapters", f"adapter_{m}.pt"), f"{CACHE_DIR}/adapter_{m}.pt") |
| load_adapter(m) |
| ``` |
|
|
