import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("lee31221/Qwen-Image-Edit-Outfit-2511-SFT", dtype=torch.bfloat16, device_map="cuda")
prompt = "Turn this cat into a dog"
input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png")
image = pipe(image=input_image, prompt=prompt).images[0]Qwen-Image-Edit-Outfit-2511-SFT
A full-parameter fine-tune of Qwen/Qwen-Image-Edit-2511 for garment / outfit virtual try-on (image editing): given a person photo and a garment photo, the model dresses the person in that outfit.
⚠️ Non-commercial, research use only. See LICENSE. This restriction is inherited from the training data (VITON-HD / IDM-VTON derivatives, CC BY-NC). Do not tag this model Apache-2.0 — the base is Apache-2.0, but the fine-tune carries the upstream NC constraint.
What changed from the base
Only the DiT (transformer) weights were updated via full-parameter SFT. All other components (text encoder, VAE, tokenizer, processor, scheduler) are unchanged from the base Qwen-Image-Edit-2511. So this directory loads exactly like the base model:
from diffusers import QwenImageEditPlusPipeline
import torch
pipe = QwenImageEditPlusPipeline.from_pretrained(
"lee31221/Qwen-Image-Edit-Outfit-2511-SFT", torch_dtype=torch.bfloat16
).to("cuda")
Training summary
| Base | Qwen/Qwen-Image-Edit-2511 (Apache-2.0) |
| Trainable | DiT only (20.43B params, BF16) |
| Optimizer | AdamW, lr 1e-5, weight_decay 0.01 |
| Schedule | 1 epoch, grad_accum 1, gradient checkpointing on |
| Data | ~11,415 outfit-edit pairs; prompt ≈ 1592 chars (full-text v2 template) |
| Inputs | image=person, edit_image=garment (condition) |
| Compute | 8× NVIDIA H100 80GB, DeepSpeed ZeRO-3, bf16 |
| Wall-clock | 2h22m for 1427 steps (06:41 → 09:03, +08:00) |
| Throughput | ~5.87 s/it, ~1.36 samples/s |
| Peak VRAM | 74.8 GB / GPU (91.7% of 80 GB), 8-GPU balance < 0.9% spread |
| Loss | 0.029 → 0.024 (window mean ↓ 23.7% over training) |
| Framework | DiffSynth-Studio + Accelerate + DeepSpeed (torch 2.6.0+cu124) |
Loss curve was logged to TensorBoard (1427 scalar points). See the run report in the source repository for a full breakdown (memory economics, per-GPU balance, loss noise analysis).
Training data & attribution
| Asset | Source | License |
|---|---|---|
| Base model (DiT, text encoder, VAE, tokenizer, processor, scheduler) | Qwen/Qwen-Image-Edit-2511 | Apache-2.0 |
| Synthetic outfit-edit training pairs | lee31221/Outfit_Qwen-Image-Edit-2511_in_Kling | derived; non-commercial |
| Underlying person/garment images | VITON-HD | CC BY-NC 4.0 |
| Teacher pipeline (derivative) | IDM-VTON | CC BY-NC-SA 4.0 |
| Training framework | DiffSynth-Studio | Apache-2.0 |
If you use this model, please attribute Qwen-Image-Edit-2511 and VITON-HD / IDM-VTON.
License
Non-Commercial Research Use only — see LICENSE and NOTICE.md. Commercial use is prohibited. This is stricter than the base model's Apache-2.0, because the fine-tune was trained on CC BY-NC derivatives.
Intended use & limitations
- Intended: academic research on image editing / virtual try-on.
- Not intended: any commercial product, service, or paid feature; generating non-consensual depictions of real people.
- Like all diffusion editors, outputs can be imperfect (garment detail drift, identity changes). Validate on a held-out test set before drawing conclusions.
- Downloads last month
- 62
Model tree for lee31221/Qwen-Image-Edit-Outfit-2511-SFT
Base model
Qwen/Qwen-Image-Edit-2511