| --- |
| license: other |
| license_name: nsclv1 |
| license_link: https://huggingface.co/nvidia/PixelDiT2-ImageNet |
| library_name: diffusers |
| tags: |
| - diffusers |
| - pixeldit2 |
| - image-generation |
| - class-conditional |
| - imagenet |
| - flow-matching |
| - pixel-space |
| inference: true |
| widget: |
| - text: golden retriever |
| output: |
| url: PixelDiT2-H-16-256/demo.png |
| - text: golden retriever |
| output: |
| url: PixelDiT2-H-16-512/demo.png |
| language: |
| - en |
| pipeline_tag: unconditional-image-generation |
| --- |
| |
| # BiliSakura/PixelDiT2-diffusers |
|
|
| Self-contained PixelDiT2 class-conditional checkpoints for Hugging Face diffusers. PixelDiT2 denoises in RGB pixel space (no VAE). Each variant ships its original DINOv3 checkpoint in `encoder/` and the mapped copy inside the transformer weights, so inference does not download it. 256px uses DINOv3-S/16. 512px uses DINOv3-B/16. |
|
|
| This repository is PixelDiT2 only. The earlier PixelDiT family is [BiliSakura/PixelDiT-diffusers](https://huggingface.co/BiliSakura/PixelDiT-diffusers). |
|
|
| ## Demo |
|
|
|  |
|
|
| Class: *golden retriever* (ImageNet 207). PixelDiT2-H/16 at 256×256, 50 Heun steps, `guidance_scale=2.4`, interval `(0.125, 0.9)`, `noise_scale=1.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`. |
|
|
|  |
|
|
| Same class at 512×512, 50 Heun steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`. |
|
|
| ## Available checkpoints |
|
|
| | Subfolder | Resolution | Source checkpoint | Encoder | Steps | CFG | CFG interval | Noise scale | |
| | --- | --- | --- | --- | ---: | ---: | --- | ---: | |
| | [`PixelDiT2-H-16-256/`](PixelDiT2-H-16-256/) | 256×256 | `pixeldit2_h16_in256_ep600.ckpt` | DINOv3-S/16 | 50 | 2.4 | (0.125, 0.9) | 1.0 | |
| | [`PixelDiT2-H-16-512/`](PixelDiT2-H-16-512/) | 512×512 | `pixeldit2_h16_in512_ep680.ckpt` | DINOv3-B/16 | 50 | 2.1 | (0.1, 0.925) | 2.0 | |
|
|
| Reported ImageNet FID is 1.46 at 256×256 and 1.48 at 512×512 (50k samples, Heun, 50 steps). |
|
|
| ## Repo layout |
|
|
| ```text |
| BiliSakura/PixelDiT2-diffusers/ |
| ├── README.md |
| ├── .gitattributes |
| ├── PixelDiT2-H-16-256/ |
| │ ├── pipeline.py |
| │ ├── model_index.json |
| │ ├── demo.png |
| │ ├── encoder/ |
| │ ├── scheduler/scheduling_pixeldit2.py |
| │ └── transformer/ |
| └── PixelDiT2-H-16-512/ |
| ├── pipeline.py |
| ├── model_index.json |
| ├── demo.png |
| ├── encoder/ |
| ├── scheduler/scheduling_pixeldit2.py |
| └── transformer/ |
| ``` |
|
|
| `scheduler/scheduling_pixeldit2.py` is `PixelDiT2Scheduler`. It integrates flow time from 0 to 1 with Heun, and uses Euler on the last interval. Each variant is self-contained. |
|
|
| ## Load from Hugging Face |
|
|
| ```python |
| import torch |
| from diffusers import DiffusionPipeline |
| |
| pipe = DiffusionPipeline.from_pretrained( |
| "BiliSakura/PixelDiT2-diffusers", |
| subfolder="PixelDiT2-H-16-256", |
| custom_pipeline="pipeline.py", |
| trust_remote_code=True, |
| torch_dtype=torch.bfloat16, |
| ).to("cuda:0") |
| |
| generator = torch.Generator(device="cuda:0").manual_seed(1) |
| image = pipe( |
| class_labels="golden retriever", |
| num_inference_steps=50, |
| guidance_scale=2.4, |
| guidance_interval_min=0.125, |
| guidance_interval_max=0.9, |
| generator=generator, |
| ).images[0] |
| image.save("demo.png") |
| ``` |
|
|
| ## Load from a local clone |
|
|
| ```python |
| from pathlib import Path |
| import torch |
| from diffusers import DiffusionPipeline |
| |
| model_dir = Path("PixelDiT2-H-16-256").resolve() |
| pipe = DiffusionPipeline.from_pretrained( |
| str(model_dir), |
| local_files_only=True, |
| custom_pipeline=str(model_dir / "pipeline.py"), |
| trust_remote_code=True, |
| torch_dtype=torch.bfloat16, |
| ).to("cuda:0") |
| |
| print(pipe.id2label[207]) |
| print(pipe.get_label_ids("golden retriever")) |
| |
| generator = torch.Generator(device="cuda:0").manual_seed(1) |
| image = pipe(class_labels="golden retriever", generator=generator).images[0] |
| image.save("demo.png") |
| ``` |
|
|
| Place the model on one GPU with `.to("cuda:0")`. Do not call `enable_model_cpu_offload()`. |
|
|
| Point `model_dir` at `PixelDiT2-H-16-512` for the 512px checkpoint. Omitting the sampling arguments uses that folder's config: 50 steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`. |
|
|
| `height` and `width` must be divisible by 16. Other sizes recompute rotary positions for that grid. One evaluation per step uses the same linspace: `pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, solver="euler")`. |
|
|
| ## ImageNet class labels |
|
|
| `id2label` is in each variant's `model_index.json`. |
|
|
| - `pipe.id2label` — id to English label |
| - `pipe.labels` — synonym to id |
| - `pipe.get_label_ids("golden retriever")` |
| - `pipe(class_labels="golden retriever")` — strings resolve automatically |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{yu2026pixeldit2, |
| title={PixelDiT2: Representation-Grounded Pixel Diffusion Transformers}, |
| author={Yongsheng Yu and Wei Xiong and Yichen Sheng and Shiqiu Liu and Jiebo Luo}, |
| booktitle={Conference on Neural Information Processing Systems (NeurIPS)}, |
| year={2026}, |
| } |
| ``` |
|
|
| ## License |
|
|
| Pipeline code in this repository is Apache-2.0. Converted weights remain under the NVIDIA NSCLv1 non-commercial research license of [nvidia/PixelDiT2-ImageNet](https://huggingface.co/nvidia/PixelDiT2-ImageNet). |
