README.md

---
license: other
license_name: nsclv1
license_link: https://huggingface.co/nvidia/PixelDiT2-ImageNet
library_name: diffusers
tags:
- diffusers
- pixeldit2
- image-generation
- class-conditional
- imagenet
- flow-matching
- pixel-space
inference: true
widget:
- text: golden retriever
  output:
    url: PixelDiT2-H-16-256/demo.png
- text: golden retriever
  output:
    url: PixelDiT2-H-16-512/demo.png
language:
- en
pipeline_tag: unconditional-image-generation
---

# BiliSakura/PixelDiT2-diffusers

Self-contained PixelDiT2 class-conditional checkpoints for Hugging Face diffusers. PixelDiT2 denoises in RGB pixel space (no VAE). Each variant ships its original DINOv3 checkpoint in `encoder/` and the mapped copy inside the transformer weights, so inference does not download it. 256px uses DINOv3-S/16. 512px uses DINOv3-B/16.

This repository is PixelDiT2 only. The earlier PixelDiT family is [BiliSakura/PixelDiT-diffusers](https://huggingface.co/BiliSakura/PixelDiT-diffusers).

## Demo

![PixelDiT2-H/16 256 demo](PixelDiT2-H-16-256/demo.png)

Class: *golden retriever* (ImageNet 207). PixelDiT2-H/16 at 256×256, 50 Heun steps, `guidance_scale=2.4`, interval `(0.125, 0.9)`, `noise_scale=1.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`.

![PixelDiT2-H/16 512 demo](PixelDiT2-H-16-512/demo.png)

Same class at 512×512, 50 Heun steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`.

## Available checkpoints

| Subfolder | Resolution | Source checkpoint | Encoder | Steps | CFG | CFG interval | Noise scale |
| --- | --- | --- | --- | ---: | ---: | --- | ---: |
| [`PixelDiT2-H-16-256/`](PixelDiT2-H-16-256/) | 256×256 | `pixeldit2_h16_in256_ep600.ckpt` | DINOv3-S/16 | 50 | 2.4 | (0.125, 0.9) | 1.0 |
| [`PixelDiT2-H-16-512/`](PixelDiT2-H-16-512/) | 512×512 | `pixeldit2_h16_in512_ep680.ckpt` | DINOv3-B/16 | 50 | 2.1 | (0.1, 0.925) | 2.0 |

Reported ImageNet FID is 1.46 at 256×256 and 1.48 at 512×512 (50k samples, Heun, 50 steps).

## Repo layout

```text
BiliSakura/PixelDiT2-diffusers/
├── README.md
├── .gitattributes
├── PixelDiT2-H-16-256/
│   ├── pipeline.py
│   ├── model_index.json
│   ├── demo.png
│   ├── encoder/
│   ├── scheduler/scheduling_pixeldit2.py
│   └── transformer/
└── PixelDiT2-H-16-512/
    ├── pipeline.py
    ├── model_index.json
    ├── demo.png
    ├── encoder/
    ├── scheduler/scheduling_pixeldit2.py
    └── transformer/
```

`scheduler/scheduling_pixeldit2.py` is `PixelDiT2Scheduler`. It integrates flow time from 0 to 1 with Heun, and uses Euler on the last interval. Each variant is self-contained.

## Load from Hugging Face

```python
import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "BiliSakura/PixelDiT2-diffusers",
    subfolder="PixelDiT2-H-16-256",
    custom_pipeline="pipeline.py",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda:0")

generator = torch.Generator(device="cuda:0").manual_seed(1)
image = pipe(
    class_labels="golden retriever",
    num_inference_steps=50,
    guidance_scale=2.4,
    guidance_interval_min=0.125,
    guidance_interval_max=0.9,
    generator=generator,
).images[0]
image.save("demo.png")
```

## Load from a local clone

```python
from pathlib import Path
import torch
from diffusers import DiffusionPipeline

model_dir = Path("PixelDiT2-H-16-256").resolve()
pipe = DiffusionPipeline.from_pretrained(
    str(model_dir),
    local_files_only=True,
    custom_pipeline=str(model_dir / "pipeline.py"),
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda:0")

print(pipe.id2label[207])
print(pipe.get_label_ids("golden retriever"))

generator = torch.Generator(device="cuda:0").manual_seed(1)
image = pipe(class_labels="golden retriever", generator=generator).images[0]
image.save("demo.png")
```

Place the model on one GPU with `.to("cuda:0")`. Do not call `enable_model_cpu_offload()`.

Point `model_dir` at `PixelDiT2-H-16-512` for the 512px checkpoint. Omitting the sampling arguments uses that folder's config: 50 steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`.

`height` and `width` must be divisible by 16. Other sizes recompute rotary positions for that grid. One evaluation per step uses the same linspace: `pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, solver="euler")`.

## ImageNet class labels

`id2label` is in each variant's `model_index.json`.

- `pipe.id2label` — id to English label
- `pipe.labels` — synonym to id
- `pipe.get_label_ids("golden retriever")`
- `pipe(class_labels="golden retriever")` — strings resolve automatically

## Citation

```bibtex
@inproceedings{yu2026pixeldit2,
      title={PixelDiT2: Representation-Grounded Pixel Diffusion Transformers},
      author={Yongsheng Yu and Wei Xiong and Yichen Sheng and Shiqiu Liu and Jiebo Luo},
      booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
      year={2026},
}
```

## License

Pipeline code in this repository is Apache-2.0. Converted weights remain under the NVIDIA NSCLv1 non-commercial research license of [nvidia/PixelDiT2-ImageNet](https://huggingface.co/nvidia/PixelDiT2-ImageNet).