BiliSakura/PixelDiT2-diffusers

Self-contained PixelDiT2 class-conditional checkpoints for Hugging Face diffusers. PixelDiT2 denoises in RGB pixel space (no VAE). Each variant ships its original DINOv3 checkpoint in encoder/ and the mapped copy inside the transformer weights, so inference does not download it. 256px uses DINOv3-S/16. 512px uses DINOv3-B/16.

This repository is PixelDiT2 only. The earlier PixelDiT family is BiliSakura/PixelDiT-diffusers.

Demo

Class: golden retriever (ImageNet 207). PixelDiT2-H/16 at 256×256, 50 Heun steps, guidance_scale=2.4, interval (0.125, 0.9), noise_scale=1.0, torch_dtype=bfloat16, seed 1, on cuda:0.

Same class at 512×512, 50 Heun steps, guidance_scale=2.1, interval (0.1, 0.925), noise_scale=2.0, torch_dtype=bfloat16, seed 1, on cuda:0.

Available checkpoints

Subfolder	Resolution	Source checkpoint	Encoder	Steps	CFG	CFG interval	Noise scale
PixelDiT2-H-16-256/	256×256	pixeldit2_h16_in256_ep600.ckpt	DINOv3-S/16	50	2.4	(0.125, 0.9)	1.0
PixelDiT2-H-16-512/	512×512	pixeldit2_h16_in512_ep680.ckpt	DINOv3-B/16	50	2.1	(0.1, 0.925)	2.0

Reported ImageNet FID is 1.46 at 256×256 and 1.48 at 512×512 (50k samples, Heun, 50 steps).

Repo layout

BiliSakura/PixelDiT2-diffusers/
├── README.md
├── .gitattributes
├── PixelDiT2-H-16-256/
│   ├── pipeline.py
│   ├── model_index.json
│   ├── demo.png
│   ├── encoder/
│   ├── scheduler/scheduling_pixeldit2.py
│   └── transformer/
└── PixelDiT2-H-16-512/
    ├── pipeline.py
    ├── model_index.json
    ├── demo.png
    ├── encoder/
    ├── scheduler/scheduling_pixeldit2.py
    └── transformer/

scheduler/scheduling_pixeldit2.py is PixelDiT2Scheduler. It integrates flow time from 0 to 1 with Heun, and uses Euler on the last interval. Each variant is self-contained.

Load from Hugging Face

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "BiliSakura/PixelDiT2-diffusers",
    subfolder="PixelDiT2-H-16-256",
    custom_pipeline="pipeline.py",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda:0")

generator = torch.Generator(device="cuda:0").manual_seed(1)
image = pipe(
    class_labels="golden retriever",
    num_inference_steps=50,
    guidance_scale=2.4,
    guidance_interval_min=0.125,
    guidance_interval_max=0.9,
    generator=generator,
).images[0]
image.save("demo.png")

Load from a local clone

from pathlib import Path
import torch
from diffusers import DiffusionPipeline

model_dir = Path("PixelDiT2-H-16-256").resolve()
pipe = DiffusionPipeline.from_pretrained(
    str(model_dir),
    local_files_only=True,
    custom_pipeline=str(model_dir / "pipeline.py"),
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda:0")

print(pipe.id2label[207])
print(pipe.get_label_ids("golden retriever"))

generator = torch.Generator(device="cuda:0").manual_seed(1)
image = pipe(class_labels="golden retriever", generator=generator).images[0]
image.save("demo.png")

Place the model on one GPU with .to("cuda:0"). Do not call enable_model_cpu_offload().

Point model_dir at PixelDiT2-H-16-512 for the 512px checkpoint. Omitting the sampling arguments uses that folder's config: 50 steps, guidance_scale=2.1, interval (0.1, 0.925), noise_scale=2.0.

height and width must be divisible by 16. Other sizes recompute rotary positions for that grid. One evaluation per step uses the same linspace: pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, solver="euler").

ImageNet class labels

id2label is in each variant's model_index.json.

• pipe.id2label — id to English label
• pipe.labels — synonym to id
• pipe.get_label_ids("golden retriever")
• pipe(class_labels="golden retriever") — strings resolve automatically

Citation

@inproceedings{yu2026pixeldit2,
      title={PixelDiT2: Representation-Grounded Pixel Diffusion Transformers},
      author={Yongsheng Yu and Wei Xiong and Yichen Sheng and Shiqiu Liu and Jiebo Luo},
      booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
      year={2026},
}

License

Pipeline code in this repository is Apache-2.0. Converted weights remain under the NVIDIA NSCLv1 non-commercial research license of nvidia/PixelDiT2-ImageNet.