Upload PixelDiT2-diffusers checkpoints

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+PixelDiT2-H-16-256/demo.png filter=lfs diff=lfs merge=lfs -text
+PixelDiT2-H-16-512/demo.png filter=lfs diff=lfs merge=lfs -text

PixelDiT2-H-16-256/encoder/LICENSE.md

@@ -0,0 +1,66 @@
+# DINOv3 License
+
+*Last Updated: August 19, 2025*
+
+**“Agreement”** means the terms and conditions for use, reproduction, distribution and modification of the DINO Materials set forth herein.
+
+**“DINO Materials”** means, collectively, Documentation and the models, software and algorithms, including machine-learning model code, trained model weights, inference-enabling code, training-enabling code, fine-tuning enabling code, and other elements of the foregoing distributed by Meta and made available under this Agreement.
+
+**“Documentation”** means the specifications, manuals and documentation accompanying
+DINO Materials distributed by Meta.
+
+**“Licensee”** or **“you”** means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
+
+**“Meta”** or **“we”** means Meta Platforms Ireland Limited (if you are located in or, if you are an entity, your principal place of business is in the EEA or Switzerland) or Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland).
+
+**“Sanctions”** means any economic or trade sanctions or restrictions administered or enforced by the United States (including the Office of Foreign Assets Control of the U.S. Department of the Treasury (“OFAC”), the U.S. Department of State and the U.S. Department of Commerce), the United Nations, the European Union, or the United Kingdom.
+
+**“Trade Controls”** means any of the following: Sanctions and applicable export and import controls.
+
+By clicking “I Accept” below or by using or distributing any portion or element of the DINO Materials, you agree to be bound by this Agreement.
+
+## 1. License Rights and Redistribution.
+
+a. <ins>Grant of Rights</ins>. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Meta’s intellectual property or other rights owned by Meta embodied in the DINO Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the DINO Materials.
+
+b. <ins>Redistribution and Use</ins>.
+
+i. Distribution of DINO Materials, and any derivative works thereof, are subject to the terms of this Agreement. If you distribute or make the DINO Materials, or any derivative works thereof, available to a third party, you may only do so under the terms of this Agreement and you shall provide a copy of this Agreement with any such DINO Materials.
+
+ii.  If you submit for publication the results of research you perform on, using, or otherwise in connection with DINO Materials, you must acknowledge the use of DINO Materials in your publication.
+
+iii. Your use of the DINO Materials must comply with applicable laws and regulations, including Trade Control Laws and applicable privacy and data protection laws.
+
+iv. Your use of the DINO Materials will not involve or encourage others to reverse engineer, decompile or discover the underlying components of the DINO Materials.
+
+v. You are not the target of Trade Controls and your use of DINO Materials must comply with Trade Controls. You agree not to use, or permit others to use, DINO Materials for any activities subject to the International Traffic in Arms Regulations (ITAR) or end uses prohibited by Trade Controls, including those related to military or warfare purposes, nuclear industries or applications, espionage, or the development or use of guns or illegal weapons.
+
+## 2. User Support.
+
+Your use of the DINO Materials is done at your own discretion; Meta does not process any information nor provide any service in relation to such use.  Meta is under no obligation to provide any support services for the DINO Materials. Any support provided is “as is”, “with all faults”, and without warranty of any kind.
+
+## 3. Disclaimer of Warranty.
+
+UNLESS REQUIRED BY APPLICABLE LAW, THE DINO MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE DINO MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE DINO MATERIALS AND ANY OUTPUT AND RESULTS.
+
+## 4. Limitation of Liability.
+
+IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+## 5. Intellectual Property.
+
+a. Subject to Meta’s ownership of DINO Materials and derivatives made by or for Meta, with respect to any derivative works and modifications of the DINO Materials that are made by you, as between you and Meta, you are and will be the owner of such derivative works and modifications.
+
+b. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the DINO Materials, outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third party arising out of or related to your use or distribution of the DINO Materials.
+
+## 6. Term and Termination.
+
+The term of this Agreement will commence upon your acceptance of this Agreement or access to the DINO Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the DINO Materials. Sections 3, 4 and 7 shall survive the termination of this Agreement.
+
+## 7. Governing Law and Jurisdiction.
+
+This Agreement will be governed and construed under the laws of the State of California without regard to choice of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of any dispute arising out of this Agreement.
+
+## 8. Modifications and Amendments.
+
+Meta may modify this Agreement from time to time; provided that they are similar in spirit to the current version of the Agreement, but may differ in detail to address new problems or concerns. All such changes will be effective immediately. Your continued use of the DINO Materials after any modification to this Agreement constitutes your agreement to such modification. Except as provided in this Agreement, no modification or addition to any provision of this Agreement will be binding unless it is in writing and signed by an authorized representative of both you and Meta.

PixelDiT2-H-16-256/encoder/README.md

@@ -0,0 +1,477 @@
+---
+extra_gated_fields:
+  First Name: text
+  Last Name: text
+  Date of birth: date_picker
+  Country: country
+  Affiliation: text
+  Job title:
+    type: select
+    options:
+      - Student
+      - Research Graduate
+      - AI researcher
+      - AI developer/engineer
+      - Reporter
+      - Other
+  geo: ip_location
+  By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the Meta Privacy Policy: checkbox
+extra_gated_description: >-
+  The information you provide will be collected, stored, processed and shared in
+  accordance with the [Meta Privacy
+  Policy](https://www.facebook.com/privacy/policy/).
+extra_gated_button_content: Submit
+language:
+- en
+tags:
+- dino
+- dinov3
+- arxiv:2508.10104
+license: other
+license_name: dinov3-license
+license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license
+base_model: facebook/dinov3-vit7b16-pretrain-lvd1689m
+pipeline_tag: image-feature-extraction
+library_name: transformers
+---
+
+# Model Card for DINOv3
+
+DINOv3 is a family of versatile vision foundation models that outperforms the specialized state of the art across a broad range of settings, without fine-tuning. DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks, significantly surpassing previous self- and weakly-supervised foundation models.
+
+## Model Details
+
+These are Vision Transformer and ConvNeXt models trained following the method described in the DINOv3 paper. 12 models are provided:
+
+- 10 models pretrained on web data (LVD-1689M dataset)
+  - 1 ViT-7B trained from scratch,
+  - 5 ViT-S/S+/B/L/H+ models distilled from the ViT-7B,
+  - 4 ConvNeXt-{T/S/B/L} models distilled from the ViT-7B,
+- 2 models pretrained on satellite data (SAT-493M dataset)
+  - 1 ViT-7B trained from scratch
+  - 1 ViT-L distilled from the ViT-7B
+
+
+Each Transformer-based model takes an image as input and returns a class token, patch tokens (and register tokens). These models follow a ViT architecture, with a patch size of 16. For a 224x224 image, this results in 1 class token + 4 register tokens + 196 patch tokens = 201 tokens (for DINOv2 with registers this resulted in 1 + 4 + 256 = 261 tokens).
+
+The models can accept larger images provided the image shapes are multiples of the patch size (16). If this condition is not verified, the model will crop to the closest smaller multiple of the patch size.
+
+### Model Description
+
+- **Developed by:** Meta AI
+- **Model type:** Vision Transformer, ConvNeXt
+- **License:** [DINOv3 License](https://ai.meta.com/resources/models-and-libraries/dinov3-license/)
+
+### Model Sources
+
+- **Repository:** [https://github.com/facebookresearch/dinov3](https://github.com/facebookresearch/dinov3)
+- **Paper:** [https://arxiv.org/abs/2508.10104](https://arxiv.org/abs/2508.10104)
+
+## Uses
+
+The models are vision backbones providing multi-purpose features for downstream tasks.
+
+### Direct Use
+
+The models can be used without fine-tuning, with downstream classifiers as simple as linear layers, to obtain competitive results:
+
+- on image classification, using k-NN classifiers on the class token
+- on image classification, with logistic regression classifiers applied on the class token
+- on image classification, with a linear layer applied on the class token and the average of the patch tokens
+- on image retrieval using nearest neighbors
+- on geometric and semantic 3D keypoint correspondances
+- on depth estimation, semantic segmentation, using linear layers
+- on unsupervised object discovery
+- on video segmentation tracking
+- on video classification, using a small 4-layer attentive probe
+
+### Downstream Use
+
+While fine-tuning the models can yield some gains, it is recommended to keep this option as a last resort: the frozen features are expected to provide good performance out-of-the-box.
+
+## Bias, Risks, and Limitations
+
+Compared to DINOv2 and SEERv2, DINOv3 delivers somewhat consistent performance across income categories on geographical fairness and diversity, although with a notable performance drop in the low-income bucket compared to the highest-income bucket.
+
+DINOv3 also achieves relatively good scores across different regions, improving over its predecessor DINOv2. However, a relative difference is still observed between Europe and Africa.
+
+### Recommendations
+
+Fine-tuning is expected to increase the biases in the features produced by the model as they will be tuned to the fine-tuning labels.
+
+## How to Get Started with the Model
+
+The example below demonstrates how to obtain an image embedding with [Pipeline] or the [AutoModel] class.
+
+```python
+from transformers import pipeline
+from transformers.image_utils import load_image
+
+url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
+image = load_image(url)
+
+feature_extractor = pipeline(
+    model="facebook/dinov3-vits16-pretrain-lvd1689m",
+    task="image-feature-extraction",
+)
+features = feature_extractor(image)
+```
+
+```python
+import torch
+from transformers import AutoImageProcessor, AutoModel
+from transformers.image_utils import load_image
+
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = load_image(url)
+
+pretrained_model_name = "facebook/dinov3-vits16-pretrain-lvd1689m"
+processor = AutoImageProcessor.from_pretrained(pretrained_model_name)
+model = AutoModel.from_pretrained(
+    pretrained_model_name, 
+    device_map="auto", 
+)
+
+inputs = processor(images=image, return_tensors="pt").to(model.device)
+with torch.inference_mode():
+    outputs = model(**inputs)
+
+pooled_output = outputs.pooler_output
+print("Pooled output shape:", pooled_output.shape)
+```
+
+## Training Details
+
+### Training Data
+
+- Web dataset (LVD-1689M): a curated dataset of 1,689 millions of images extracted from a large data
+pool of 17 billions web images collected from public posts on Instagram
+
+- Satellite dataset (SAT-493M): a dataset of 493 millions of 512x512 images sampled randomly from Maxar RGB ortho-rectified imagery at 0.6 meter resolution
+
+### Training Procedure
+
+**Training objective:**
+
+- DINO self-distillation loss with multi-crop
+- iBOT masked-image modeling loss
+- KoLeo regularization on [CLS] tokens
+- Gram anchoring
+
+- **Training regime:** PyTorch FSDP2 (with bf16 and fp8 matrix multiplications)
+
+**Distillation:**
+
+- Distillation follows the standard DINOv3 pretraining procedure, except the teacher is a frozen pretrained ViT-7B.
+
+## Evaluation
+
+**Results**
+
+The reader is referred to the associated paper for details on the evaluation protocols
+
+*Results for ViT backbones pretrained (or distilled) on web (LVD-1689M)*
+
+<table>
+  <tr>
+    <th></th>
+    <!-- <th></th> -->
+    <th colspan="4">Global Tasks</th>
+    <th colspan="5">Dense Tasks</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <!-- <th>Dataset</th> -->
+    <th>IN-ReaL</th>
+    <th>IN-R</th>
+    <th>Obj.Net</th>
+    <th>Ox.-H</th>
+    <th>ADE20k</th>
+    <th>NYU↓</th>
+    <th>DAVIS</th>
+    <th>NAVI</th>
+    <th>SPair</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-S/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">87.0</td>
+    <td align="right">60.4</td>
+    <td align="right">50.9</td>
+    <td align="right">49.5</td>
+    <td align="right">47.0</td>
+    <td align="right">0.403</td>
+    <td align="right">72.7</td>
+    <td align="right">56.3</td>
+    <td align="right">50.4</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-S+/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">88.0</td>
+    <td align="right">68.8</td>
+    <td align="right">54.6</td>
+    <td align="right">50.0</td>
+    <td align="right">48.8</td>
+    <td align="right">0.399</td>
+    <td align="right">75.5</td>
+    <td align="right">57.1</td>
+    <td align="right">55.2</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-B/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">89.3</td>
+    <td align="right">76.7</td>
+    <td align="right">64.1</td>
+    <td align="right">58.5</td>
+    <td align="right">51.8</td>
+    <td align="right">0.373</td>
+    <td align="right">77.2</td>
+    <td align="right">58.8</td>
+    <td align="right">57.2</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.2</td>
+    <td align="right">88.1</td>
+    <td align="right">74.8</td>
+    <td align="right">63.1</td>
+    <td align="right">54.9</td>
+    <td align="right">0.352</td>
+    <td align="right">79.9</td>
+    <td align="right">62.3</td>
+    <td align="right">61.3</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-H+/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.3</td>
+    <td align="right">90.0</td>
+    <td align="right">78.6</td>
+    <td align="right">64.5</td>
+    <td align="right">54.8</td>
+    <td align="right">0.352</td>
+    <td align="right">79.3</td>
+    <td align="right">63.3</td>
+    <td align="right">56.3</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.4</td>
+    <td align="right">91.1</td>
+    <td align="right">91.1</td>
+    <td align="right">72.8</td>
+    <td align="right">55.9</td>
+    <td align="right">0.309</td>
+    <td align="right">79.7</td>
+    <td align="right">64.4</td>
+    <td align="right">58.7</td>
+  </tr>
+</table>
+
+*Results for ConvNeXt backbones distilled on web (LVD-1689M)*
+
+<table>
+  <tr>
+    <th></th>
+    <th colspan="6">Global Tasks</th>
+    <th colspan="2">Dense Tasks</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <th colspan="2">IN-ReaL</th>
+    <th colspan="2">IN-R</th>
+    <th colspan="2">Obj.Net</th>
+    <th>ADE20k</th>
+    <th>NYU↓</th>
+  </tr>
+  <tr>
+    <td></th>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td colspan="2"></td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Tiny</td>
+    <td align="right">86.6</td>
+    <td align="right">87.7</td>
+    <td align="right">73.7</td>
+    <td align="right">74.1</td>
+    <td align="right">52.6</td>
+    <td align="right">58.7</td>
+    <td align="right">42.7</td>
+    <td align="right">0.448</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Small</td>
+    <td align="right">87.9</td>
+    <td align="right">88.7</td>
+    <td align="right">73.7</td>
+    <td align="right">74.1</td>
+    <td align="right">52.6</td>
+    <td align="right">58.7</td>
+    <td align="right">44.8</td>
+    <td align="right">0.432</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Base</td>
+    <td align="right">88.5</td>
+    <td align="right">89.2</td>
+    <td align="right">77.2</td>
+    <td align="right">78.2</td>
+    <td align="right">56.2</td>
+    <td align="right">61.3</td>
+    <td align="right">46.3</td>
+    <td align="right">0.420</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Large</td>
+    <td align="right">88.9</td>
+    <td align="right">89.4</td>
+    <td align="right">81.3</td>
+    <td align="right">82.4</td>
+    <td align="right">59.3</td>
+    <td align="right">65.2</td>
+    <td align="right">47.8</td>
+    <td align="right">0.403</td>
+  </tr>
+</table>
+
+*Results for ViT backbones pretrained (or distilled) on satellite (SAT-493M)*
+
+<table>
+  <tr>
+    <th></th>
+    <th colspan="7">(GEO-Bench) Classification</th>
+  </tr>
+  <tr>
+    <th>Model</ht>
+    <th>m-BEnet</th>
+    <th>m-brick-kiln
+    <th>m-eurosat</th>
+    <th>m-forestnet</th>
+    <th>m-pv4ger</th>
+    <th>m-so2sat</th>
+    <th>mean</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <td>73.0</td>
+    <td>96.5</td>
+    <td>94.1</td>
+    <td>60.6</td>
+    <td>96.0</td>
+    <td>57.4</td>
+    <td>79.6</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <td>74.0</td>
+    <td>97.2</td>
+    <td>94.8</td>
+    <td>62.3</td>
+    <td>96.1</td>
+    <td>62.1</td>
+    <td>81.1</td>
+  </tr>
+  <tr>
+    <th></th>
+    <th colspan="7">(GEO-Bench) Segmentation</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <th>m-cashew</th>
+    <th>m-chesapeake</th>
+    <th>m-NeonTree</th>
+    <th>m-nz-cattle</th>
+    <th>m-pv4ger-seg</th>
+    <th>m-SA-crop</th>
+    <th>mean</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <td>94.2</td>
+    <td>75.6</td>
+    <td>61.8</td>
+    <td>83.7</td>
+    <td>95.2</td>
+    <td>36.8</td>
+    <td>74.5</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <td>94.1</td>
+    <td>76.6</td>
+    <td>62.6</td>
+    <td>83.4</td>
+    <td>95.5</td>
+    <td>37.6</td>
+    <td>75.0</td>
+  </tr>
+</table>
+
+
+## Environmental Impact
+
+- **Hardware Type:** Nvidia H100
+- **Hours used:** 61,440 hours for ViT-7B model training
+- **Cloud Provider:** Private infrastructure
+- **Compute Region:** USA
+- **Carbon Emitted:** 18t CO2eq
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+Vision Transformer models:
+
+- ViT-S (21M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, MLP FFN, RoPE
+- ViT-S+ (29M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, SwiGLU FFN, RoPE
+- ViT-B (86M parameters): patch size 16, embedding dimension 768, 4 register tokens, 12 heads, MLP FFN, RoPE
+- ViT-L (300M parameters): patch size 16, embedding dimension 1024, 4 register tokens, 16 heads, MLP FFN, RoPE
+- ViT-H+ (840M parameters): patch size 16, embedding dimension 1280, 4 register tokens, 20 heads, SwiGLU FFN, RoPE
+- ViT-7B (6716M parameters): patch size 16, embedding dimension 4096, 4 register tokens, 32 heads, SwiGLU FFN, RoPE
+
+ConvNeXt models:
+
+- ConvNeXt Tiny (29M parameters)
+- ConvNeXt Small (50M parameters)
+- ConvNeXt Base (89M parameters)
+- ConvNeXt Large (198M parameters)
+
+### Compute Infrastructure
+
+#### Hardware
+
+Nvidia H100 GPUs
+
+#### Software
+
+PyTorch 2.7
+
+## More Information
+
+See the [blog post](https://ai.meta.com/blog/dinov3-self-supervised-vision-model/) and the associated [website](https://ai.meta.com/dinov3/).
+
+## Citation
+
+**BibTeX**
+
+```
+@misc{simeoni2025dinov3,
+  title={{DINOv3}},
+  author={Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
+  year={2025},
+  eprint={2508.10104},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2508.10104},
+}
+```

PixelDiT2-H-16-256/encoder/config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "DINOv3ViTModel"
+  ],
+  "attention_dropout": 0.0,
+  "drop_path_rate": 0.0,
+  "hidden_act": "gelu",
+  "hidden_size": 384,
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "key_bias": false,
+  "layer_norm_eps": 1e-05,
+  "layerscale_value": 1.0,
+  "mlp_bias": true,
+  "model_type": "dinov3_vit",
+  "num_attention_heads": 6,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "num_register_tokens": 4,
+  "patch_size": 16,
+  "pos_embed_jitter": null,
+  "pos_embed_rescale": 2.0,
+  "pos_embed_shift": null,
+  "proj_bias": true,
+  "query_bias": true,
+  "rope_theta": 100.0,
+  "torch_dtype": "float32",
+  "transformers_version": "4.56.0.dev0",
+  "use_gated_mlp": false,
+  "value_bias": true
+}

PixelDiT2-H-16-256/encoder/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4610ad75edef83e75afdebf162d148dc628045ea6cbb83d67d4708c709c4f91d
+size 86406384

PixelDiT2-H-16-256/encoder/preprocessor_config.json

@@ -0,0 +1,31 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "DINOv3ViTImageProcessorFast",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "input_data_format": null,
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

PixelDiT2-H-16-256/model_index.json

@@ -0,0 +1,1021 @@
+{
+  "_class_name": [
+    "pipeline",
+    "PixelDiT2Pipeline"
+  ],
+  "_diffusers_version": "0.39.0",
+  "encoder": [
+    "transformers",
+    "DINOv3ViTModel"
+  ],
+  "scheduler": [
+    "scheduling_pixeldit2",
+    "PixelDiT2Scheduler"
+  ],
+  "transformer": [
+    "transformer_pixeldit2",
+    "PixelDiT2Transformer2DModel"
+  ],
+  "id2label": {
+    "0": "tench, Tinca tinca",
+    "1": "goldfish, Carassius auratus",
+    "2": "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias",
+    "3": "tiger shark, Galeocerdo cuvieri",
+    "4": "hammerhead, hammerhead shark",
+    "5": "electric ray, crampfish, numbfish, torpedo",
+    "6": "stingray",
+    "7": "cock",
+    "8": "hen",
+    "9": "ostrich, Struthio camelus",
+    "10": "brambling, Fringilla montifringilla",
+    "11": "goldfinch, Carduelis carduelis",
+    "12": "house finch, linnet, Carpodacus mexicanus",
+    "13": "junco, snowbird",
+    "14": "indigo bunting, indigo finch, indigo bird, Passerina cyanea",
+    "15": "robin, American robin, Turdus migratorius",
+    "16": "bulbul",
+    "17": "jay",
+    "18": "magpie",
+    "19": "chickadee",
+    "20": "water ouzel, dipper",
+    "21": "kite",
+    "22": "bald eagle, American eagle, Haliaeetus leucocephalus",
+    "23": "vulture",
+    "24": "great grey owl, great gray owl, Strix nebulosa",
+    "25": "European fire salamander, Salamandra salamandra",
+    "26": "common newt, Triturus vulgaris",
+    "27": "eft",
+    "28": "spotted salamander, Ambystoma maculatum",
+    "29": "axolotl, mud puppy, Ambystoma mexicanum",
+    "30": "bullfrog, Rana catesbeiana",
+    "31": "tree frog, tree-frog",
+    "32": "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui",
+    "33": "loggerhead, loggerhead turtle, Caretta caretta",
+    "34": "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea",
+    "35": "mud turtle",
+    "36": "terrapin",
+    "37": "box turtle, box tortoise",
+    "38": "banded gecko",
+    "39": "common iguana, iguana, Iguana iguana",
+    "40": "American chameleon, anole, Anolis carolinensis",
+    "41": "whiptail, whiptail lizard",
+    "42": "agama",
+    "43": "frilled lizard, Chlamydosaurus kingi",
+    "44": "alligator lizard",
+    "45": "Gila monster, Heloderma suspectum",
+    "46": "green lizard, Lacerta viridis",
+    "47": "African chameleon, Chamaeleo chamaeleon",
+    "48": "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis",
+    "49": "African crocodile, Nile crocodile, Crocodylus niloticus",
+    "50": "American alligator, Alligator mississipiensis",
+    "51": "triceratops",
+    "52": "thunder snake, worm snake, Carphophis amoenus",
+    "53": "ringneck snake, ring-necked snake, ring snake",
+    "54": "hognose snake, puff adder, sand viper",
+    "55": "green snake, grass snake",
+    "56": "king snake, kingsnake",
+    "57": "garter snake, grass snake",
+    "58": "water snake",
+    "59": "vine snake",
+    "60": "night snake, Hypsiglena torquata",
+    "61": "boa constrictor, Constrictor constrictor",
+    "62": "rock python, rock snake, Python sebae",
+    "63": "Indian cobra, Naja naja",
+    "64": "green mamba",
+    "65": "sea snake",
+    "66": "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus",
+    "67": "diamondback, diamondback rattlesnake, Crotalus adamanteus",
+    "68": "sidewinder, horned rattlesnake, Crotalus cerastes",
+    "69": "trilobite",
+    "70": "harvestman, daddy longlegs, Phalangium opilio",
+    "71": "scorpion",
+    "72": "black and gold garden spider, Argiope aurantia",
+    "73": "barn spider, Araneus cavaticus",
+    "74": "garden spider, Aranea diademata",
+    "75": "black widow, Latrodectus mactans",
+    "76": "tarantula",
+    "77": "wolf spider, hunting spider",
+    "78": "tick",
+    "79": "centipede",
+    "80": "black grouse",
+    "81": "ptarmigan",
+    "82": "ruffed grouse, partridge, Bonasa umbellus",
+    "83": "prairie chicken, prairie grouse, prairie fowl",
+    "84": "peacock",
+    "85": "quail",
+    "86": "partridge",
+    "87": "African grey, African gray, Psittacus erithacus",
+    "88": "macaw",
+    "89": "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita",
+    "90": "lorikeet",
+    "91": "coucal",
+    "92": "bee eater",
+    "93": "hornbill",
+    "94": "hummingbird",
+    "95": "jacamar",
+    "96": "toucan",
+    "97": "drake",
+    "98": "red-breasted merganser, Mergus serrator",
+    "99": "goose",
+    "100": "black swan, Cygnus atratus",
+    "101": "tusker",
+    "102": "echidna, spiny anteater, anteater",
+    "103": "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus",
+    "104": "wallaby, brush kangaroo",
+    "105": "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus",
+    "106": "wombat",
+    "107": "jellyfish",
+    "108": "sea anemone, anemone",
+    "109": "brain coral",
+    "110": "flatworm, platyhelminth",
+    "111": "nematode, nematode worm, roundworm",
+    "112": "conch",
+    "113": "snail",
+    "114": "slug",
+    "115": "sea slug, nudibranch",
+    "116": "chiton, coat-of-mail shell, sea cradle, polyplacophore",
+    "117": "chambered nautilus, pearly nautilus, nautilus",
+    "118": "Dungeness crab, Cancer magister",
+    "119": "rock crab, Cancer irroratus",
+    "120": "fiddler crab",
+    "121": "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica",
+    "122": "American lobster, Northern lobster, Maine lobster, Homarus americanus",
+    "123": "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
+    "124": "crayfish, crawfish, crawdad, crawdaddy",
+    "125": "hermit crab",
+    "126": "isopod",
+    "127": "white stork, Ciconia ciconia",
+    "128": "black stork, Ciconia nigra",
+    "129": "spoonbill",
+    "130": "flamingo",
+    "131": "little blue heron, Egretta caerulea",
+    "132": "American egret, great white heron, Egretta albus",
+    "133": "bittern",
+    "134": "crane",
+    "135": "limpkin, Aramus pictus",
+    "136": "European gallinule, Porphyrio porphyrio",
+    "137": "American coot, marsh hen, mud hen, water hen, Fulica americana",
+    "138": "bustard",
+    "139": "ruddy turnstone, Arenaria interpres",
+    "140": "red-backed sandpiper, dunlin, Erolia alpina",
+    "141": "redshank, Tringa totanus",
+    "142": "dowitcher",
+    "143": "oystercatcher, oyster catcher",
+    "144": "pelican",
+    "145": "king penguin, Aptenodytes patagonica",
+    "146": "albatross, mollymawk",
+    "147": "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus",
+    "148": "killer whale, killer, orca, grampus, sea wolf, Orcinus orca",
+    "149": "dugong, Dugong dugon",
+    "150": "sea lion",
+    "151": "Chihuahua",
+    "152": "Japanese spaniel",
+    "153": "Maltese dog, Maltese terrier, Maltese",
+    "154": "Pekinese, Pekingese, Peke",
+    "155": "Shih-Tzu",
+    "156": "Blenheim spaniel",
+    "157": "papillon",
+    "158": "toy terrier",
+    "159": "Rhodesian ridgeback",
+    "160": "Afghan hound, Afghan",
+    "161": "basset, basset hound",
+    "162": "beagle",
+    "163": "bloodhound, sleuthhound",
+    "164": "bluetick",
+    "165": "black-and-tan coonhound",
+    "166": "Walker hound, Walker foxhound",
+    "167": "English foxhound",
+    "168": "redbone",
+    "169": "borzoi, Russian wolfhound",
+    "170": "Irish wolfhound",
+    "171": "Italian greyhound",
+    "172": "whippet",
+    "173": "Ibizan hound, Ibizan Podenco",
+    "174": "Norwegian elkhound, elkhound",
+    "175": "otterhound, otter hound",
+    "176": "Saluki, gazelle hound",
+    "177": "Scottish deerhound, deerhound",
+    "178": "Weimaraner",
+    "179": "Staffordshire bullterrier, Staffordshire bull terrier",
+    "180": "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier",
+    "181": "Bedlington terrier",
+    "182": "Border terrier",
+    "183": "Kerry blue terrier",
+    "184": "Irish terrier",
+    "185": "Norfolk terrier",
+    "186": "Norwich terrier",
+    "187": "Yorkshire terrier",
+    "188": "wire-haired fox terrier",
+    "189": "Lakeland terrier",
+    "190": "Sealyham terrier, Sealyham",
+    "191": "Airedale, Airedale terrier",
+    "192": "cairn, cairn terrier",
+    "193": "Australian terrier",
+    "194": "Dandie Dinmont, Dandie Dinmont terrier",
+    "195": "Boston bull, Boston terrier",
+    "196": "miniature schnauzer",
+    "197": "giant schnauzer",
+    "198": "standard schnauzer",
+    "199": "Scotch terrier, Scottish terrier, Scottie",
+    "200": "Tibetan terrier, chrysanthemum dog",
+    "201": "silky terrier, Sydney silky",
+    "202": "soft-coated wheaten terrier",
+    "203": "West Highland white terrier",
+    "204": "Lhasa, Lhasa apso",
+    "205": "flat-coated retriever",
+    "206": "curly-coated retriever",
+    "207": "golden retriever",
+    "208": "Labrador retriever",
+    "209": "Chesapeake Bay retriever",
+    "210": "German short-haired pointer",
+    "211": "vizsla, Hungarian pointer",
+    "212": "English setter",
+    "213": "Irish setter, red setter",
+    "214": "Gordon setter",
+    "215": "Brittany spaniel",
+    "216": "clumber, clumber spaniel",
+    "217": "English springer, English springer spaniel",
+    "218": "Welsh springer spaniel",
+    "219": "cocker spaniel, English cocker spaniel, cocker",
+    "220": "Sussex spaniel",
+    "221": "Irish water spaniel",
+    "222": "kuvasz",
+    "223": "schipperke",
+    "224": "groenendael",
+    "225": "malinois",
+    "226": "briard",
+    "227": "kelpie",
+    "228": "komondor",
+    "229": "Old English sheepdog, bobtail",
+    "230": "Shetland sheepdog, Shetland sheep dog, Shetland",
+    "231": "collie",
+    "232": "Border collie",
+    "233": "Bouvier des Flandres, Bouviers des Flandres",
+    "234": "Rottweiler",
+    "235": "German shepherd, German shepherd dog, German police dog, alsatian",
+    "236": "Doberman, Doberman pinscher",
+    "237": "miniature pinscher",
+    "238": "Greater Swiss Mountain dog",
+    "239": "Bernese mountain dog",
+    "240": "Appenzeller",
+    "241": "EntleBucher",
+    "242": "boxer",
+    "243": "bull mastiff",
+    "244": "Tibetan mastiff",
+    "245": "French bulldog",
+    "246": "Great Dane",
+    "247": "Saint Bernard, St Bernard",
+    "248": "Eskimo dog, husky",
+    "249": "malamute, malemute, Alaskan malamute",
+    "250": "Siberian husky",
+    "251": "dalmatian, coach dog, carriage dog",
+    "252": "affenpinscher, monkey pinscher, monkey dog",
+    "253": "basenji",
+    "254": "pug, pug-dog",
+    "255": "Leonberg",
+    "256": "Newfoundland, Newfoundland dog",
+    "257": "Great Pyrenees",
+    "258": "Samoyed, Samoyede",
+    "259": "Pomeranian",
+    "260": "chow, chow chow",
+    "261": "keeshond",
+    "262": "Brabancon griffon",
+    "263": "Pembroke, Pembroke Welsh corgi",
+    "264": "Cardigan, Cardigan Welsh corgi",
+    "265": "toy poodle",
+    "266": "miniature poodle",
+    "267": "standard poodle",
+    "268": "Mexican hairless",
+    "269": "timber wolf, grey wolf, gray wolf, Canis lupus",
+    "270": "white wolf, Arctic wolf, Canis lupus tundrarum",
+    "271": "red wolf, maned wolf, Canis rufus, Canis niger",
+    "272": "coyote, prairie wolf, brush wolf, Canis latrans",
+    "273": "dingo, warrigal, warragal, Canis dingo",
+    "274": "dhole, Cuon alpinus",
+    "275": "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus",
+    "276": "hyena, hyaena",
+    "277": "red fox, Vulpes vulpes",
+    "278": "kit fox, Vulpes macrotis",
+    "279": "Arctic fox, white fox, Alopex lagopus",
+    "280": "grey fox, gray fox, Urocyon cinereoargenteus",
+    "281": "tabby, tabby cat",
+    "282": "tiger cat",
+    "283": "Persian cat",
+    "284": "Siamese cat, Siamese",
+    "285": "Egyptian cat",
+    "286": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
+    "287": "lynx, catamount",
+    "288": "leopard, Panthera pardus",
+    "289": "snow leopard, ounce, Panthera uncia",
+    "290": "jaguar, panther, Panthera onca, Felis onca",
+    "291": "lion, king of beasts, Panthera leo",
+    "292": "tiger, Panthera tigris",
+    "293": "cheetah, chetah, Acinonyx jubatus",
+    "294": "brown bear, bruin, Ursus arctos",
+    "295": "American black bear, black bear, Ursus americanus, Euarctos americanus",
+    "296": "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus",
+    "297": "sloth bear, Melursus ursinus, Ursus ursinus",
+    "298": "mongoose",
+    "299": "meerkat, mierkat",
+    "300": "tiger beetle",
+    "301": "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle",
+    "302": "ground beetle, carabid beetle",
+    "303": "long-horned beetle, longicorn, longicorn beetle",
+    "304": "leaf beetle, chrysomelid",
+    "305": "dung beetle",
+    "306": "rhinoceros beetle",
+    "307": "weevil",
+    "308": "fly",
+    "309": "bee",
+    "310": "ant, emmet, pismire",
+    "311": "grasshopper, hopper",
+    "312": "cricket",
+    "313": "walking stick, walkingstick, stick insect",
+    "314": "cockroach, roach",
+    "315": "mantis, mantid",
+    "316": "cicada, cicala",
+    "317": "leafhopper",
+    "318": "lacewing, lacewing fly",
+    "319": "dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+    "320": "damselfly",
+    "321": "admiral",
+    "322": "ringlet, ringlet butterfly",
+    "323": "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus",
+    "324": "cabbage butterfly",
+    "325": "sulphur butterfly, sulfur butterfly",
+    "326": "lycaenid, lycaenid butterfly",
+    "327": "starfish, sea star",
+    "328": "sea urchin",
+    "329": "sea cucumber, holothurian",
+    "330": "wood rabbit, cottontail, cottontail rabbit",
+    "331": "hare",
+    "332": "Angora, Angora rabbit",
+    "333": "hamster",
+    "334": "porcupine, hedgehog",
+    "335": "fox squirrel, eastern fox squirrel, Sciurus niger",
+    "336": "marmot",
+    "337": "beaver",
+    "338": "guinea pig, Cavia cobaya",
+    "339": "sorrel",
+    "340": "zebra",
+    "341": "hog, pig, grunter, squealer, Sus scrofa",
+    "342": "wild boar, boar, Sus scrofa",
+    "343": "warthog",
+    "344": "hippopotamus, hippo, river horse, Hippopotamus amphibius",
+    "345": "ox",
+    "346": "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis",
+    "347": "bison",
+    "348": "ram, tup",
+    "349": "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis",
+    "350": "ibex, Capra ibex",
+    "351": "hartebeest",
+    "352": "impala, Aepyceros melampus",
+    "353": "gazelle",
+    "354": "Arabian camel, dromedary, Camelus dromedarius",
+    "355": "llama",
+    "356": "weasel",
+    "357": "mink",
+    "358": "polecat, fitch, foulmart, foumart, Mustela putorius",
+    "359": "black-footed ferret, ferret, Mustela nigripes",
+    "360": "otter",
+    "361": "skunk, polecat, wood pussy",
+    "362": "badger",
+    "363": "armadillo",
+    "364": "three-toed sloth, ai, Bradypus tridactylus",
+    "365": "orangutan, orang, orangutang, Pongo pygmaeus",
+    "366": "gorilla, Gorilla gorilla",
+    "367": "chimpanzee, chimp, Pan troglodytes",
+    "368": "gibbon, Hylobates lar",
+    "369": "siamang, Hylobates syndactylus, Symphalangus syndactylus",
+    "370": "guenon, guenon monkey",
+    "371": "patas, hussar monkey, Erythrocebus patas",
+    "372": "baboon",
+    "373": "macaque",
+    "374": "langur",
+    "375": "colobus, colobus monkey",
+    "376": "proboscis monkey, Nasalis larvatus",
+    "377": "marmoset",
+    "378": "capuchin, ringtail, Cebus capucinus",
+    "379": "howler monkey, howler",
+    "380": "titi, titi monkey",
+    "381": "spider monkey, Ateles geoffroyi",
+    "382": "squirrel monkey, Saimiri sciureus",
+    "383": "Madagascar cat, ring-tailed lemur, Lemur catta",
+    "384": "indri, indris, Indri indri, Indri brevicaudatus",
+    "385": "Indian elephant, Elephas maximus",
+    "386": "African elephant, Loxodonta africana",
+    "387": "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens",
+    "388": "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca",
+    "389": "barracouta, snoek",
+    "390": "eel",
+    "391": "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch",
+    "392": "rock beauty, Holocanthus tricolor",
+    "393": "anemone fish",
+    "394": "sturgeon",
+    "395": "gar, garfish, garpike, billfish, Lepisosteus osseus",
+    "396": "lionfish",
+    "397": "puffer, pufferfish, blowfish, globefish",
+    "398": "abacus",
+    "399": "abaya",
+    "400": "academic gown, academic robe, judge robe",
+    "401": "accordion, piano accordion, squeeze box",
+    "402": "acoustic guitar",
+    "403": "aircraft carrier, carrier, flattop, attack aircraft carrier",
+    "404": "airliner",
+    "405": "airship, dirigible",
+    "406": "altar",
+    "407": "ambulance",
+    "408": "amphibian, amphibious vehicle",
+    "409": "analog clock",
+    "410": "apiary, bee house",
+    "411": "apron",
+    "412": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
+    "413": "assault rifle, assault gun",
+    "414": "backpack, back pack, knapsack, packsack, rucksack, haversack",
+    "415": "bakery, bakeshop, bakehouse",
+    "416": "balance beam, beam",
+    "417": "balloon",
+    "418": "ballpoint, ballpoint pen, ballpen, Biro",
+    "419": "Band Aid",
+    "420": "banjo",
+    "421": "bannister, banister, balustrade, balusters, handrail",
+    "422": "barbell",
+    "423": "barber chair",
+    "424": "barbershop",
+    "425": "barn",
+    "426": "barometer",
+    "427": "barrel, cask",
+    "428": "barrow, garden cart, lawn cart, wheelbarrow",
+    "429": "baseball",
+    "430": "basketball",
+    "431": "bassinet",
+    "432": "bassoon",
+    "433": "bathing cap, swimming cap",
+    "434": "bath towel",
+    "435": "bathtub, bathing tub, bath, tub",
+    "436": "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon",
+    "437": "beacon, lighthouse, beacon light, pharos",
+    "438": "beaker",
+    "439": "bearskin, busby, shako",
+    "440": "beer bottle",
+    "441": "beer glass",
+    "442": "bell cote, bell cot",
+    "443": "bib",
+    "444": "bicycle-built-for-two, tandem bicycle, tandem",
+    "445": "bikini, two-piece",
+    "446": "binder, ring-binder",
+    "447": "binoculars, field glasses, opera glasses",
+    "448": "birdhouse",
+    "449": "boathouse",
+    "450": "bobsled, bobsleigh, bob",
+    "451": "bolo tie, bolo, bola tie, bola",
+    "452": "bonnet, poke bonnet",
+    "453": "bookcase",
+    "454": "bookshop, bookstore, bookstall",
+    "455": "bottlecap",
+    "456": "bow",
+    "457": "bow tie, bow-tie, bowtie",
+    "458": "brass, memorial tablet, plaque",
+    "459": "brassiere, bra, bandeau",
+    "460": "breakwater, groin, groyne, mole, bulwark, seawall, jetty",
+    "461": "breastplate, aegis, egis",
+    "462": "broom",
+    "463": "bucket, pail",
+    "464": "buckle",
+    "465": "bulletproof vest",
+    "466": "bullet train, bullet",
+    "467": "butcher shop, meat market",
+    "468": "cab, hack, taxi, taxicab",
+    "469": "caldron, cauldron",
+    "470": "candle, taper, wax light",
+    "471": "cannon",
+    "472": "canoe",
+    "473": "can opener, tin opener",
+    "474": "cardigan",
+    "475": "car mirror",
+    "476": "carousel, carrousel, merry-go-round, roundabout, whirligig",
+    "477": "carpenters kit, tool kit",
+    "478": "carton",
+    "479": "car wheel",
+    "480": "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
+    "481": "cassette",
+    "482": "cassette player",
+    "483": "castle",
+    "484": "catamaran",
+    "485": "CD player",
+    "486": "cello, violoncello",
+    "487": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
+    "488": "chain",
+    "489": "chainlink fence",
+    "490": "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour",
+    "491": "chain saw, chainsaw",
+    "492": "chest",
+    "493": "chiffonier, commode",
+    "494": "chime, bell, gong",
+    "495": "china cabinet, china closet",
+    "496": "Christmas stocking",
+    "497": "church, church building",
+    "498": "cinema, movie theater, movie theatre, movie house, picture palace",
+    "499": "cleaver, meat cleaver, chopper",
+    "500": "cliff dwelling",
+    "501": "cloak",
+    "502": "clog, geta, patten, sabot",
+    "503": "cocktail shaker",
+    "504": "coffee mug",
+    "505": "coffeepot",
+    "506": "coil, spiral, volute, whorl, helix",
+    "507": "combination lock",
+    "508": "computer keyboard, keypad",
+    "509": "confectionery, confectionary, candy store",
+    "510": "container ship, containership, container vessel",
+    "511": "convertible",
+    "512": "corkscrew, bottle screw",
+    "513": "cornet, horn, trumpet, trump",
+    "514": "cowboy boot",
+    "515": "cowboy hat, ten-gallon hat",
+    "516": "cradle",
+    "517": "crane",
+    "518": "crash helmet",
+    "519": "crate",
+    "520": "crib, cot",
+    "521": "Crock Pot",
+    "522": "croquet ball",
+    "523": "crutch",
+    "524": "cuirass",
+    "525": "dam, dike, dyke",
+    "526": "desk",
+    "527": "desktop computer",
+    "528": "dial telephone, dial phone",
+    "529": "diaper, nappy, napkin",
+    "530": "digital clock",
+    "531": "digital watch",
+    "532": "dining table, board",
+    "533": "dishrag, dishcloth",
+    "534": "dishwasher, dish washer, dishwashing machine",
+    "535": "disk brake, disc brake",
+    "536": "dock, dockage, docking facility",
+    "537": "dogsled, dog sled, dog sleigh",
+    "538": "dome",
+    "539": "doormat, welcome mat",
+    "540": "drilling platform, offshore rig",
+    "541": "drum, membranophone, tympan",
+    "542": "drumstick",
+    "543": "dumbbell",
+    "544": "Dutch oven",
+    "545": "electric fan, blower",
+    "546": "electric guitar",
+    "547": "electric locomotive",
+    "548": "entertainment center",
+    "549": "envelope",
+    "550": "espresso maker",
+    "551": "face powder",
+    "552": "feather boa, boa",
+    "553": "file, file cabinet, filing cabinet",
+    "554": "fireboat",
+    "555": "fire engine, fire truck",
+    "556": "fire screen, fireguard",
+    "557": "flagpole, flagstaff",
+    "558": "flute, transverse flute",
+    "559": "folding chair",
+    "560": "football helmet",
+    "561": "forklift",
+    "562": "fountain",
+    "563": "fountain pen",
+    "564": "four-poster",
+    "565": "freight car",
+    "566": "French horn, horn",
+    "567": "frying pan, frypan, skillet",
+    "568": "fur coat",
+    "569": "garbage truck, dustcart",
+    "570": "gasmask, respirator, gas helmet",
+    "571": "gas pump, gasoline pump, petrol pump, island dispenser",
+    "572": "goblet",
+    "573": "go-kart",
+    "574": "golf ball",
+    "575": "golfcart, golf cart",
+    "576": "gondola",
+    "577": "gong, tam-tam",
+    "578": "gown",
+    "579": "grand piano, grand",
+    "580": "greenhouse, nursery, glasshouse",
+    "581": "grille, radiator grille",
+    "582": "grocery store, grocery, food market, market",
+    "583": "guillotine",
+    "584": "hair slide",
+    "585": "hair spray",
+    "586": "half track",
+    "587": "hammer",
+    "588": "hamper",
+    "589": "hand blower, blow dryer, blow drier, hair dryer, hair drier",
+    "590": "hand-held computer, hand-held microcomputer",
+    "591": "handkerchief, hankie, hanky, hankey",
+    "592": "hard disc, hard disk, fixed disk",
+    "593": "harmonica, mouth organ, harp, mouth harp",
+    "594": "harp",
+    "595": "harvester, reaper",
+    "596": "hatchet",
+    "597": "holster",
+    "598": "home theater, home theatre",
+    "599": "honeycomb",
+    "600": "hook, claw",
+    "601": "hoopskirt, crinoline",
+    "602": "horizontal bar, high bar",
+    "603": "horse cart, horse-cart",
+    "604": "hourglass",
+    "605": "iPod",
+    "606": "iron, smoothing iron",
+    "607": "jack-o-lantern",
+    "608": "jean, blue jean, denim",
+    "609": "jeep, landrover",
+    "610": "jersey, T-shirt, tee shirt",
+    "611": "jigsaw puzzle",
+    "612": "jinrikisha, ricksha, rickshaw",
+    "613": "joystick",
+    "614": "kimono",
+    "615": "knee pad",
+    "616": "knot",
+    "617": "lab coat, laboratory coat",
+    "618": "ladle",
+    "619": "lampshade, lamp shade",
+    "620": "laptop, laptop computer",
+    "621": "lawn mower, mower",
+    "622": "lens cap, lens cover",
+    "623": "letter opener, paper knife, paperknife",
+    "624": "library",
+    "625": "lifeboat",
+    "626": "lighter, light, igniter, ignitor",
+    "627": "limousine, limo",
+    "628": "liner, ocean liner",
+    "629": "lipstick, lip rouge",
+    "630": "Loafer",
+    "631": "lotion",
+    "632": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
+    "633": "loupe, jewelers loupe",
+    "634": "lumbermill, sawmill",
+    "635": "magnetic compass",
+    "636": "mailbag, postbag",
+    "637": "mailbox, letter box",
+    "638": "maillot",
+    "639": "maillot, tank suit",
+    "640": "manhole cover",
+    "641": "maraca",
+    "642": "marimba, xylophone",
+    "643": "mask",
+    "644": "matchstick",
+    "645": "maypole",
+    "646": "maze, labyrinth",
+    "647": "measuring cup",
+    "648": "medicine chest, medicine cabinet",
+    "649": "megalith, megalithic structure",
+    "650": "microphone, mike",
+    "651": "microwave, microwave oven",
+    "652": "military uniform",
+    "653": "milk can",
+    "654": "minibus",
+    "655": "miniskirt, mini",
+    "656": "minivan",
+    "657": "missile",
+    "658": "mitten",
+    "659": "mixing bowl",
+    "660": "mobile home, manufactured home",
+    "661": "Model T",
+    "662": "modem",
+    "663": "monastery",
+    "664": "monitor",
+    "665": "moped",
+    "666": "mortar",
+    "667": "mortarboard",
+    "668": "mosque",
+    "669": "mosquito net",
+    "670": "motor scooter, scooter",
+    "671": "mountain bike, all-terrain bike, off-roader",
+    "672": "mountain tent",
+    "673": "mouse, computer mouse",
+    "674": "mousetrap",
+    "675": "moving van",
+    "676": "muzzle",
+    "677": "nail",
+    "678": "neck brace",
+    "679": "necklace",
+    "680": "nipple",
+    "681": "notebook, notebook computer",
+    "682": "obelisk",
+    "683": "oboe, hautboy, hautbois",
+    "684": "ocarina, sweet potato",
+    "685": "odometer, hodometer, mileometer, milometer",
+    "686": "oil filter",
+    "687": "organ, pipe organ",
+    "688": "oscilloscope, scope, cathode-ray oscilloscope, CRO",
+    "689": "overskirt",
+    "690": "oxcart",
+    "691": "oxygen mask",
+    "692": "packet",
+    "693": "paddle, boat paddle",
+    "694": "paddlewheel, paddle wheel",
+    "695": "padlock",
+    "696": "paintbrush",
+    "697": "pajama, pyjama, pjs, jammies",
+    "698": "palace",
+    "699": "panpipe, pandean pipe, syrinx",
+    "700": "paper towel",
+    "701": "parachute, chute",
+    "702": "parallel bars, bars",
+    "703": "park bench",
+    "704": "parking meter",
+    "705": "passenger car, coach, carriage",
+    "706": "patio, terrace",
+    "707": "pay-phone, pay-station",
+    "708": "pedestal, plinth, footstall",
+    "709": "pencil box, pencil case",
+    "710": "pencil sharpener",
+    "711": "perfume, essence",
+    "712": "Petri dish",
+    "713": "photocopier",
+    "714": "pick, plectrum, plectron",
+    "715": "pickelhaube",
+    "716": "picket fence, paling",
+    "717": "pickup, pickup truck",
+    "718": "pier",
+    "719": "piggy bank, penny bank",
+    "720": "pill bottle",
+    "721": "pillow",
+    "722": "ping-pong ball",
+    "723": "pinwheel",
+    "724": "pirate, pirate ship",
+    "725": "pitcher, ewer",
+    "726": "plane, carpenters plane, woodworking plane",
+    "727": "planetarium",
+    "728": "plastic bag",
+    "729": "plate rack",
+    "730": "plow, plough",
+    "731": "plunger, plumbers helper",
+    "732": "Polaroid camera, Polaroid Land camera",
+    "733": "pole",
+    "734": "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria",
+    "735": "poncho",
+    "736": "pool table, billiard table, snooker table",
+    "737": "pop bottle, soda bottle",
+    "738": "pot, flowerpot",
+    "739": "potters wheel",
+    "740": "power drill",
+    "741": "prayer rug, prayer mat",
+    "742": "printer",
+    "743": "prison, prison house",
+    "744": "projectile, missile",
+    "745": "projector",
+    "746": "puck, hockey puck",
+    "747": "punching bag, punch bag, punching ball, punchball",
+    "748": "purse",
+    "749": "quill, quill pen",
+    "750": "quilt, comforter, comfort, puff",
+    "751": "racer, race car, racing car",
+    "752": "racket, racquet",
+    "753": "radiator",
+    "754": "radio, wireless",
+    "755": "radio telescope, radio reflector",
+    "756": "rain barrel",
+    "757": "recreational vehicle, RV, R.V.",
+    "758": "reel",
+    "759": "reflex camera",
+    "760": "refrigerator, icebox",
+    "761": "remote control, remote",
+    "762": "restaurant, eating house, eating place, eatery",
+    "763": "revolver, six-gun, six-shooter",
+    "764": "rifle",
+    "765": "rocking chair, rocker",
+    "766": "rotisserie",
+    "767": "rubber eraser, rubber, pencil eraser",
+    "768": "rugby ball",
+    "769": "rule, ruler",
+    "770": "running shoe",
+    "771": "safe",
+    "772": "safety pin",
+    "773": "saltshaker, salt shaker",
+    "774": "sandal",
+    "775": "sarong",
+    "776": "sax, saxophone",
+    "777": "scabbard",
+    "778": "scale, weighing machine",
+    "779": "school bus",
+    "780": "schooner",
+    "781": "scoreboard",
+    "782": "screen, CRT screen",
+    "783": "screw",
+    "784": "screwdriver",
+    "785": "seat belt, seatbelt",
+    "786": "sewing machine",
+    "787": "shield, buckler",
+    "788": "shoe shop, shoe-shop, shoe store",
+    "789": "shoji",
+    "790": "shopping basket",
+    "791": "shopping cart",
+    "792": "shovel",
+    "793": "shower cap",
+    "794": "shower curtain",
+    "795": "ski",
+    "796": "ski mask",
+    "797": "sleeping bag",
+    "798": "slide rule, slipstick",
+    "799": "sliding door",
+    "800": "slot, one-armed bandit",
+    "801": "snorkel",
+    "802": "snowmobile",
+    "803": "snowplow, snowplough",
+    "804": "soap dispenser",
+    "805": "soccer ball",
+    "806": "sock",
+    "807": "solar dish, solar collector, solar furnace",
+    "808": "sombrero",
+    "809": "soup bowl",
+    "810": "space bar",
+    "811": "space heater",
+    "812": "space shuttle",
+    "813": "spatula",
+    "814": "speedboat",
+    "815": "spider web, spiders web",
+    "816": "spindle",
+    "817": "sports car, sport car",
+    "818": "spotlight, spot",
+    "819": "stage",
+    "820": "steam locomotive",
+    "821": "steel arch bridge",
+    "822": "steel drum",
+    "823": "stethoscope",
+    "824": "stole",
+    "825": "stone wall",
+    "826": "stopwatch, stop watch",
+    "827": "stove",
+    "828": "strainer",
+    "829": "streetcar, tram, tramcar, trolley, trolley car",
+    "830": "stretcher",
+    "831": "studio couch, day bed",
+    "832": "stupa, tope",
+    "833": "submarine, pigboat, sub, U-boat",
+    "834": "suit, suit of clothes",
+    "835": "sundial",
+    "836": "sunglass",
+    "837": "sunglasses, dark glasses, shades",
+    "838": "sunscreen, sunblock, sun blocker",
+    "839": "suspension bridge",
+    "840": "swab, swob, mop",
+    "841": "sweatshirt",
+    "842": "swimming trunks, bathing trunks",
+    "843": "swing",
+    "844": "switch, electric switch, electrical switch",
+    "845": "syringe",
+    "846": "table lamp",
+    "847": "tank, army tank, armored combat vehicle, armoured combat vehicle",
+    "848": "tape player",
+    "849": "teapot",
+    "850": "teddy, teddy bear",
+    "851": "television, television system",
+    "852": "tennis ball",
+    "853": "thatch, thatched roof",
+    "854": "theater curtain, theatre curtain",
+    "855": "thimble",
+    "856": "thresher, thrasher, threshing machine",
+    "857": "throne",
+    "858": "tile roof",
+    "859": "toaster",
+    "860": "tobacco shop, tobacconist shop, tobacconist",
+    "861": "toilet seat",
+    "862": "torch",
+    "863": "totem pole",
+    "864": "tow truck, tow car, wrecker",
+    "865": "toyshop",
+    "866": "tractor",
+    "867": "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi",
+    "868": "tray",
+    "869": "trench coat",
+    "870": "tricycle, trike, velocipede",
+    "871": "trimaran",
+    "872": "tripod",
+    "873": "triumphal arch",
+    "874": "trolleybus, trolley coach, trackless trolley",
+    "875": "trombone",
+    "876": "tub, vat",
+    "877": "turnstile",
+    "878": "typewriter keyboard",
+    "879": "umbrella",
+    "880": "unicycle, monocycle",
+    "881": "upright, upright piano",
+    "882": "vacuum, vacuum cleaner",
+    "883": "vase",
+    "884": "vault",
+    "885": "velvet",
+    "886": "vending machine",
+    "887": "vestment",
+    "888": "viaduct",
+    "889": "violin, fiddle",
+    "890": "volleyball",
+    "891": "waffle iron",
+    "892": "wall clock",
+    "893": "wallet, billfold, notecase, pocketbook",
+    "894": "wardrobe, closet, press",
+    "895": "warplane, military plane",
+    "896": "washbasin, handbasin, washbowl, lavabo, wash-hand basin",
+    "897": "washer, automatic washer, washing machine",
+    "898": "water bottle",
+    "899": "water jug",
+    "900": "water tower",
+    "901": "whiskey jug",
+    "902": "whistle",
+    "903": "wig",
+    "904": "window screen",
+    "905": "window shade",
+    "906": "Windsor tie",
+    "907": "wine bottle",
+    "908": "wing",
+    "909": "wok",
+    "910": "wooden spoon",
+    "911": "wool, woolen, woollen",
+    "912": "worm fence, snake fence, snake-rail fence, Virginia fence",
+    "913": "wreck",
+    "914": "yawl",
+    "915": "yurt",
+    "916": "web site, website, internet site, site",
+    "917": "comic book",
+    "918": "crossword puzzle, crossword",
+    "919": "street sign",
+    "920": "traffic light, traffic signal, stoplight",
+    "921": "book jacket, dust cover, dust jacket, dust wrapper",
+    "922": "menu",
+    "923": "plate",
+    "924": "guacamole",
+    "925": "consomme",
+    "926": "hot pot, hotpot",
+    "927": "trifle",
+    "928": "ice cream, icecream",
+    "929": "ice lolly, lolly, lollipop, popsicle",
+    "930": "French loaf",
+    "931": "bagel, beigel",
+    "932": "pretzel",
+    "933": "cheeseburger",
+    "934": "hotdog, hot dog, red hot",
+    "935": "mashed potato",
+    "936": "head cabbage",
+    "937": "broccoli",
+    "938": "cauliflower",
+    "939": "zucchini, courgette",
+    "940": "spaghetti squash",
+    "941": "acorn squash",
+    "942": "butternut squash",
+    "943": "cucumber, cuke",
+    "944": "artichoke, globe artichoke",
+    "945": "bell pepper",
+    "946": "cardoon",
+    "947": "mushroom",
+    "948": "Granny Smith",
+    "949": "strawberry",
+    "950": "orange",
+    "951": "lemon",
+    "952": "fig",
+    "953": "pineapple, ananas",
+    "954": "banana",
+    "955": "jackfruit, jak, jack",
+    "956": "custard apple",
+    "957": "pomegranate",
+    "958": "hay",
+    "959": "carbonara",
+    "960": "chocolate sauce, chocolate syrup",
+    "961": "dough",
+    "962": "meat loaf, meatloaf",
+    "963": "pizza, pizza pie",
+    "964": "potpie",
+    "965": "burrito",
+    "966": "red wine",
+    "967": "espresso",
+    "968": "cup",
+    "969": "eggnog",
+    "970": "alp",
+    "971": "bubble",
+    "972": "cliff, drop, drop-off",
+    "973": "coral reef",
+    "974": "geyser",
+    "975": "lakeside, lakeshore",
+    "976": "promontory, headland, head, foreland",
+    "977": "sandbar, sand bar",
+    "978": "seashore, coast, seacoast, sea-coast",
+    "979": "valley, vale",
+    "980": "volcano",
+    "981": "ballplayer, baseball player",
+    "982": "groom, bridegroom",
+    "983": "scuba diver",
+    "984": "rapeseed",
+    "985": "daisy",
+    "986": "yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+    "987": "corn",
+    "988": "acorn",
+    "989": "hip, rose hip, rosehip",
+    "990": "buckeye, horse chestnut, conker",
+    "991": "coral fungus",
+    "992": "agaric",
+    "993": "gyromitra",
+    "994": "stinkhorn, carrion fungus",
+    "995": "earthstar",
+    "996": "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa",
+    "997": "bolete",
+    "998": "ear, spike, capitulum",
+    "999": "toilet tissue, toilet paper, bathroom tissue"
+  }
+}

PixelDiT2-H-16-256/pipeline.py

@@ -0,0 +1,839 @@
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import importlib.util
+import inspect
+import json
+import sys
+from pathlib import Path
+from typing import Any, Callable
+
+import torch
+from diffusers.callbacks import MultiPipelineCallbacks, PipelineCallback
+from transformers import DINOv3ViTModel
+from diffusers.models.modeling_utils import ModelMixin
+from diffusers.pipelines.pipeline_utils import DiffusionPipeline, ImagePipelineOutput
+from diffusers.schedulers.scheduling_utils import SchedulerMixin
+from diffusers.utils import deprecate, is_torch_xla_available, logging, replace_example_docstring
+from diffusers.utils.torch_utils import randn_tensor
+from PIL import Image
+
+if is_torch_xla_available():
+    import torch_xla.core.xla_model as xm
+
+    XLA_AVAILABLE = True
+else:
+    XLA_AVAILABLE = False
+
+logger = logging.get_logger(__name__)
+
+EXAMPLE_DOC_STRING = """
+    Examples:
+        ```py
+        >>> from pathlib import Path
+        >>> import torch
+        >>> from diffusers import DiffusionPipeline
+
+        >>> model_dir = Path("pixeldit2-diffusers/PixelDiT2-H-16-256").resolve()
+        >>> pipe = DiffusionPipeline.from_pretrained(
+        ...     str(model_dir),
+        ...     local_files_only=True,
+        ...     custom_pipeline=str(model_dir / "pipeline.py"),
+        ...     trust_remote_code=True,
+        ...     torch_dtype=torch.bfloat16,
+        ... ).to("cuda:0")
+
+        >>> image = pipe(
+        ...     class_labels="golden retriever",
+        ...     num_inference_steps=50,
+        ...     guidance_scale=2.4,
+        ...     generator=torch.Generator(device="cuda:0").manual_seed(0),
+        ... ).images[0]
+        >>> image.save("sample.png")
+
+        >>> # Hugging Face Hub style model id: UserID/RepoID
+        >>> # RepoID is usually like "modelname-diffusers"
+        >>> # Example: "BiliSakura/PixelDiT2-diffusers" with subfolder "PixelDiT2-H-16-256"
+        ```
+"""
+
+
+class PixelDiT2Pipeline(DiffusionPipeline):
+    r"""
+    Class-conditional image pipeline for PixelDiT2.
+
+    PixelDiT2 denoises directly in RGB pixel space (no VAE). The transformer predicts the clean
+    image `x0`. [`PixelDiT2Scheduler`] integrates that prediction on `linspace(0, 1, steps + 1)`:
+    Heun on every interval except the last, which is Euler. `solver="euler"` uses the same knots
+    with one evaluation per interval.
+
+    Classifier-free guidance mixes a grounded class-conditional forward with an ungrounded null-class
+    forward. Guidance is applied only while the flow time is strictly greater than
+    `guidance_interval_min` and strictly less than `guidance_interval_max`.
+
+    The pipeline inherits from [`DiffusionPipeline`]. Check the superclass documentation for the
+    generic methods (download, save, device placement, CPU offload).
+
+    Args:
+        transformer ([`ModelMixin`]):
+            PixelDiT2 denoiser (`PixelDiT2Transformer2DModel`) that predicts `x0` in pixel space.
+        scheduler ([`PixelDiT2Scheduler`]):
+            Flow scheduler. The paper setting is Heun, `t_eps=0.05`. Loaded from `scheduler/`.
+        encoder (`DINOv3ViTModel`, *optional*):
+            Original DINOv3 checkpoint shipped in `encoder/`. The 256px variant includes DINOv3-S/16.
+            Sampling uses the mapped copy stored inside the transformer, so this module is not called.
+        id2label (`dict[int, str]`, *optional*):
+            ImageNet class id to English label mapping. Values may contain comma-separated synonyms.
+            Loaded from `model_index.json` when omitted.
+
+        Sampling defaults are not pipeline config. When `__call__` omits them, they are read from
+        `transformer.config`: `default_num_inference_steps`, `guidance_scale`, `guidance_interval_min`,
+        `guidance_interval_max`, `noise_scale`, and `t_eps`. Missing transformer entries fall back to
+        50 steps, CFG 2.4 on `(0.125, 0.9)`, `noise_scale` 1.0, and `t_eps` 0.05.
+    """
+
+    _optional_components = ["encoder"]
+    _callback_tensor_inputs = ["latents"]
+
+    def __init__(
+        self,
+        transformer: ModelMixin,
+        scheduler: SchedulerMixin,
+        encoder: DINOv3ViTModel | None = None,
+        id2label: dict[int, str] | dict[str, str] | None = None,
+    ):
+        super().__init__()
+        if scheduler is None:
+            scheduler = self._default_scheduler()
+
+        self.register_modules(transformer=transformer, scheduler=scheduler, encoder=encoder)
+        normalized_labels = self._normalize_id2label(id2label)
+        self.register_to_config(id2label={str(key): value for key, value in normalized_labels.items()})
+        self._id2label = normalized_labels
+        self.labels = self._build_label2id(self._id2label)
+        self._labels_loaded_from_model_index = bool(self._id2label)
+
+    def _sampling_default(self, name: str, fallback: float) -> float:
+        r"""
+        Read a sampling default from the transformer config.
+
+        Args:
+            name (`str`):
+                Key on `transformer.config`, such as `guidance_scale` or `noise_scale`.
+            fallback (`float`):
+                Value used when the transformer config does not define `name`.
+
+        Returns:
+            `float`: The configured value, or `fallback`.
+        """
+        value = getattr(self.transformer.config, name, None)
+        if value is None:
+            return fallback
+        return value
+
+    @staticmethod
+    def _default_scheduler() -> SchedulerMixin:
+        r"""
+        Build [`PixelDiT2Scheduler`] from the variant's scheduler module.
+
+        Returns:
+            [`PixelDiT2Scheduler`]: Heun on the paper linspace, `t_eps=0.05`.
+        """
+        module_path = Path(__file__).resolve().parent / "scheduler" / "scheduling_pixeldit2.py"
+        spec = importlib.util.spec_from_file_location("scheduling_pixeldit2", module_path)
+        if spec is None or spec.loader is None:
+            raise ImportError(f"Could not load the PixelDiT2 scheduler from {module_path}.")
+        module = importlib.util.module_from_spec(spec)
+        sys.modules["scheduling_pixeldit2"] = module
+        spec.loader.exec_module(module)
+        return module.PixelDiT2Scheduler()
+
+    def _ensure_labels_loaded(self) -> None:
+        r"""
+        Fill `id2label` from `model_index.json` when the constructor did not receive it.
+        """
+        if self._labels_loaded_from_model_index:
+            return
+        loaded = self._read_id2label_from_model_index(getattr(self.config, "_name_or_path", None))
+        if loaded:
+            self._id2label = loaded
+            self.labels = self._build_label2id(self._id2label)
+        self._labels_loaded_from_model_index = True
+
+    @staticmethod
+    def _normalize_id2label(id2label: dict[int, str] | dict[str, str] | None) -> dict[int, str]:
+        r"""
+        Cast label-map keys to integers.
+
+        Args:
+            id2label (`dict`, *optional*):
+                Class id to English label. Keys may be strings, which is how JSON stores them.
+
+        Returns:
+            `dict[int, str]`: Integer-keyed map. Empty when `id2label` is missing.
+        """
+        if not id2label:
+            return {}
+        return {int(key): value for key, value in id2label.items()}
+
+    @staticmethod
+    def _read_id2label_from_model_index(variant_path: str | None) -> dict[int, str]:
+        r"""
+        Read `id2label` from a local variant directory.
+
+        Args:
+            variant_path (`str`, *optional*):
+                Directory that contains `model_index.json`.
+
+        Returns:
+            `dict[int, str]`: Loaded map, or an empty dict when the file or key is absent.
+        """
+        if not variant_path:
+            return {}
+        model_index_path = Path(variant_path).resolve() / "model_index.json"
+        if not model_index_path.exists():
+            return {}
+        raw = json.loads(model_index_path.read_text(encoding="utf-8"))
+        id2label = raw.get("id2label")
+        if not isinstance(id2label, dict):
+            return {}
+        return {int(key): value for key, value in id2label.items()}
+
+    @staticmethod
+    def _build_label2id(id2label: dict[int, str]) -> dict[str, int]:
+        r"""
+        Invert `id2label`, splitting comma-separated synonyms.
+
+        Args:
+            id2label (`dict[int, str]`):
+                Class id to one or more English names.
+
+        Returns:
+            `dict[str, int]`: Synonym to class id, sorted by synonym.
+        """
+        label2id: dict[str, int] = {}
+        for class_id, value in id2label.items():
+            for synonym in value.split(","):
+                synonym = synonym.strip()
+                if synonym:
+                    label2id[synonym] = int(class_id)
+        return dict(sorted(label2id.items()))
+
+    @property
+    def id2label(self) -> dict[int, str]:
+        r"""
+        ImageNet id to English label map.
+
+        Returns:
+            `dict[int, str]`: The map loaded from the constructor or `model_index.json`.
+        """
+        self._ensure_labels_loaded()
+        return self._id2label
+
+    def get_label_ids(self, label: str | list[str]) -> list[int]:
+        r"""
+        Map ImageNet English labels to class ids.
+
+        Args:
+            label (`str` or `list[str]`):
+                One label or a list of labels. Each string must match a synonym in `id2label`
+                after stripping whitespace. Synonyms are the comma-separated pieces of each value.
+
+        Returns:
+            `list[int]`: Class ids in the same order as `label`.
+        """
+        self._ensure_labels_loaded()
+        label2id = self.labels
+        if not label2id:
+            raise ValueError("No English labels loaded. Ensure `id2label` exists in model_index.json.")
+        if isinstance(label, str):
+            label = [label]
+        missing = [item for item in label if item not in label2id]
+        if missing:
+            preview = ", ".join(list(label2id.keys())[:8])
+            raise ValueError(f"Unknown English label(s): {missing}. Example valid labels: {preview}, ...")
+        return [label2id[item] for item in label]
+
+    def _normalize_class_labels(self, class_labels: int | str | list[int | str]) -> list[int]:
+        r"""
+        Turn a class-label argument into a list of integer ids.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                One id, one English label, or a list of either. A list must not mix types.
+
+        Returns:
+            `list[int]`: One id per requested image.
+        """
+        if isinstance(class_labels, int):
+            return [class_labels]
+        if isinstance(class_labels, str):
+            return self.get_label_ids(class_labels)
+        if len(class_labels) == 0:
+            raise ValueError("`class_labels` must contain at least one class id or label.")
+        if isinstance(class_labels[0], str):
+            if not all(isinstance(item, str) for item in class_labels):
+                raise TypeError("`class_labels` must be all strings or all integers.")
+            return self.get_label_ids(class_labels)
+        if not all(isinstance(item, int) for item in class_labels):
+            raise TypeError("`class_labels` must be all strings or all integers.")
+        return list(class_labels)
+
+    def prepare_extra_step_kwargs(
+        self,
+        generator: torch.Generator | list[torch.Generator] | None,
+        eta: float,
+    ) -> dict[str, Any]:
+        r"""
+        Extra arguments forwarded to `scheduler.step`, depending on what that method accepts.
+
+        Args:
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                Generator passed through when the scheduler step samples noise.
+            eta (`float`):
+                DDIM eta in `[0, 1]`. Ignored by schedulers whose `step` has no `eta` argument.
+
+        Returns:
+            `dict`: Keyword arguments for `scheduler.step`.
+        """
+        extra_step_kwargs: dict[str, Any] = {}
+        step_params = set(inspect.signature(self.scheduler.step).parameters.keys())
+        if "eta" in step_params:
+            extra_step_kwargs["eta"] = eta
+        if "generator" in step_params:
+            extra_step_kwargs["generator"] = generator
+        return extra_step_kwargs
+
+    def check_inputs(
+        self,
+        class_labels: int | str | list[int | str],
+        height: int,
+        width: int,
+        num_inference_steps: int,
+        guidance_scale: float,
+        guidance_interval_min: float,
+        guidance_interval_max: float,
+        noise_scale: float,
+        t_eps: float,
+        output_type: str,
+        callback_steps: int | None = None,
+        callback_on_step_end_tensor_inputs: list[str] | None = None,
+    ) -> None:
+        r"""
+        Reject arguments that the denoiser or the sampler cannot run.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                Class ids or English labels. A list must be non-empty.
+            height (`int`):
+                Output height in pixels. Must be positive and divisible by the patch size.
+            width (`int`):
+                Output width in pixels. Must be positive and divisible by the patch size.
+            num_inference_steps (`int`):
+                Number of integration intervals. Must be >= 1.
+            guidance_scale (`float`):
+                CFG scale. Must be finite.
+            guidance_interval_min (`float`):
+                Exclusive lower flow-time bound of CFG.
+            guidance_interval_max (`float`):
+                Exclusive upper flow-time bound of CFG. Must be greater than the minimum and at most 1.
+            noise_scale (`float`):
+                Initial Gaussian scale. Must be positive.
+            t_eps (`float`):
+                Velocity denominator floor. Must be positive.
+            output_type (`str`):
+                One of `"pil"`, `"np"`, `"pt"`, `"latent"`.
+            callback_steps (`int`, *optional*):
+                Deprecated callback period. When set, must be a positive integer.
+            callback_on_step_end_tensor_inputs (`list[str]`, *optional*):
+                Tensor names the step callback may read. Each name must be in `_callback_tensor_inputs`.
+        """
+        if isinstance(class_labels, list) and len(class_labels) == 0:
+            raise ValueError("`class_labels` must contain at least one class id or label.")
+        patch_size = int(self.transformer.config.patch_size)
+        if height <= 0 or width <= 0:
+            raise ValueError(f"`height` and `width` must be positive, got {(height, width)}.")
+        if height % patch_size != 0 or width % patch_size != 0:
+            raise ValueError(f"`height` and `width` must be divisible by patch_size={patch_size}. Got {(height, width)}.")
+        if num_inference_steps < 1:
+            raise ValueError(f"`num_inference_steps` must be >= 1, got {num_inference_steps}.")
+        if not torch.isfinite(torch.tensor(guidance_scale)):
+            raise ValueError(f"`guidance_scale` must be finite, got {guidance_scale}.")
+        if not 0.0 <= guidance_interval_min < guidance_interval_max <= 1.0:
+            raise ValueError(
+                "Guidance interval must satisfy 0 <= min < max <= 1, got "
+                f"[{guidance_interval_min}, {guidance_interval_max}]."
+            )
+        if noise_scale <= 0:
+            raise ValueError(f"`noise_scale` must be positive, got {noise_scale}.")
+        if t_eps <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {t_eps}.")
+        if output_type not in {"pil", "np", "pt", "latent"}:
+            raise ValueError(f"Unsupported `output_type` {output_type!r}. Choose from 'pil', 'np', 'pt', 'latent'.")
+        if callback_steps is not None and (not isinstance(callback_steps, int) or callback_steps < 1):
+            raise ValueError(f"`callback_steps` must be a positive integer, got {callback_steps}.")
+        if callback_on_step_end_tensor_inputs is not None:
+            unknown = [name for name in callback_on_step_end_tensor_inputs if name not in self._callback_tensor_inputs]
+            if unknown:
+                raise ValueError(
+                    f"`callback_on_step_end_tensor_inputs` contains {unknown}, which are not in "
+                    f"{self._callback_tensor_inputs}."
+                )
+
+    def prepare_latents(
+        self,
+        batch_size: int,
+        num_channels: int,
+        height: int,
+        width: int,
+        dtype: torch.dtype,
+        device: torch.device,
+        generator: torch.Generator | list[torch.Generator] | None,
+        latents: torch.Tensor | None,
+        noise_scale: float,
+    ) -> torch.Tensor:
+        r"""
+        Sample the initial pixel-space noise, or validate a tensor the caller already sampled.
+
+        PixelDiT2 has no VAE, so these "latents" are RGB images. Fresh noise is multiplied by
+        `noise_scale`. A provided `latents` tensor is not rescaled.
+
+        Args:
+            batch_size (`int`):
+                Number of images.
+            num_channels (`int`):
+                Channel count. 3 for RGB.
+            height (`int`):
+                Image height in pixels.
+            width (`int`):
+                Image width in pixels.
+            dtype (`torch.dtype`):
+                Dtype of freshly sampled noise. The integration itself is accumulated in float32.
+            device (`torch.device`):
+                Device of the returned tensor.
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                Per-call RNG. A list must have length `batch_size`.
+            latents (`torch.Tensor`, *optional*):
+                Starting noise of shape `(batch_size, num_channels, height, width)`.
+            noise_scale (`float`):
+                Scale applied only when `latents` is `None`.
+
+        Returns:
+            `torch.Tensor`: Starting noise of shape `(batch_size, num_channels, height, width)`.
+        """
+        shape = (batch_size, num_channels, height, width)
+        if isinstance(generator, list) and len(generator) != batch_size:
+            raise ValueError(
+                f"You passed a list of {len(generator)} generators for a batch of {batch_size}. "
+                "The two lengths must match."
+            )
+        if latents is None:
+            latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
+            latents = latents * float(noise_scale)
+        else:
+            latents = latents.to(device=device)
+            if tuple(latents.shape) != shape:
+                raise ValueError(f"`latents` shape {tuple(latents.shape)} does not match the expected {shape}.")
+        return latents
+
+    def _prepare_scheduler(
+        self,
+        num_inference_steps: int,
+        device: torch.device,
+        timesteps: list[float] | None,
+        sigmas: list[float] | None,
+    ) -> torch.Tensor:
+        r"""
+        Install the paper linspace on `self.scheduler` and return its flow times.
+
+        [`PixelDiT2Scheduler`] owns the grid. `timesteps` and `sigmas` are rejected. Use
+        `solver="euler"` for one model evaluation per interval.
+
+        Args:
+            num_inference_steps (`int`):
+                Number of flow-time intervals.
+            device (`torch.device`):
+                Device of the scheduler tensors.
+            timesteps (`list[float]`, *optional*):
+                Not used. The scheduler builds `linspace(0, 1, steps + 1)`.
+            sigmas (`list[float]`, *optional*):
+                Not used. PixelDiT2 integrates flow time, not a sigma schedule.
+
+        Returns:
+            `torch.Tensor`: Flow times at which the denoiser is evaluated, in `[0, 1]`.
+        """
+        if timesteps is not None or sigmas is not None:
+            raise ValueError(
+                "PixelDiT2Scheduler builds its own linspace and does not accept custom `timesteps` or "
+                '`sigmas`. Use PixelDiT2Scheduler(solver="euler") for one evaluation per step.'
+            )
+        self.scheduler.set_timesteps(num_inference_steps, device=device)
+        return self.scheduler.timesteps
+
+    def _predict_x0(
+        self,
+        latents: torch.Tensor,
+        flow_time: torch.Tensor,
+        class_labels: torch.Tensor,
+        skip_grounding: bool,
+    ) -> torch.Tensor:
+        r"""
+        Run the denoiser and return an x0 prediction in float32.
+
+        Args:
+            latents (`torch.Tensor`):
+                Current sample `(batch, channels, height, width)`.
+            flow_time (`torch.Tensor`):
+                Scalar flow time in `[0, 1]`, broadcast inside the denoiser.
+            class_labels (`torch.Tensor`):
+                Class ids `(batch,)`.
+            skip_grounding (`bool`):
+                When `True`, skip the DINOv3 encoder. Used for the unconditional branch.
+
+        Returns:
+            `torch.Tensor`: x0 prediction `(batch, channels, height, width)` in float32.
+        """
+        model_input = latents
+        parameter_dtype = self.transformer.dtype
+        device_type = model_input.device.type
+        use_amp = device_type in {"cuda", "xpu"} and parameter_dtype in (torch.float16, torch.bfloat16)
+        if not use_amp:
+            model_input = model_input.to(dtype=parameter_dtype)
+        timesteps = flow_time.to(device=model_input.device, dtype=parameter_dtype).expand(model_input.shape[0])
+        with torch.autocast(device_type if device_type != "mps" else "cpu", dtype=parameter_dtype, enabled=use_amp):
+            prediction = self.transformer(
+                model_input,
+                timestep=timesteps,
+                class_labels=class_labels,
+                skip_grounding=skip_grounding,
+            ).sample
+        return prediction.float()
+
+    def _guided_x0(
+        self,
+        latents: torch.Tensor,
+        flow_time: float,
+        class_labels: torch.Tensor,
+        class_null: torch.Tensor,
+        guidance_scale: float,
+        guidance_interval_min: float,
+        guidance_interval_max: float,
+    ) -> torch.Tensor:
+        r"""
+        Classifier-free guidance on x0, restricted to the paper's time interval.
+
+        Outside the interval the scale is 1, which is exactly the conditional prediction, so the
+        ungrounded forward is skipped.
+
+        Args:
+            latents (`torch.Tensor`):
+                Current sample.
+            flow_time (`float`):
+                Scalar flow time of this evaluation.
+            class_labels (`torch.Tensor`):
+                Conditional class ids.
+            class_null (`torch.Tensor`):
+                Null class ids, one per batch row.
+            guidance_scale (`float`):
+                CFG scale used inside the interval.
+            guidance_interval_min (`float`):
+                Exclusive lower bound.
+            guidance_interval_max (`float`):
+                Exclusive upper bound.
+
+        Returns:
+            `torch.Tensor`: Guided x0, same shape as `latents`, in float32.
+        """
+        flow = torch.tensor(flow_time, device=latents.device, dtype=torch.float32)
+        conditional = self._predict_x0(latents, flow, class_labels, skip_grounding=False)
+        inside = guidance_interval_min < flow_time < guidance_interval_max
+        if guidance_scale == 1.0 or not inside:
+            return conditional
+        unconditional = self._predict_x0(latents, flow, class_null, skip_grounding=True)
+        return unconditional + guidance_scale * (conditional - unconditional)
+
+    @property
+    def guidance_scale(self) -> float:
+        r"""
+        Classifier-free guidance scale of the call that is currently running.
+
+        Returns:
+            `float`: The scale set by the active `__call__`.
+        """
+        return self._guidance_scale
+
+    @property
+    def do_classifier_free_guidance(self) -> bool:
+        r"""
+        Whether the active call can run an unconditional forward.
+
+        The unconditional forward is still skipped at timesteps outside the guidance interval.
+
+        Returns:
+            `bool`: True when `guidance_scale > 1`.
+        """
+        return self._guidance_scale > 1.0
+
+    @property
+    def num_timesteps(self) -> int:
+        r"""
+        Number of scheduler timesteps in the active call.
+
+        Returns:
+            `int`: Length of the timestep schedule, including Heun's repeated states.
+        """
+        return self._num_timesteps
+
+    @property
+    def interrupt(self) -> bool:
+        r"""
+        Whether the active denoising loop should skip remaining steps.
+
+        Returns:
+            `bool`: True after the caller sets `pipeline._interrupt = True`.
+        """
+        return self._interrupt
+
+    def _images_from_latents(self, latents: torch.Tensor, output_type: str) -> torch.Tensor | list[Image.Image]:
+        r"""
+        Convert pixel-space samples to the requested output type.
+
+        Quantization matches the original sampler: `uint8(clip((x + 1) * 127.5 + 0.5, 0, 255))`.
+
+        Args:
+            latents (`torch.Tensor`):
+                Samples of shape `(batch, channels, height, width)` in model range around `[-1, 1]`.
+            output_type (`str`):
+                `"latent"` returns `latents` unchanged. `"pt"` is float RGB in `[0, 1]` after that
+                quantization. `"np"` is `uint8` HWC arrays. `"pil"` is a list of `PIL.Image.Image`.
+
+        Returns:
+            Images in the requested type.
+        """
+        if output_type == "latent":
+            return latents
+        images = torch.clamp((latents.float() + 1.0) * 127.5 + 0.5, 0, 255).to(torch.uint8)
+        if output_type == "pt":
+            return images.float() / 255.0
+        arrays = images.permute(0, 2, 3, 1).cpu().numpy()
+        if output_type == "np":
+            return arrays
+        return [Image.fromarray(image) for image in arrays]
+
+    @torch.no_grad()
+    @replace_example_docstring(EXAMPLE_DOC_STRING)
+    def __call__(
+        self,
+        class_labels: int | str | list[int | str],
+        height: int | None = None,
+        width: int | None = None,
+        num_inference_steps: int | None = None,
+        timesteps: list[float] | None = None,
+        sigmas: list[float] | None = None,
+        guidance_scale: float | None = None,
+        guidance_interval_min: float | None = None,
+        guidance_interval_max: float | None = None,
+        noise_scale: float | None = None,
+        t_eps: float | None = None,
+        eta: float = 0.0,
+        generator: torch.Generator | list[torch.Generator] | None = None,
+        latents: torch.Tensor | None = None,
+        output_type: str = "pil",
+        return_dict: bool = True,
+        callback_on_step_end: Callable[[int, int, dict], dict] | PipelineCallback | MultiPipelineCallbacks | None = None,
+        callback_on_step_end_tensor_inputs: list[str] | None = None,
+        **kwargs,
+    ) -> ImagePipelineOutput | tuple:
+        r"""
+        Generate images from ImageNet class labels.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                One ImageNet class id, one English label, or a list of either. English labels are
+                resolved with [`get_label_ids`]. The list length is the batch size.
+            height (`int`, *optional*):
+                Image height in pixels. Defaults to `transformer.config.sample_size`. Must be
+                divisible by the patch size. Other sizes recompute rotary positions for that grid.
+            width (`int`, *optional*):
+                Image width in pixels. Defaults to `transformer.config.sample_size`.
+            num_inference_steps (`int`, *optional*):
+                Denoising intervals. Defaults to `transformer.config.default_num_inference_steps` (50).
+            timesteps (`list[float]`, *optional*):
+                Not accepted. [`PixelDiT2Scheduler`] builds `linspace(0, 1, steps + 1)`.
+            sigmas (`list[float]`, *optional*):
+                Not accepted. PixelDiT2 integrates flow time. Use `solver="euler"` for one evaluation per step.
+            guidance_scale (`float`, *optional*):
+                Classifier-free guidance scale. Defaults to `transformer.config.guidance_scale`.
+                Guidance is on when this is greater than 1 and the flow time is inside the interval.
+            guidance_interval_min (`float`, *optional*):
+                Exclusive lower flow-time bound. Defaults to `transformer.config.guidance_interval_min`.
+            guidance_interval_max (`float`, *optional*):
+                Exclusive upper flow-time bound. Defaults to `transformer.config.guidance_interval_max`.
+            noise_scale (`float`, *optional*):
+                Initial noise scale. Defaults to `transformer.config.noise_scale`. Ignored when `latents` is passed.
+            t_eps (`float`, *optional*):
+                Floor of the velocity denominator `1 - t`. Defaults to `transformer.config.t_eps` (0.05).
+            eta (`float`, defaults to 0.0):
+                DDIM eta. Ignored by [`PixelDiT2Scheduler`].
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                RNG for the initial noise.
+            latents (`torch.Tensor`, *optional*):
+                Initial noise `(batch, 3, height, width)`. Not multiplied by `noise_scale`.
+            output_type (`str`, defaults to `"pil"`):
+                `"pil"`, `"np"`, `"pt"` (float RGB in `[0, 1]`), or `"latent"` (pixels in model range).
+            return_dict (`bool`, defaults to `True`):
+                Return [`ImagePipelineOutput`] when `True`, otherwise a one-tuple of images.
+            callback_on_step_end (`Callable` or `PipelineCallback`, *optional*):
+                Called as `callback_on_step_end(pipeline, step, timestep, callback_kwargs)` after each
+                scheduler step. Return a dict to replace tensors listed in
+                `callback_on_step_end_tensor_inputs`.
+            callback_on_step_end_tensor_inputs (`list[str]`, *optional*):
+                Tensor names passed to the step callback. Defaults to `["latents"]`.
+
+        Examples:
+
+        Returns:
+            [`ImagePipelineOutput`] or `tuple`:
+                When `return_dict` is `True`, [`ImagePipelineOutput`] with the images. Otherwise a tuple
+                whose first element is the images.
+        """
+        callback = kwargs.pop("callback", None)
+        callback_steps = kwargs.pop("callback_steps", None)
+        if kwargs:
+            raise TypeError(f"Unexpected arguments: {sorted(kwargs)}.")
+        if callback_on_step_end_tensor_inputs is None:
+            callback_on_step_end_tensor_inputs = ["latents"]
+
+        if callback is not None:
+            deprecate(
+                "callback",
+                "1.0.0",
+                "Passing `callback` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
+            )
+        if callback_steps is not None:
+            deprecate(
+                "callback_steps",
+                "1.0.0",
+                "Passing `callback_steps` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
+            )
+        if isinstance(callback_on_step_end, (PipelineCallback, MultiPipelineCallbacks)):
+            callback_on_step_end_tensor_inputs = callback_on_step_end.tensor_inputs
+
+        image_size = int(self.transformer.config.sample_size)
+        height = image_size if height is None else int(height)
+        width = image_size if width is None else int(width)
+        if num_inference_steps is None:
+            num_inference_steps = int(self._sampling_default("default_num_inference_steps", 50))
+        if guidance_scale is None:
+            guidance_scale = float(self._sampling_default("guidance_scale", 2.4))
+        if guidance_interval_min is None:
+            guidance_interval_min = float(self._sampling_default("guidance_interval_min", 0.125))
+        if guidance_interval_max is None:
+            guidance_interval_max = float(self._sampling_default("guidance_interval_max", 0.9))
+        if noise_scale is None:
+            noise_scale = float(self._sampling_default("noise_scale", 1.0))
+        if t_eps is None:
+            t_eps = float(self._sampling_default("t_eps", 0.05))
+
+        # 1. Check inputs.
+        self.check_inputs(
+            class_labels,
+            height,
+            width,
+            num_inference_steps,
+            guidance_scale,
+            guidance_interval_min,
+            guidance_interval_max,
+            noise_scale,
+            t_eps,
+            output_type,
+            callback_steps,
+            callback_on_step_end_tensor_inputs,
+        )
+
+        self._guidance_scale = float(guidance_scale)
+        self._interrupt = False
+
+        # 2. Define call parameters.
+        class_label_ids = self._normalize_class_labels(class_labels)
+        batch_size = len(class_label_ids)
+        device = self._execution_device
+        null_class = int(self.transformer.config.num_classes)
+        class_labels_t = torch.tensor(class_label_ids, device=device, dtype=torch.long)
+        if torch.any(class_labels_t < 0) or torch.any(class_labels_t > null_class):
+            raise ValueError(f"Class ids must lie in [0, {null_class}], got {class_label_ids}.")
+        class_null = torch.full_like(class_labels_t, null_class)
+
+        # 3. Encode input condition (class ids are already the condition; nothing else is encoded).
+
+        # 4. Prepare timesteps.
+        scheduled_timesteps = self._prepare_scheduler(num_inference_steps, device, timesteps, sigmas)
+
+        # 5. Prepare latent variables (pixel-space noise; there is no VAE).
+        latents = self.prepare_latents(
+            batch_size,
+            int(self.transformer.config.in_channels),
+            height,
+            width,
+            torch.float32,
+            device,
+            generator,
+            latents,
+            noise_scale,
+        ).float()
+
+        # 6. Prepare extra step kwargs.
+        extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
+        num_warmup_steps = len(scheduled_timesteps) - num_inference_steps * self.scheduler.order
+        self._num_timesteps = len(scheduled_timesteps)
+
+        # 7. Denoising loop.
+        with self.progress_bar(total=num_inference_steps) as progress_bar:
+            for i, timestep in enumerate(scheduled_timesteps):
+                if self.interrupt:
+                    continue
+
+                flow_time = float(timestep.to(dtype=torch.float32).reshape(-1)[0])
+                x0 = self._guided_x0(
+                    latents,
+                    flow_time,
+                    class_labels_t,
+                    class_null,
+                    self.guidance_scale,
+                    guidance_interval_min,
+                    guidance_interval_max,
+                )
+                latents = self.scheduler.step(
+                    x0, timestep, latents, t_eps=t_eps, **extra_step_kwargs, return_dict=False
+                )[0]
+
+                if callback_on_step_end is not None:
+                    callback_kwargs = {key: locals()[key] for key in callback_on_step_end_tensor_inputs}
+                    callback_outputs = callback_on_step_end(self, i, timestep, callback_kwargs)
+                    latents = callback_outputs.pop("latents", latents)
+
+                if i == len(scheduled_timesteps) - 1 or ((i + 1) > num_warmup_steps and (i + 1) % self.scheduler.order == 0):
+                    progress_bar.update()
+                    if callback is not None and i % callback_steps == 0:
+                        callback(i, timestep, latents)
+
+                if XLA_AVAILABLE:
+                    xm.mark_step()
+
+        images = self._images_from_latents(latents, output_type)
+        self.maybe_free_model_hooks()
+
+        if not return_dict:
+            return (images,)
+        return ImagePipelineOutput(images=images)

PixelDiT2-H-16-256/scheduler/scheduler_config.json

@@ -0,0 +1,8 @@
+{
+  "_class_name": "PixelDiT2Scheduler",
+  "_diffusers_version": "0.39.0",
+  "num_train_timesteps": 1000,
+  "prediction_type": "sample",
+  "solver": "heun",
+  "t_eps": 0.05
+}

PixelDiT2-H-16-256/scheduler/scheduling_pixeldit2.py

@@ -0,0 +1,216 @@
+"""PixelDiT2 flow scheduler.
+
+Time runs from 0 (noise) to 1 (data). The transformer predicts the clean sample x0.
+Velocity is `(x0 - x) / max(1 - t, t_eps)`. Heun is used on every interval except the
+last, which is Euler. This matches `GroundedHeunSampler` rather than the sigma grid
+inside `FlowMatchHeunDiscreteScheduler`.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Optional, Union
+
+import torch
+from diffusers.configuration_utils import ConfigMixin, register_to_config
+from diffusers.schedulers.scheduling_utils import SchedulerMixin
+from diffusers.utils import BaseOutput
+
+
+@dataclass
+class PixelDiT2SchedulerOutput(BaseOutput):
+    prev_sample: torch.Tensor
+
+
+class PixelDiT2Scheduler(SchedulerMixin, ConfigMixin):
+    """Uniform flow-time scheduler for PixelDiT2.
+
+    `solver="heun"` evaluates the model twice on each interval except the last.
+    `solver="euler"` evaluates it once per interval on the same linspace.
+    """
+
+    config_name = "scheduler_config.json"
+    _compatibles: list[str] = []
+    order = 2
+
+    @register_to_config
+    def __init__(
+        self,
+        num_train_timesteps: int = 1000,
+        solver: str = "heun",
+        prediction_type: str = "sample",
+        t_eps: float = 0.05,
+    ):
+        if solver not in {"heun", "euler"}:
+            raise ValueError(f'`solver` must be "heun" or "euler", got {solver}.')
+        if prediction_type not in {"sample", "velocity"}:
+            raise ValueError(f'`prediction_type` must be "sample" or "velocity", got {prediction_type}.')
+        if num_train_timesteps < 1:
+            raise ValueError(f"`num_train_timesteps` must be >= 1, got {num_train_timesteps}.")
+        if t_eps <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {t_eps}.")
+
+        self.timesteps: Optional[torch.Tensor] = None
+        self.schedule: Optional[torch.Tensor] = None
+        self.num_inference_steps: Optional[int] = None
+        self._step_plan: list[dict] = []
+        self._step_index: Optional[int] = None
+        self._begin_index: Optional[int] = None
+        self._heun_sample: Optional[torch.Tensor] = None
+        self._heun_velocity: Optional[torch.Tensor] = None
+
+    @property
+    def init_noise_sigma(self) -> float:
+        return 1.0
+
+    @property
+    def step_index(self) -> Optional[int]:
+        return self._step_index
+
+    @property
+    def begin_index(self) -> Optional[int]:
+        return self._begin_index
+
+    def set_begin_index(self, begin_index: int = 0) -> None:
+        self._begin_index = int(begin_index)
+
+    def set_timesteps(
+        self,
+        num_inference_steps: int,
+        device: Union[str, torch.device, None] = None,
+        **kwargs,
+    ) -> None:
+        """Install `linspace(0, 1, steps + 1)` and the Heun or Euler step plan.
+
+        Args:
+            num_inference_steps (`int`):
+                Number of intervals from noise to data.
+            device (`str` or `torch.device`, *optional*):
+                Device of `timesteps`.
+        """
+        if kwargs:
+            raise ValueError(f"PixelDiT2Scheduler.set_timesteps does not accept {sorted(kwargs)}.")
+        if num_inference_steps < 1:
+            raise ValueError(f"`num_inference_steps` must be >= 1, got {num_inference_steps}.")
+
+        knots = torch.linspace(0.0, 1.0, num_inference_steps + 1, device=device, dtype=torch.float32)
+        plan: list[dict] = []
+        use_heun = self.config.solver == "heun" and num_inference_steps > 1
+        if use_heun:
+            for index in range(num_inference_steps - 1):
+                current = float(knots[index])
+                nxt = float(knots[index + 1])
+                plan.append({"kind": "heun_predict", "t_model": current, "t_cur": current, "t_next": nxt})
+                plan.append({"kind": "heun_correct", "t_model": nxt, "t_cur": current, "t_next": nxt})
+            last = num_inference_steps - 1
+            plan.append(
+                {
+                    "kind": "euler",
+                    "t_model": float(knots[last]),
+                    "t_cur": float(knots[last]),
+                    "t_next": float(knots[last + 1]),
+                }
+            )
+        else:
+            for index in range(num_inference_steps):
+                plan.append(
+                    {
+                        "kind": "euler",
+                        "t_model": float(knots[index]),
+                        "t_cur": float(knots[index]),
+                        "t_next": float(knots[index + 1]),
+                    }
+                )
+
+        self.num_inference_steps = int(num_inference_steps)
+        self.schedule = knots
+        self._step_plan = plan
+        self.timesteps = torch.tensor([item["t_model"] for item in plan], device=device, dtype=torch.float32)
+        self.order = 2 if use_heun else 1
+        self._step_index = 0 if self._begin_index is None else self._begin_index
+        self._heun_sample = None
+        self._heun_velocity = None
+
+    def scale_model_input(self, sample: torch.Tensor, timestep: Union[float, torch.Tensor]) -> torch.Tensor:
+        del timestep
+        return sample
+
+    def _velocity(
+        self,
+        sample: torch.Tensor,
+        model_output: torch.Tensor,
+        flow_time: float,
+        t_eps: float,
+    ) -> torch.Tensor:
+        if self.config.prediction_type == "velocity":
+            return model_output.float()
+        denominator = max(1.0 - float(flow_time), float(t_eps))
+        return (model_output.float() - sample.float()) / denominator
+
+    def step(
+        self,
+        model_output: torch.Tensor,
+        timestep: Union[float, torch.Tensor],
+        sample: torch.Tensor,
+        return_dict: bool = True,
+        generator: Optional[Union[torch.Generator, list[torch.Generator]]] = None,
+        t_eps: Optional[float] = None,
+    ) -> Union[PixelDiT2SchedulerOutput, tuple]:
+        """Advance one planned step.
+
+        Args:
+            model_output (`torch.Tensor`):
+                x0 prediction when `prediction_type` is `"sample"`.
+            timestep (`float` or `torch.Tensor`):
+                Flow time at which `model_output` was evaluated. The step plan is the
+                source of the update; this value is checked against that plan.
+            sample (`torch.Tensor`):
+                Current sample. For a Heun corrector this is the predictor state.
+            return_dict (`bool`, defaults to `True`):
+                Return [`PixelDiT2SchedulerOutput`] when `True`.
+            generator (`torch.Generator`, *optional*):
+                Unused. The integrator is deterministic.
+            t_eps (`float`, *optional*):
+                Floor on `1 - t`. Defaults to the scheduler config.
+
+        Returns:
+            [`PixelDiT2SchedulerOutput`] or `tuple`:
+                The updated sample in float32.
+        """
+        del generator
+        if not self._step_plan or self._step_index is None:
+            raise ValueError("Call `set_timesteps` before `step`.")
+        if self._step_index >= len(self._step_plan):
+            raise ValueError("PixelDiT2Scheduler has already reached the final timestep.")
+
+        plan = self._step_plan[self._step_index]
+        sample = sample.to(dtype=torch.float32)
+        floor = float(self.config.t_eps if t_eps is None else t_eps)
+        if floor <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {floor}.")
+        velocity = self._velocity(sample, model_output, plan["t_model"], floor)
+        dt = plan["t_next"] - plan["t_cur"]
+
+        kind = plan["kind"]
+        if kind == "heun_predict":
+            self._heun_sample = sample
+            self._heun_velocity = velocity
+            prev_sample = sample + dt * velocity
+        elif kind == "heun_correct":
+            if self._heun_sample is None or self._heun_velocity is None:
+                raise RuntimeError("Heun corrector was called before its predictor.")
+            prev_sample = self._heun_sample + dt * 0.5 * (self._heun_velocity + velocity)
+            self._heun_sample = None
+            self._heun_velocity = None
+        else:
+            prev_sample = sample + dt * velocity
+            self._heun_sample = None
+            self._heun_velocity = None
+
+        self._step_index += 1
+        if not return_dict:
+            return (prev_sample,)
+        return PixelDiT2SchedulerOutput(prev_sample=prev_sample)
+
+    def __len__(self) -> int:
+        return int(self.config.num_train_timesteps)

PixelDiT2-H-16-256/transformer/config.json

@@ -0,0 +1,26 @@
+{
+  "_class_name": "PixelDiT2Transformer2DModel",
+  "_diffusers_version": "0.39.0",
+  "attn_dropout": 0.0,
+  "default_num_inference_steps": 50,
+  "depth": 32,
+  "grounding_encoder": "dinov3_vits16",
+  "grounding_proj_blocks": 1,
+  "grounding_proj_heads": 16,
+  "guidance_interval_max": 0.9,
+  "guidance_interval_min": 0.125,
+  "guidance_scale": 2.4,
+  "hidden_size": 1280,
+  "in_channels": 3,
+  "in_context_len": 32,
+  "in_context_start": 8,
+  "mlp_hidden_multiple": 0,
+  "noise_scale": 1.0,
+  "num_classes": 1000,
+  "num_heads": 16,
+  "patch_size": 16,
+  "proj_dropout": 0.2,
+  "sample_size": 256,
+  "single_silu_cond": false,
+  "t_eps": 0.05
+}

PixelDiT2-H-16-256/transformer/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5d43740198ec13c90c0c678701018c7d55c22b6e4d9ee4865143366ab34904d
+size 2015645984

PixelDiT2-H-16-256/transformer/transformer_pixeldit2.py

@@ -0,0 +1,1094 @@
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""PixelDiT2 denoiser as a diffusers transformer.
+
+PixelDiT2 predicts the clean image x0 directly in RGB pixel space. A frozen
+DINOv3 encoder grounds every patch token; the unconditional branch of
+classifier-free guidance skips that encoder (`skip_grounding=True`) and uses
+the null class id `num_classes`.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+import timm
+import torch
+import torch.nn.functional as F
+from diffusers.configuration_utils import ConfigMixin, register_to_config
+from diffusers.models.modeling_outputs import Transformer2DModelOutput
+from diffusers.models.modeling_utils import ModelMixin
+from torch import nn
+
+
+@dataclass(frozen=True)
+class EncoderSpec:
+    r"""
+    Frozen grounding encoder used by PixelDiT2.
+
+    Args:
+        timm_name (`str`):
+            `timm` model id, including the pretrained tag when one exists.
+        feature_dim (`int`):
+            Channel count of the encoder patch tokens.
+        patch_size (`int`):
+            Encoder patch size in pixels.
+    """
+
+    timm_name: str
+    feature_dim: int
+    patch_size: int
+
+
+GROUNDING_ENCODERS: dict[str, EncoderSpec] = {
+    "dinov3_vits16": EncoderSpec("vit_small_patch16_dinov3_qkvb.lvd1689m", 384, 16),
+    "dinov3_vitb16": EncoderSpec("vit_base_patch16_dinov3_qkvb.lvd1689m", 768, 16),
+    "dinov3_vitl16": EncoderSpec("vit_large_patch16_dinov3_qkvb.lvd1689m", 1024, 16),
+}
+
+IMAGENET_MEAN = (0.485, 0.456, 0.406)
+IMAGENET_STD = (0.229, 0.224, 0.225)
+
+
+def apply_adaln(hidden_states: torch.Tensor, shift: torch.Tensor, scale: torch.Tensor) -> torch.Tensor:
+    r"""
+    Apply adaptive layer-norm modulation.
+
+    Args:
+        hidden_states (`torch.Tensor`):
+            Token states of shape `(batch, tokens, hidden)`.
+        shift (`torch.Tensor`):
+            Shift term broadcastable to `hidden_states`.
+        scale (`torch.Tensor`):
+            Scale term broadcastable to `hidden_states`. The applied factor is `1 + scale`.
+
+    Returns:
+        `torch.Tensor`: Modulated tokens, same shape as `hidden_states`.
+    """
+    return hidden_states * (1 + scale) + shift
+
+
+def precompute_freqs_cis_2d(
+    dim: int,
+    height: int,
+    width: int,
+    theta: float = 10000.0,
+    scale: float = 16.0,
+) -> torch.Tensor:
+    r"""
+    Build 2D rotary frequencies for a patch grid.
+
+    Positions are `linspace(0, scale, length)` on each axis, matching the PixelDiT2 training code.
+    A different grid size recomputes the table, which is how non-native resolutions are handled.
+
+    Args:
+        dim (`int`):
+            Head dimension. Must be divisible by 4.
+        height (`int`):
+            Grid height in patches.
+        width (`int`):
+            Grid width in patches.
+        theta (`float`, defaults to 10000.0):
+            RoPE base period.
+        scale (`float`, defaults to 16.0):
+            Coordinate of the last patch on each axis.
+
+    Returns:
+        `torch.Tensor`: Complex frequencies of shape `(height * width, dim / 2)`.
+    """
+    x_pos = torch.linspace(0, scale, width)
+    y_pos = torch.linspace(0, scale, height)
+    y_pos, x_pos = torch.meshgrid(y_pos, x_pos, indexing="ij")
+    y_pos = y_pos.reshape(-1)
+    x_pos = x_pos.reshape(-1)
+    freqs = 1.0 / (theta ** (torch.arange(0, dim, 4)[: (dim // 4)].float() / dim))
+    x_freqs = torch.outer(x_pos, freqs).float()
+    y_freqs = torch.outer(y_pos, freqs).float()
+    x_cis = torch.polar(torch.ones_like(x_freqs), x_freqs)
+    y_cis = torch.polar(torch.ones_like(y_freqs), y_freqs)
+    freqs_cis = torch.cat([x_cis.unsqueeze(dim=-1), y_cis.unsqueeze(dim=-1)], dim=-1)
+    return freqs_cis.reshape(height * width, -1)
+
+
+def apply_rotary_emb(
+    query: torch.Tensor,
+    key: torch.Tensor,
+    freqs_cis: torch.Tensor,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    r"""
+    Apply rotary embeddings to query and key.
+
+    Args:
+        query (`torch.Tensor`):
+            Queries of shape `(batch, tokens, heads, head_dim)`.
+        key (`torch.Tensor`):
+            Keys of shape `(batch, tokens, heads, head_dim)`.
+        freqs_cis (`torch.Tensor`):
+            Complex frequencies of shape `(tokens, head_dim / 2)`.
+
+    Returns:
+        `tuple[torch.Tensor, torch.Tensor]`: Rotated query and key, same shapes as the inputs.
+    """
+    freqs_cis = freqs_cis[None, :, None, :]
+    query_complex = torch.view_as_complex(query.float().reshape(*query.shape[:-1], -1, 2))
+    key_complex = torch.view_as_complex(key.float().reshape(*key.shape[:-1], -1, 2))
+    query_out = torch.view_as_real(query_complex * freqs_cis).flatten(3)
+    key_out = torch.view_as_real(key_complex * freqs_cis).flatten(3)
+    return query_out.type_as(query), key_out.type_as(key)
+
+
+class TimestepConditioner(nn.Module):
+    r"""
+    Sinusoidal timestep embedding followed by a two-layer MLP.
+
+    PixelDiT2 uses `max_period=10`, not the more common 10000.
+
+    Args:
+        hidden_size (`int`):
+            Width of the MLP output.
+        frequency_embedding_size (`int`, defaults to 256):
+            Width of the sinusoidal embedding.
+    """
+
+    def __init__(self, hidden_size: int, frequency_embedding_size: int = 256):
+        super().__init__()
+        self.mlp = nn.Sequential(
+            nn.Linear(frequency_embedding_size, hidden_size, bias=True),
+            nn.SiLU(),
+            nn.Linear(hidden_size, hidden_size, bias=True),
+        )
+        self.frequency_embedding_size = frequency_embedding_size
+
+    @staticmethod
+    def timestep_embedding(timesteps: torch.Tensor, dim: int, max_period: int = 10) -> torch.Tensor:
+        r"""
+        Build a sinusoidal embedding of scalar timesteps.
+
+        Args:
+            timesteps (`torch.Tensor`):
+                Timesteps of shape `(batch,)`.
+            dim (`int`):
+                Embedding width.
+            max_period (`int`, defaults to 10):
+                Maximum period of the sinusoid.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, dim)`.
+        """
+        half = dim // 2
+        freqs = torch.exp(
+            -math.log(max_period) * torch.arange(start=0, end=half, dtype=torch.float32, device=timesteps.device) / half
+        )
+        args = timesteps[..., None].float() * freqs[None, ...]
+        embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)
+        if dim % 2:
+            embedding = torch.cat([embedding, torch.zeros_like(embedding[:, :1])], dim=-1)
+        return embedding
+
+    def forward(self, timesteps: torch.Tensor) -> torch.Tensor:
+        r"""
+        Embed timesteps.
+
+        Args:
+            timesteps (`torch.Tensor`):
+                Flow time `t` of shape `(batch,)`, in `[0, 1]`.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, hidden_size)`.
+        """
+        timestep_freq = self.timestep_embedding(timesteps, self.frequency_embedding_size)
+        mlp_dtype = self.mlp[0].weight.dtype
+        if timestep_freq.dtype != mlp_dtype:
+            timestep_freq = timestep_freq.to(mlp_dtype)
+        return self.mlp(timestep_freq)
+
+
+class ClassEmbedder(nn.Module):
+    r"""
+    Embedding table for ImageNet class ids, including the null class.
+
+    Args:
+        num_classes (`int`):
+            Number of rows. PixelDiT2 uses `num_classes + 1` so index `num_classes` is unconditional.
+        hidden_size (`int`):
+            Embedding width.
+    """
+
+    def __init__(self, num_classes: int, hidden_size: int):
+        super().__init__()
+        self.embedding_table = nn.Embedding(num_classes, hidden_size)
+        self.num_classes = num_classes
+
+    def forward(self, labels: torch.Tensor) -> torch.Tensor:
+        r"""
+        Look up class embeddings.
+
+        Args:
+            labels (`torch.Tensor`):
+                Integer class ids of shape `(batch,)`.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, hidden_size)`.
+        """
+        return self.embedding_table(labels)
+
+
+class RMSNorm(nn.Module):
+    r"""
+    Root-mean-square normalization.
+
+    Args:
+        hidden_size (`int`):
+            Channel width.
+        eps (`float`, defaults to 1e-6):
+            Numerical stability term.
+    """
+
+    def __init__(self, hidden_size: int, eps: float = 1e-6):
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(hidden_size))
+        self.variance_epsilon = eps
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Normalize the last dimension.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Input of shape `(..., hidden_size)`.
+
+        Returns:
+            `torch.Tensor`: Normalized tensor, same shape as the input.
+        """
+        input_dtype = hidden_states.dtype
+        hidden_states = hidden_states.to(torch.float32)
+        variance = hidden_states.pow(2).mean(-1, keepdim=True)
+        hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
+        return self.weight * hidden_states.to(input_dtype)
+
+
+class FeedForward(nn.Module):
+    r"""
+    SwiGLU feed-forward block.
+
+    Args:
+        dim (`int`):
+            Input and output width.
+        hidden_dim (`int`):
+            Width before the SwiGLU 2/3 reduction. The inner width is `int(2 * hidden_dim / 3)`,
+            then rounded up to `hidden_multiple` when that value is positive.
+        drop (`float`, defaults to 0.0):
+            Dropout on the hidden activation. Disabled by `eval()`.
+        hidden_multiple (`int`, defaults to 0):
+            Round the inner width up to a multiple of this value. `0` disables rounding.
+            The 512px PixelDiT2 checkpoint uses 32.
+    """
+
+    def __init__(self, dim: int, hidden_dim: int, drop: float = 0.0, hidden_multiple: int = 0):
+        super().__init__()
+        hidden_dim = int(2 * hidden_dim / 3)
+        hidden_multiple = int(hidden_multiple)
+        if hidden_multiple > 0:
+            hidden_dim = ((hidden_dim + hidden_multiple - 1) // hidden_multiple) * hidden_multiple
+        self.w1 = nn.Linear(dim, hidden_dim, bias=False)
+        self.w3 = nn.Linear(dim, hidden_dim, bias=False)
+        self.w2 = nn.Linear(hidden_dim, dim, bias=False)
+        self.ffn_dropout = nn.Dropout(drop)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Apply SwiGLU.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(..., dim)`.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(..., dim)`.
+        """
+        hidden = F.silu(self.w1(hidden_states)) * self.w3(hidden_states)
+        return self.w2(self.ffn_dropout(hidden))
+
+
+class RotaryAttention(nn.Module):
+    r"""
+    Multi-head self-attention with QK RMSNorm and 2D rotary embeddings.
+
+    Args:
+        dim (`int`):
+            Token width. Must be divisible by `num_heads`.
+        num_heads (`int`, defaults to 8):
+            Number of attention heads.
+        qkv_bias (`bool`, defaults to `False`):
+            Whether the fused QKV projection has a bias. PixelDiT2 checkpoints use `False`.
+        qk_norm (`bool`, defaults to `True`):
+            Apply RMSNorm to query and key per head.
+        attn_drop (`float`, defaults to 0.0):
+            Attention dropout. The released models use 0.
+        proj_drop (`float`, defaults to 0.0):
+            Dropout after the output projection.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int = 8,
+        qkv_bias: bool = False,
+        qk_norm: bool = True,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+    ):
+        super().__init__()
+        if dim % num_heads != 0:
+            raise ValueError(f"dim={dim} must be divisible by num_heads={num_heads}.")
+        self.dim = dim
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.q_norm = RMSNorm(self.head_dim) if qk_norm else nn.Identity()
+        self.k_norm = RMSNorm(self.head_dim) if qk_norm else nn.Identity()
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop)
+
+    def forward(self, hidden_states: torch.Tensor, pos: torch.Tensor, mask: torch.Tensor | None = None) -> torch.Tensor:
+        r"""
+        Run self-attention.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, tokens, dim)`.
+            pos (`torch.Tensor`):
+                Complex RoPE table of shape `(tokens, head_dim / 2)`.
+            mask (`torch.Tensor`, *optional*):
+                Attention mask forwarded to scaled-dot-product attention. `None` attends to every token.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, dim)`.
+        """
+        batch, tokens, channels = hidden_states.shape
+        qkv = self.qkv(hidden_states).reshape(batch, tokens, 3, self.num_heads, channels // self.num_heads)
+        qkv = qkv.permute(2, 0, 1, 3, 4)
+        query, key, value = qkv[0], qkv[1], qkv[2]
+        query = self.q_norm(query)
+        key = self.k_norm(key)
+        query, key = apply_rotary_emb(query, key, freqs_cis=pos)
+        query = query.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2)
+        key = key.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2).contiguous()
+        value = value.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2).contiguous()
+        # The training forward always uses dropout_p=0 and ignores `attn_drop`.
+        hidden_states = F.scaled_dot_product_attention(query, key, value, attn_mask=mask, dropout_p=0.0)
+        hidden_states = hidden_states.transpose(1, 2).reshape(batch, tokens, channels)
+        hidden_states = self.proj(hidden_states)
+        hidden_states = self.proj_drop(hidden_states)
+        return hidden_states
+
+
+class PatchTokenEmbedder(nn.Module):
+    r"""
+    Linear embedding of flattened pixel patches.
+
+    Args:
+        in_channels (`int`):
+            Flattened patch width, `in_channels * patch_size ** 2` at the call site.
+        embed_dim (`int`):
+            Token width.
+        bias (`bool`, defaults to `True`):
+            Bias on the linear projection. PixelDiT2 uses `True`.
+    """
+
+    def __init__(self, in_channels: int, embed_dim: int, bias: bool = True):
+        super().__init__()
+        self.in_channels = in_channels
+        self.embed_dim = embed_dim
+        self.proj = nn.Linear(in_channels, embed_dim, bias=bias)
+        self.norm = nn.Identity()
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Embed patch tokens.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Flattened patches of shape `(batch, tokens, in_channels)`.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, embed_dim)`.
+        """
+        return self.norm(self.proj(hidden_states))
+
+
+class ModulatedDiTBlock(nn.Module):
+    r"""
+    AdaLN-Zero transformer block with rotary attention and SwiGLU.
+
+    Args:
+        hidden_size (`int`):
+            Token width.
+        num_heads (`int`):
+            Attention heads.
+        mlp_ratio (`float`, defaults to 4.0):
+            MLP expansion before the SwiGLU 2/3 reduction.
+        attn_drop (`float`, defaults to 0.0):
+            Attention dropout.
+        proj_drop (`float`, defaults to 0.0):
+            Dropout on the attention projection and the MLP.
+        mlp_hidden_multiple (`int`, defaults to 0):
+            Forwarded to [`FeedForward`].
+    """
+
+    def __init__(
+        self,
+        hidden_size: int,
+        num_heads: int,
+        mlp_ratio: float = 4.0,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+        mlp_hidden_multiple: int = 0,
+    ):
+        super().__init__()
+        self.norm1 = RMSNorm(hidden_size, eps=1e-6)
+        self.attn = RotaryAttention(
+            hidden_size,
+            num_heads=num_heads,
+            qkv_bias=False,
+            attn_drop=attn_drop,
+            proj_drop=proj_drop,
+        )
+        self.norm2 = RMSNorm(hidden_size, eps=1e-6)
+        self.mlp = FeedForward(
+            hidden_size,
+            int(hidden_size * mlp_ratio),
+            drop=proj_drop,
+            hidden_multiple=mlp_hidden_multiple,
+        )
+        self.adaLN_modulation = nn.Sequential(
+            nn.SiLU(),
+            nn.Linear(hidden_size, 6 * hidden_size, bias=True),
+        )
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        conditioning: torch.Tensor,
+        pos: torch.Tensor,
+        mask: torch.Tensor | None = None,
+    ) -> torch.Tensor:
+        r"""
+        Run one modulated block.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, tokens, hidden)`.
+            conditioning (`torch.Tensor`):
+                AdaLN conditioning of shape `(batch, tokens, hidden)` or `(batch, 1, hidden)`.
+            pos (`torch.Tensor`):
+                RoPE table aligned with `tokens`.
+            mask (`torch.Tensor`, *optional*):
+                Attention mask. `None` uses full attention.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, hidden)`.
+        """
+        shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp = self.adaLN_modulation(conditioning).chunk(6, dim=-1)
+        hidden_states = hidden_states + gate_msa * self.attn(
+            apply_adaln(self.norm1(hidden_states), shift_msa, scale_msa), pos, mask=mask
+        )
+        hidden_states = hidden_states + gate_mlp * self.mlp(apply_adaln(self.norm2(hidden_states), shift_mlp, scale_mlp))
+        return hidden_states
+
+
+class PatchFinalLayer(nn.Module):
+    r"""
+    AdaLN-modulated projection from tokens back to pixel patches.
+
+    Args:
+        hidden_size (`int`):
+            Token width.
+        patch_size (`int`):
+            Spatial patch size.
+        out_channels (`int`):
+            Image channels, 3 for RGB.
+    """
+
+    def __init__(self, hidden_size: int, patch_size: int, out_channels: int):
+        super().__init__()
+        self.norm = RMSNorm(hidden_size, eps=1e-6)
+        self.linear = nn.Linear(hidden_size, patch_size * patch_size * out_channels, bias=True)
+        self.adaLN_modulation = nn.Sequential(
+            nn.SiLU(),
+            nn.Linear(hidden_size, 2 * hidden_size, bias=True),
+        )
+
+    def forward(self, hidden_states: torch.Tensor, conditioning: torch.Tensor) -> torch.Tensor:
+        r"""
+        Project tokens to flattened patches.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, patches, hidden)`.
+            conditioning (`torch.Tensor`):
+                Conditioning of shape `(batch, patches, hidden)`. A 2D tensor is unsqueezed.
+
+        Returns:
+            `torch.Tensor`: Patch pixels of shape `(batch, patches, patch_size ** 2 * out_channels)`.
+        """
+        shift, scale = self.adaLN_modulation(conditioning).chunk(2, dim=-1)
+        if shift.dim() == 2:
+            shift = shift.unsqueeze(1)
+            scale = scale.unsqueeze(1)
+        return self.linear(apply_adaln(self.norm(hidden_states), shift, scale))
+
+
+class TimestepProjection(nn.Module):
+    r"""
+    Projects DINOv3 tokens into the denoiser width with timestep modulation.
+
+    Args:
+        in_dim (`int`):
+            Encoder feature width.
+        out_dim (`int`):
+            Denoiser hidden size.
+        hidden_size (`int`):
+            Width of the internal transformer blocks. PixelDiT2 uses the denoiser width.
+        num_blocks (`int`, defaults to 1):
+            Number of modulated blocks. Released checkpoints use 1.
+        num_heads (`int`, defaults to 16):
+            Heads inside those blocks. `hidden_size` must be divisible by this value.
+        mlp_ratio (`float`, defaults to 4.0):
+            MLP expansion of the internal blocks.
+    """
+
+    def __init__(
+        self,
+        in_dim: int,
+        out_dim: int,
+        hidden_size: int,
+        num_blocks: int = 1,
+        num_heads: int = 16,
+        mlp_ratio: float = 4.0,
+    ):
+        super().__init__()
+        if hidden_size % num_heads != 0:
+            raise ValueError(f"TimestepProjection hidden_size={hidden_size} must be divisible by num_heads={num_heads}.")
+        self.head_dim = hidden_size // num_heads
+        self.input_proj = nn.Linear(in_dim, hidden_size, bias=True)
+        self.t_embedder = TimestepConditioner(hidden_size)
+        self.blocks = nn.ModuleList(
+            [ModulatedDiTBlock(hidden_size, num_heads, mlp_ratio=mlp_ratio) for _ in range(num_blocks)]
+        )
+        self.output_proj = nn.Linear(hidden_size, out_dim, bias=True)
+        for block in self.blocks:
+            nn.init.zeros_(block.adaLN_modulation[-1].weight)
+            nn.init.zeros_(block.adaLN_modulation[-1].bias)
+        self._pos_cache: dict[tuple[int, int], torch.Tensor] = {}
+
+    def fetch_pos(self, height: int, width: int, device: torch.device) -> torch.Tensor:
+        r"""
+        Return cached RoPE frequencies for a patch grid.
+
+        Args:
+            height (`int`):
+                Grid height in patches.
+            width (`int`):
+                Grid width in patches.
+            device (`torch.device`):
+                Device of the returned table.
+
+        Returns:
+            `torch.Tensor`: Complex frequencies of shape `(height * width, head_dim / 2)`.
+        """
+        key = (height, width)
+        if key not in self._pos_cache:
+            self._pos_cache[key] = precompute_freqs_cis_2d(self.head_dim, height, width)
+        return self._pos_cache[key].to(device)
+
+    def forward(self, tokens: torch.Tensor, timesteps: torch.Tensor, grid_hw: tuple[int, int]) -> torch.Tensor:
+        r"""
+        Project grounding tokens.
+
+        Args:
+            tokens (`torch.Tensor`):
+                Encoder tokens of shape `(batch, patches, in_dim)`.
+            timesteps (`torch.Tensor`):
+                Flow time of shape `(batch,)`.
+            grid_hw (`tuple[int, int]`):
+                `(height, width)` of the token grid in patches.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, patches, out_dim)`.
+        """
+        hidden_states = self.input_proj(tokens)
+        conditioning = self.t_embedder(timesteps).unsqueeze(1).to(dtype=hidden_states.dtype)
+        pos = self.fetch_pos(grid_hw[0], grid_hw[1], hidden_states.device)
+        for block in self.blocks:
+            hidden_states = block(hidden_states, conditioning, pos)
+        return self.output_proj(hidden_states)
+
+
+class GroundingConditioner(nn.Module):
+    r"""
+    Frozen DINOv3 encoder plus the trainable projection `P_g`.
+
+    The encoder is built with `pretrained=False`. Conversion copies either the
+    checkpoint's encoder tensors or a `timm` pretrained checkpoint into this module
+    so inference does not download weights.
+
+    Args:
+        encoder_name (`str`):
+            Key of [`GROUNDING_ENCODERS`], for example `dinov3_vits16`.
+        hidden_size (`int`):
+            Denoiser width.
+        target_patch_size (`int`):
+            Denoiser patch size. The encoder grid is resized to this patch grid.
+        proj_num_blocks (`int`, defaults to 1):
+            Blocks inside the projection.
+        proj_num_heads (`int`, defaults to 16):
+            Heads inside the projection.
+    """
+
+    def __init__(
+        self,
+        encoder_name: str,
+        hidden_size: int,
+        target_patch_size: int,
+        proj_num_blocks: int = 1,
+        proj_num_heads: int = 16,
+    ):
+        super().__init__()
+        if encoder_name not in GROUNDING_ENCODERS:
+            raise ValueError(f"Unknown grounding encoder {encoder_name!r}. Available: {sorted(GROUNDING_ENCODERS)}.")
+        self.spec = GROUNDING_ENCODERS[encoder_name]
+        self.encoder_name = encoder_name
+        self.target_patch_size = int(target_patch_size)
+        self.encoder = timm.create_model(self.spec.timm_name, pretrained=False, dynamic_img_size=True)
+        self.encoder.requires_grad_(False)
+        self.dino_proj = TimestepProjection(
+            in_dim=self.spec.feature_dim,
+            out_dim=hidden_size,
+            hidden_size=hidden_size,
+            num_blocks=proj_num_blocks,
+            num_heads=proj_num_heads,
+        )
+        self.register_buffer("image_mean", torch.tensor(IMAGENET_MEAN).view(1, 3, 1, 1), persistent=False)
+        self.register_buffer("image_std", torch.tensor(IMAGENET_STD).view(1, 3, 1, 1), persistent=False)
+
+    def train(self, mode: bool = True) -> "GroundingConditioner":
+        r"""
+        Keep the frozen encoder in eval mode.
+
+        Args:
+            mode (`bool`, defaults to `True`):
+                Whether the projection should train.
+
+        Returns:
+            `GroundingConditioner`: This module.
+        """
+        super().train(mode)
+        self.encoder.eval()
+        return self
+
+    def preprocess(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, tuple[int, int]]:
+        r"""
+        Map pixels from `[-1, 1]` to the encoder's ImageNet normalization.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)` in model range `[-1, 1]`.
+
+        Returns:
+            `tuple[torch.Tensor, tuple[int, int]]`:
+                Normalized encoder input and the denoiser patch grid `(grid_h, grid_w)`.
+        """
+        height, width = hidden_states.shape[-2:]
+        grid_hw = (height // self.target_patch_size, width // self.target_patch_size)
+        encoder_hw = (grid_hw[0] * self.spec.patch_size, grid_hw[1] * self.spec.patch_size)
+        hidden_states = (hidden_states + 1.0) * 0.5
+        hidden_states = (hidden_states - self.image_mean.to(hidden_states.dtype)) / self.image_std.to(hidden_states.dtype)
+        if encoder_hw != (height, width):
+            hidden_states = F.interpolate(hidden_states, size=encoder_hw, mode="bicubic", align_corners=False)
+        return hidden_states, grid_hw
+
+    def _zero_key_bias(self) -> None:
+        """Keep DINOv3 key bias at zero.
+
+        timm's qkvb attention stores a zero key bias as a non-persistent buffer, so it
+        is absent from the checkpoint. `from_pretrained` builds the encoder on the meta
+        device and materializes missing buffers as uninitialized memory. A non-zero key
+        bias makes attention NaN.
+        """
+        for module in self.encoder.modules():
+            bias = getattr(module, "k_bias", None)
+            if isinstance(bias, torch.Tensor) and bias.device.type != "meta":
+                bias.zero_()
+
+    def encode(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, tuple[int, int]]:
+        r"""
+        Encode an image into patch tokens aligned with the denoiser grid.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)` in `[-1, 1]`.
+
+        Returns:
+            `tuple[torch.Tensor, tuple[int, int]]`:
+                Tokens `(batch, grid_h * grid_w, feature_dim)` and the grid size.
+        """
+        self._zero_key_bias()
+        encoder_input, grid_hw = self.preprocess(hidden_states)
+        with torch.no_grad():
+            features = self.encoder.forward_features(encoder_input)
+        prefix = int(getattr(self.encoder, "num_prefix_tokens", 0))
+        tokens = features[:, prefix:, :]
+        encoder_grid = (
+            encoder_input.shape[-2] // self.spec.patch_size,
+            encoder_input.shape[-1] // self.spec.patch_size,
+        )
+        if encoder_grid != grid_hw:
+            batch, _, channels = tokens.shape
+            tokens = tokens.transpose(1, 2).reshape(batch, channels, *encoder_grid)
+            tokens = F.interpolate(tokens, size=grid_hw, mode="bilinear", align_corners=False)
+            tokens = tokens.flatten(2).transpose(1, 2)
+        return tokens, grid_hw
+
+    def forward(self, hidden_states: torch.Tensor, timesteps: torch.Tensor) -> torch.Tensor:
+        r"""
+        Encode and project grounding tokens.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)`.
+            timesteps (`torch.Tensor`):
+                Flow time of shape `(batch,)`.
+
+        Returns:
+            `torch.Tensor`: Grounding of shape `(batch, patches, hidden_size)`.
+        """
+        tokens, grid_hw = self.encode(hidden_states)
+        return self.dino_proj(tokens.to(dtype=self.dino_proj.input_proj.weight.dtype), timesteps, grid_hw)
+
+
+class PixelDiT2Transformer2DModel(ModelMixin, ConfigMixin):
+    r"""
+    PixelDiT2 class-conditional denoiser.
+
+    The module names match the training checkpoint (`t_embedder`, `y_embedder`, `s_embedder`,
+    `patch_blocks`, `final_layer`, `dino_conditioner`) so an EMA state dict loads without renaming.
+
+    Args:
+        sample_size (`int`, defaults to 256):
+            Native square resolution in pixels. Other multiples of `patch_size` are valid at inference
+            because rotary positions are recomputed for the requested grid.
+        in_channels (`int`, defaults to 3):
+            Image channels.
+        patch_size (`int`, defaults to 16):
+            Pixel patch size. `height` and `width` must be divisible by this value.
+        hidden_size (`int`, defaults to 1280):
+            Token width of PixelDiT2-H.
+        num_heads (`int`, defaults to 16):
+            Attention heads.
+        depth (`int`, defaults to 32):
+            Number of patch blocks.
+        num_classes (`int`, defaults to 1000):
+            ImageNet classes. The embedding table has `num_classes + 1` rows; the last row is the null class.
+        in_context_len (`int`, defaults to 32):
+            Number of in-context class tokens inserted at `in_context_start`.
+        in_context_start (`int`, defaults to 8):
+            Block index at which the in-context tokens are inserted. Must lie in `[0, depth)`.
+        attn_dropout (`float`, defaults to 0.0):
+            Attention dropout on the middle half of the blocks.
+        proj_dropout (`float`, defaults to 0.2):
+            Projection dropout on the middle half of the blocks. Inactive in `eval()`.
+        mlp_hidden_multiple (`int`, defaults to 0):
+            SwiGLU width rounding. `0` for 256px, `32` for 512px.
+        grounding_encoder (`str`, defaults to `"dinov3_vits16"`):
+            Frozen encoder key. 256px uses `dinov3_vits16`; 512px uses `dinov3_vitb16`.
+        grounding_proj_blocks (`int`, defaults to 1):
+            Blocks in the grounding projection.
+        grounding_proj_heads (`int`, defaults to 16):
+            Heads in the grounding projection.
+        single_silu_cond (`bool`, defaults to `False`):
+            When `False` (256px), SiLU is applied to the summed condition. When `True` (512px), it is not.
+        guidance_scale (`float`, defaults to 2.4):
+            Recommended classifier-free guidance scale. Read by the pipeline when the caller omits it.
+        guidance_interval_min (`float`, defaults to 0.125):
+            Lower flow-time bound of the guidance interval, exclusive.
+        guidance_interval_max (`float`, defaults to 0.9):
+            Upper flow-time bound of the guidance interval, exclusive.
+        noise_scale (`float`, defaults to 1.0):
+            Standard deviation of the initial Gaussian. 1.0 at 256px and 2.0 at 512px.
+        t_eps (`float`, defaults to 0.05):
+            Minimum denominator when converting an x0 prediction into a velocity.
+        default_num_inference_steps (`int`, defaults to 50):
+            Heun steps used when the pipeline caller omits `num_inference_steps`.
+    """
+
+    @register_to_config
+    def __init__(
+        self,
+        sample_size: int = 256,
+        in_channels: int = 3,
+        patch_size: int = 16,
+        hidden_size: int = 1280,
+        num_heads: int = 16,
+        depth: int = 32,
+        num_classes: int = 1000,
+        in_context_len: int = 32,
+        in_context_start: int = 8,
+        attn_dropout: float = 0.0,
+        proj_dropout: float = 0.2,
+        mlp_hidden_multiple: int = 0,
+        grounding_encoder: str = "dinov3_vits16",
+        grounding_proj_blocks: int = 1,
+        grounding_proj_heads: int = 16,
+        single_silu_cond: bool = False,
+        guidance_scale: float = 2.4,
+        guidance_interval_min: float = 0.125,
+        guidance_interval_max: float = 0.9,
+        noise_scale: float = 1.0,
+        t_eps: float = 0.05,
+        default_num_inference_steps: int = 50,
+    ):
+        super().__init__()
+        if not 0 <= in_context_start < depth:
+            raise ValueError(f"in_context_start must be in [0, {depth - 1}], got {in_context_start}.")
+        if hidden_size % num_heads != 0:
+            raise ValueError(f"hidden_size={hidden_size} must be divisible by num_heads={num_heads}.")
+
+        self.in_channels = in_channels
+        self.out_channels = in_channels
+        self.hidden_size = hidden_size
+        self.num_heads = num_heads
+        self.depth = depth
+        self.patch_size = patch_size
+        self.num_classes = num_classes
+        self.in_context_len = in_context_len
+        self.in_context_start = in_context_start
+        self.single_silu_cond = single_silu_cond
+
+        self.t_embedder = TimestepConditioner(hidden_size)
+        self.y_embedder = ClassEmbedder(num_classes + 1, hidden_size)
+        self.s_embedder = PatchTokenEmbedder(in_channels * patch_size**2, hidden_size, bias=True)
+        self.in_context_posemb = nn.Parameter(torch.zeros(1, in_context_len, hidden_size)) if in_context_len > 0 else None
+        self.patch_blocks = nn.ModuleList(
+            [
+                ModulatedDiTBlock(
+                    hidden_size,
+                    num_heads,
+                    attn_drop=attn_dropout if depth // 4 <= i < depth // 4 * 3 else 0.0,
+                    proj_drop=proj_dropout if depth // 4 <= i < depth // 4 * 3 else 0.0,
+                    mlp_hidden_multiple=mlp_hidden_multiple,
+                )
+                for i in range(depth)
+            ]
+        )
+        self.final_layer = PatchFinalLayer(hidden_size, patch_size, self.out_channels)
+        self._init_denoiser_weights()
+        self.dino_conditioner = GroundingConditioner(
+            encoder_name=grounding_encoder,
+            hidden_size=hidden_size,
+            target_patch_size=patch_size,
+            proj_num_blocks=grounding_proj_blocks,
+            proj_num_heads=grounding_proj_heads,
+        )
+        self._pos_cache: dict[tuple[int, int, int], torch.Tensor] = {}
+
+    def _init_denoiser_weights(self) -> None:
+        r"""
+        Initialize the trainable denoiser the same way as the training code.
+
+        The grounding encoder is created afterwards and is not touched here.
+        """
+        weight = self.s_embedder.proj.weight.data
+        nn.init.xavier_uniform_(weight.view([weight.shape[0], -1]))
+        nn.init.constant_(self.s_embedder.proj.bias, 0)
+        nn.init.normal_(self.y_embedder.embedding_table.weight, std=0.02)
+        nn.init.normal_(self.t_embedder.mlp[0].weight, std=0.02)
+        nn.init.normal_(self.t_embedder.mlp[2].weight, std=0.02)
+        if self.in_context_posemb is not None:
+            nn.init.normal_(self.in_context_posemb, std=0.02)
+        for block in self.patch_blocks:
+            nn.init.zeros_(block.adaLN_modulation[-1].weight)
+            nn.init.zeros_(block.adaLN_modulation[-1].bias)
+        nn.init.zeros_(self.final_layer.linear.weight)
+        nn.init.zeros_(self.final_layer.linear.bias)
+        nn.init.zeros_(self.final_layer.adaLN_modulation[-1].weight)
+        nn.init.zeros_(self.final_layer.adaLN_modulation[-1].bias)
+
+    def fetch_pos(self, height: int, width: int, device: torch.device, num_context_tokens: int = 0) -> torch.Tensor:
+        r"""
+        RoPE table for a patch grid, with an identity prefix when context tokens are present.
+
+        Args:
+            height (`int`):
+                Grid height in patches.
+            width (`int`):
+                Grid width in patches.
+            device (`torch.device`):
+                Device of the returned table.
+            num_context_tokens (`int`, defaults to 0):
+                Leading tokens that receive an identity rotation (real part 1, imaginary part 0).
+
+        Returns:
+            `torch.Tensor`: Complex frequencies of shape `(num_context_tokens + height * width, head_dim / 2)`.
+        """
+        key = (height, width, num_context_tokens)
+        if key not in self._pos_cache:
+            pos = precompute_freqs_cis_2d(self.hidden_size // self.num_heads, height, width)
+            if num_context_tokens > 0:
+                identity = torch.ones(num_context_tokens, pos.shape[-1], dtype=pos.dtype)
+                pos = torch.cat([identity, pos], dim=0)
+            self._pos_cache[key] = pos
+        return self._pos_cache[key].to(device)
+
+    def _grounded_condition(
+        self,
+        hidden_states: torch.Tensor,
+        timesteps: torch.Tensor,
+        timestep_embedding: torch.Tensor,
+        class_embedding: torch.Tensor,
+    ) -> torch.Tensor:
+        r"""
+        Conditioning used by the class-conditional branch.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Noisy images `(batch, channels, height, width)`.
+            timesteps (`torch.Tensor`):
+                Flow time `(batch,)`.
+            timestep_embedding (`torch.Tensor`):
+                Timestep embedding `(batch, hidden)`.
+            class_embedding (`torch.Tensor`):
+                Class embedding `(batch, 1, hidden)`.
+
+        Returns:
+            `torch.Tensor`: Per-patch conditioning `(batch, patches, hidden)`.
+        """
+        grounding = self.dino_conditioner(hidden_states, timesteps)
+        conditioning = grounding + timestep_embedding.unsqueeze(1) + class_embedding
+        if self.single_silu_cond:
+            return conditioning
+        return F.silu(conditioning)
+
+    def _null_condition(
+        self,
+        timestep_embedding: torch.Tensor,
+        class_embedding: torch.Tensor,
+        num_patches: int,
+    ) -> torch.Tensor:
+        r"""
+        Conditioning used when grounding is dropped.
+
+        Args:
+            timestep_embedding (`torch.Tensor`):
+                Timestep embedding `(batch, hidden)`.
+            class_embedding (`torch.Tensor`):
+                Class embedding `(batch, 1, hidden)`.
+            num_patches (`int`):
+                Number of spatial patches to broadcast over.
+
+        Returns:
+            `torch.Tensor`: Conditioning `(batch, num_patches, hidden)`.
+        """
+        conditioning = timestep_embedding.unsqueeze(1) + class_embedding
+        if not self.single_silu_cond:
+            conditioning = F.silu(conditioning)
+        return conditioning.expand(-1, num_patches, -1)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        timestep: torch.Tensor,
+        class_labels: torch.Tensor,
+        skip_grounding: bool = False,
+        skip_grounding_mask: torch.Tensor | None = None,
+        return_dict: bool = True,
+    ) -> Transformer2DModelOutput | tuple:
+        r"""
+        Predict the clean image x0.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Noisy RGB sample of shape `(batch, in_channels, height, width)`, in model range around `[-1, 1]`.
+                `height` and `width` must be divisible by `patch_size`.
+            timestep (`torch.Tensor`):
+                Flow-matching time `t` in `[0, 1]`, shape `(batch,)` or a scalar broadcast to the batch.
+                `t = 0` is noise and `t = 1` is data.
+            class_labels (`torch.Tensor`):
+                Integer ImageNet ids of shape `(batch,)`. The null class is `num_classes`.
+            skip_grounding (`bool`, defaults to `False`):
+                When `True`, do not run the DINOv3 encoder. The unconditional CFG branch sets this.
+            skip_grounding_mask (`torch.Tensor`, *optional*):
+                Boolean mask of shape `(batch,)`. `True` rows use the null grounding condition.
+                Ignored when `skip_grounding` is `True`.
+            return_dict (`bool`, defaults to `True`):
+                Return [`Transformer2DModelOutput`] when `True`, otherwise a one-tuple of the sample.
+
+        Returns:
+            [`Transformer2DModelOutput`] or `tuple`:
+                The x0 prediction of shape `(batch, in_channels, height, width)`.
+        """
+        batch, _, height, width = hidden_states.shape
+        if height % self.patch_size or width % self.patch_size:
+            raise ValueError(
+                f"Input resolution ({height}, {width}) must be divisible by patch_size={self.patch_size}."
+            )
+        timestep = timestep.reshape(batch)
+        class_labels = class_labels.reshape(batch).long()
+        grid_h, grid_w = height // self.patch_size, width // self.patch_size
+        num_patches = grid_h * grid_w
+
+        timestep_embedding = self.t_embedder(timestep)
+        class_embedding = self.y_embedder(class_labels).view(batch, 1, self.hidden_size)
+
+        if skip_grounding:
+            conditioning = self._null_condition(timestep_embedding, class_embedding, num_patches)
+        else:
+            conditioning = self._grounded_condition(hidden_states, timestep, timestep_embedding, class_embedding)
+            if skip_grounding_mask is not None:
+                null_conditioning = self._null_condition(timestep_embedding, class_embedding, num_patches)
+                conditioning = torch.where(skip_grounding_mask.view(batch, 1, 1), null_conditioning, conditioning)
+
+        tokens = self.s_embedder(F.unfold(hidden_states, kernel_size=self.patch_size, stride=self.patch_size).transpose(1, 2))
+        use_context = self.in_context_len > 0
+        pos = self.fetch_pos(grid_h, grid_w, hidden_states.device)
+        pos_with_context = self.fetch_pos(grid_h, grid_w, hidden_states.device, self.in_context_len) if use_context else None
+        if use_context:
+            context_conditioning = conditioning.mean(dim=1, keepdim=True).expand(-1, self.in_context_len, -1)
+            conditioning_extended = torch.cat([context_conditioning, conditioning], dim=1)
+
+        context_inserted = False
+        for index, block in enumerate(self.patch_blocks):
+            if use_context and not context_inserted and index == self.in_context_start:
+                context_tokens = class_embedding.expand(-1, self.in_context_len, -1)
+                tokens = torch.cat([context_tokens + self.in_context_posemb.to(tokens.dtype), tokens], dim=1)
+                context_inserted = True
+            tokens = block(
+                tokens,
+                conditioning_extended if context_inserted else conditioning,
+                pos_with_context if context_inserted else pos,
+            )
+        if context_inserted:
+            tokens = tokens[:, self.in_context_len :, :]
+
+        output = self.final_layer(tokens, conditioning)
+        output = output.view(batch, num_patches, self.patch_size**2, self.out_channels)
+        output = output.permute(0, 3, 2, 1).reshape(batch, self.out_channels * self.patch_size**2, num_patches)
+        output = F.fold(output, (height, width), kernel_size=self.patch_size, stride=self.patch_size)
+
+        if not return_dict:
+            return (output,)
+        return Transformer2DModelOutput(sample=output)

PixelDiT2-H-16-512/encoder/LICENSE.md

@@ -0,0 +1,66 @@
+# DINOv3 License
+
+*Last Updated: August 19, 2025*
+
+**“Agreement”** means the terms and conditions for use, reproduction, distribution and modification of the DINO Materials set forth herein.
+
+**“DINO Materials”** means, collectively, Documentation and the models, software and algorithms, including machine-learning model code, trained model weights, inference-enabling code, training-enabling code, fine-tuning enabling code, and other elements of the foregoing distributed by Meta and made available under this Agreement.
+
+**“Documentation”** means the specifications, manuals and documentation accompanying
+DINO Materials distributed by Meta.
+
+**“Licensee”** or **“you”** means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity’s behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf.
+
+**“Meta”** or **“we”** means Meta Platforms Ireland Limited (if you are located in or, if you are an entity, your principal place of business is in the EEA or Switzerland) or Meta Platforms, Inc. (if you are located outside of the EEA or Switzerland).
+
+**“Sanctions”** means any economic or trade sanctions or restrictions administered or enforced by the United States (including the Office of Foreign Assets Control of the U.S. Department of the Treasury (“OFAC”), the U.S. Department of State and the U.S. Department of Commerce), the United Nations, the European Union, or the United Kingdom.
+
+**“Trade Controls”** means any of the following: Sanctions and applicable export and import controls.
+
+By clicking “I Accept” below or by using or distributing any portion or element of the DINO Materials, you agree to be bound by this Agreement.
+
+## 1. License Rights and Redistribution.
+
+a. <ins>Grant of Rights</ins>. You are granted a non-exclusive, worldwide, non-transferable and royalty-free limited license under Meta’s intellectual property or other rights owned by Meta embodied in the DINO Materials to use, reproduce, distribute, copy, create derivative works of, and make modifications to the DINO Materials.
+
+b. <ins>Redistribution and Use</ins>.
+
+i. Distribution of DINO Materials, and any derivative works thereof, are subject to the terms of this Agreement. If you distribute or make the DINO Materials, or any derivative works thereof, available to a third party, you may only do so under the terms of this Agreement and you shall provide a copy of this Agreement with any such DINO Materials.
+
+ii.  If you submit for publication the results of research you perform on, using, or otherwise in connection with DINO Materials, you must acknowledge the use of DINO Materials in your publication.
+
+iii. Your use of the DINO Materials must comply with applicable laws and regulations, including Trade Control Laws and applicable privacy and data protection laws.
+
+iv. Your use of the DINO Materials will not involve or encourage others to reverse engineer, decompile or discover the underlying components of the DINO Materials.
+
+v. You are not the target of Trade Controls and your use of DINO Materials must comply with Trade Controls. You agree not to use, or permit others to use, DINO Materials for any activities subject to the International Traffic in Arms Regulations (ITAR) or end uses prohibited by Trade Controls, including those related to military or warfare purposes, nuclear industries or applications, espionage, or the development or use of guns or illegal weapons.
+
+## 2. User Support.
+
+Your use of the DINO Materials is done at your own discretion; Meta does not process any information nor provide any service in relation to such use.  Meta is under no obligation to provide any support services for the DINO Materials. Any support provided is “as is”, “with all faults”, and without warranty of any kind.
+
+## 3. Disclaimer of Warranty.
+
+UNLESS REQUIRED BY APPLICABLE LAW, THE DINO MATERIALS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN “AS IS” BASIS, WITHOUT WARRANTIES OF ANY KIND, AND META DISCLAIMS ALL WARRANTIES OF ANY KIND, BOTH EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE DINO MATERIALS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE DINO MATERIALS AND ANY OUTPUT AND RESULTS.
+
+## 4. Limitation of Liability.
+
+IN NO EVENT WILL META OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY DIRECT OR INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF META OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+
+## 5. Intellectual Property.
+
+a. Subject to Meta’s ownership of DINO Materials and derivatives made by or for Meta, with respect to any derivative works and modifications of the DINO Materials that are made by you, as between you and Meta, you are and will be the owner of such derivative works and modifications.
+
+b. If you institute litigation or other proceedings against Meta or any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the DINO Materials, outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Meta from and against any claim by any third party arising out of or related to your use or distribution of the DINO Materials.
+
+## 6. Term and Termination.
+
+The term of this Agreement will commence upon your acceptance of this Agreement or access to the DINO Materials and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Meta may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the DINO Materials. Sections 3, 4 and 7 shall survive the termination of this Agreement.
+
+## 7. Governing Law and Jurisdiction.
+
+This Agreement will be governed and construed under the laws of the State of California without regard to choice of law principles, and the UN Convention on Contracts for the International Sale of Goods does not apply to this Agreement. The courts of California shall have exclusive jurisdiction of any dispute arising out of this Agreement.
+
+## 8. Modifications and Amendments.
+
+Meta may modify this Agreement from time to time; provided that they are similar in spirit to the current version of the Agreement, but may differ in detail to address new problems or concerns. All such changes will be effective immediately. Your continued use of the DINO Materials after any modification to this Agreement constitutes your agreement to such modification. Except as provided in this Agreement, no modification or addition to any provision of this Agreement will be binding unless it is in writing and signed by an authorized representative of both you and Meta.

PixelDiT2-H-16-512/encoder/README.md

@@ -0,0 +1,477 @@
+---
+extra_gated_fields:
+  First Name: text
+  Last Name: text
+  Date of birth: date_picker
+  Country: country
+  Affiliation: text
+  Job title:
+    type: select
+    options:
+      - Student
+      - Research Graduate
+      - AI researcher
+      - AI developer/engineer
+      - Reporter
+      - Other
+  geo: ip_location
+  By clicking Submit below I accept the terms of the license and acknowledge that the information I provide will be collected stored processed and shared in accordance with the Meta Privacy Policy: checkbox
+extra_gated_description: >-
+  The information you provide will be collected, stored, processed and shared in
+  accordance with the [Meta Privacy
+  Policy](https://www.facebook.com/privacy/policy/).
+extra_gated_button_content: Submit
+language:
+- en
+tags:
+- dino
+- dinov3
+- arxiv:2508.10104
+license: other
+license_name: dinov3-license
+license_link: https://ai.meta.com/resources/models-and-libraries/dinov3-license
+base_model: facebook/dinov3-vit7b16-pretrain-lvd1689m
+pipeline_tag: image-feature-extraction
+library_name: transformers
+---
+
+# Model Card for DINOv3
+
+DINOv3 is a family of versatile vision foundation models that outperforms the specialized state of the art across a broad range of settings, without fine-tuning. DINOv3 produces high-quality dense features that achieve outstanding performance on various vision tasks, significantly surpassing previous self- and weakly-supervised foundation models.
+
+## Model Details
+
+These are Vision Transformer and ConvNeXt models trained following the method described in the DINOv3 paper. 12 models are provided:
+
+- 10 models pretrained on web data (LVD-1689M dataset)
+  - 1 ViT-7B trained from scratch,
+  - 5 ViT-S/S+/B/L/H+ models distilled from the ViT-7B,
+  - 4 ConvNeXt-{T/S/B/L} models distilled from the ViT-7B,
+- 2 models pretrained on satellite data (SAT-493M dataset)
+  - 1 ViT-7B trained from scratch
+  - 1 ViT-L distilled from the ViT-7B
+
+
+Each Transformer-based model takes an image as input and returns a class token, patch tokens (and register tokens). These models follow a ViT architecture, with a patch size of 16. For a 224x224 image, this results in 1 class token + 4 register tokens + 196 patch tokens = 201 tokens (for DINOv2 with registers this resulted in 1 + 4 + 256 = 261 tokens).
+
+The models can accept larger images provided the image shapes are multiples of the patch size (16). If this condition is not verified, the model will crop to the closest smaller multiple of the patch size.
+
+### Model Description
+
+- **Developed by:** Meta AI
+- **Model type:** Vision Transformer, ConvNeXt
+- **License:** [DINOv3 License](https://ai.meta.com/resources/models-and-libraries/dinov3-license/)
+
+### Model Sources
+
+- **Repository:** [https://github.com/facebookresearch/dinov3](https://github.com/facebookresearch/dinov3)
+- **Paper:** [https://arxiv.org/abs/2508.10104](https://arxiv.org/abs/2508.10104)
+
+## Uses
+
+The models are vision backbones providing multi-purpose features for downstream tasks.
+
+### Direct Use
+
+The models can be used without fine-tuning, with downstream classifiers as simple as linear layers, to obtain competitive results:
+
+- on image classification, using k-NN classifiers on the class token
+- on image classification, with logistic regression classifiers applied on the class token
+- on image classification, with a linear layer applied on the class token and the average of the patch tokens
+- on image retrieval using nearest neighbors
+- on geometric and semantic 3D keypoint correspondances
+- on depth estimation, semantic segmentation, using linear layers
+- on unsupervised object discovery
+- on video segmentation tracking
+- on video classification, using a small 4-layer attentive probe
+
+### Downstream Use
+
+While fine-tuning the models can yield some gains, it is recommended to keep this option as a last resort: the frozen features are expected to provide good performance out-of-the-box.
+
+## Bias, Risks, and Limitations
+
+Compared to DINOv2 and SEERv2, DINOv3 delivers somewhat consistent performance across income categories on geographical fairness and diversity, although with a notable performance drop in the low-income bucket compared to the highest-income bucket.
+
+DINOv3 also achieves relatively good scores across different regions, improving over its predecessor DINOv2. However, a relative difference is still observed between Europe and Africa.
+
+### Recommendations
+
+Fine-tuning is expected to increase the biases in the features produced by the model as they will be tuned to the fine-tuning labels.
+
+## How to Get Started with the Model
+
+The example below demonstrates how to obtain an image embedding with [Pipeline] or the [AutoModel] class.
+
+```python
+from transformers import pipeline
+from transformers.image_utils import load_image
+
+url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg"
+image = load_image(url)
+
+feature_extractor = pipeline(
+    model="facebook/dinov3-vitb16-pretrain-lvd1689m",
+    task="image-feature-extraction", 
+)
+features = feature_extractor(image)
+```
+
+```python
+import torch
+from transformers import AutoImageProcessor, AutoModel
+from transformers.image_utils import load_image
+
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = load_image(url)
+
+pretrained_model_name = "facebook/dinov3-vitb16-pretrain-lvd1689m"
+processor = AutoImageProcessor.from_pretrained(pretrained_model_name)
+model = AutoModel.from_pretrained(
+    pretrained_model_name, 
+    device_map="auto", 
+)
+
+inputs = processor(images=image, return_tensors="pt").to(model.device)
+with torch.inference_mode():
+    outputs = model(**inputs)
+
+pooled_output = outputs.pooler_output
+print("Pooled output shape:", pooled_output.shape)
+```
+
+## Training Details
+
+### Training Data
+
+- Web dataset (LVD-1689M): a curated dataset of 1,689 millions of images extracted from a large data
+pool of 17 billions web images collected from public posts on Instagram
+
+- Satellite dataset (SAT-493M): a dataset of 493 millions of 512x512 images sampled randomly from Maxar RGB ortho-rectified imagery at 0.6 meter resolution
+
+### Training Procedure
+
+**Training objective:**
+
+- DINO self-distillation loss with multi-crop
+- iBOT masked-image modeling loss
+- KoLeo regularization on [CLS] tokens
+- Gram anchoring
+
+- **Training regime:** PyTorch FSDP2 (with bf16 and fp8 matrix multiplications)
+
+**Distillation:**
+
+- Distillation follows the standard DINOv3 pretraining procedure, except the teacher is a frozen pretrained ViT-7B.
+
+## Evaluation
+
+**Results**
+
+The reader is referred to the associated paper for details on the evaluation protocols
+
+*Results for ViT backbones pretrained (or distilled) on web (LVD-1689M)*
+
+<table>
+  <tr>
+    <th></th>
+    <!-- <th></th> -->
+    <th colspan="4">Global Tasks</th>
+    <th colspan="5">Dense Tasks</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <!-- <th>Dataset</th> -->
+    <th>IN-ReaL</th>
+    <th>IN-R</th>
+    <th>Obj.Net</th>
+    <th>Ox.-H</th>
+    <th>ADE20k</th>
+    <th>NYU↓</th>
+    <th>DAVIS</th>
+    <th>NAVI</th>
+    <th>SPair</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-S/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">87.0</td>
+    <td align="right">60.4</td>
+    <td align="right">50.9</td>
+    <td align="right">49.5</td>
+    <td align="right">47.0</td>
+    <td align="right">0.403</td>
+    <td align="right">72.7</td>
+    <td align="right">56.3</td>
+    <td align="right">50.4</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-S+/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">88.0</td>
+    <td align="right">68.8</td>
+    <td align="right">54.6</td>
+    <td align="right">50.0</td>
+    <td align="right">48.8</td>
+    <td align="right">0.399</td>
+    <td align="right">75.5</td>
+    <td align="right">57.1</td>
+    <td align="right">55.2</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-B/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">89.3</td>
+    <td align="right">76.7</td>
+    <td align="right">64.1</td>
+    <td align="right">58.5</td>
+    <td align="right">51.8</td>
+    <td align="right">0.373</td>
+    <td align="right">77.2</td>
+    <td align="right">58.8</td>
+    <td align="right">57.2</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.2</td>
+    <td align="right">88.1</td>
+    <td align="right">74.8</td>
+    <td align="right">63.1</td>
+    <td align="right">54.9</td>
+    <td align="right">0.352</td>
+    <td align="right">79.9</td>
+    <td align="right">62.3</td>
+    <td align="right">61.3</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-H+/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.3</td>
+    <td align="right">90.0</td>
+    <td align="right">78.6</td>
+    <td align="right">64.5</td>
+    <td align="right">54.8</td>
+    <td align="right">0.352</td>
+    <td align="right">79.3</td>
+    <td align="right">63.3</td>
+    <td align="right">56.3</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <!-- <td>LVD-1689M</td> -->
+    <td align="right">90.4</td>
+    <td align="right">91.1</td>
+    <td align="right">91.1</td>
+    <td align="right">72.8</td>
+    <td align="right">55.9</td>
+    <td align="right">0.309</td>
+    <td align="right">79.7</td>
+    <td align="right">64.4</td>
+    <td align="right">58.7</td>
+  </tr>
+</table>
+
+*Results for ConvNeXt backbones distilled on web (LVD-1689M)*
+
+<table>
+  <tr>
+    <th></th>
+    <th colspan="6">Global Tasks</th>
+    <th colspan="2">Dense Tasks</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <th colspan="2">IN-ReaL</th>
+    <th colspan="2">IN-R</th>
+    <th colspan="2">Obj.Net</th>
+    <th>ADE20k</th>
+    <th>NYU↓</th>
+  </tr>
+  <tr>
+    <td></th>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td>@256px</td>
+    <td>@512px</td>
+    <td colspan="2"></td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Tiny</td>
+    <td align="right">86.6</td>
+    <td align="right">87.7</td>
+    <td align="right">73.7</td>
+    <td align="right">74.1</td>
+    <td align="right">52.6</td>
+    <td align="right">58.7</td>
+    <td align="right">42.7</td>
+    <td align="right">0.448</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Small</td>
+    <td align="right">87.9</td>
+    <td align="right">88.7</td>
+    <td align="right">73.7</td>
+    <td align="right">74.1</td>
+    <td align="right">52.6</td>
+    <td align="right">58.7</td>
+    <td align="right">44.8</td>
+    <td align="right">0.432</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Base</td>
+    <td align="right">88.5</td>
+    <td align="right">89.2</td>
+    <td align="right">77.2</td>
+    <td align="right">78.2</td>
+    <td align="right">56.2</td>
+    <td align="right">61.3</td>
+    <td align="right">46.3</td>
+    <td align="right">0.420</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ConvNeXt Large</td>
+    <td align="right">88.9</td>
+    <td align="right">89.4</td>
+    <td align="right">81.3</td>
+    <td align="right">82.4</td>
+    <td align="right">59.3</td>
+    <td align="right">65.2</td>
+    <td align="right">47.8</td>
+    <td align="right">0.403</td>
+  </tr>
+</table>
+
+*Results for ViT backbones pretrained (or distilled) on satellite (SAT-493M)*
+
+<table>
+  <tr>
+    <th></th>
+    <th colspan="7">(GEO-Bench) Classification</th>
+  </tr>
+  <tr>
+    <th>Model</ht>
+    <th>m-BEnet</th>
+    <th>m-brick-kiln
+    <th>m-eurosat</th>
+    <th>m-forestnet</th>
+    <th>m-pv4ger</th>
+    <th>m-so2sat</th>
+    <th>mean</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <td>73.0</td>
+    <td>96.5</td>
+    <td>94.1</td>
+    <td>60.6</td>
+    <td>96.0</td>
+    <td>57.4</td>
+    <td>79.6</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <td>74.0</td>
+    <td>97.2</td>
+    <td>94.8</td>
+    <td>62.3</td>
+    <td>96.1</td>
+    <td>62.1</td>
+    <td>81.1</td>
+  </tr>
+  <tr>
+    <th></th>
+    <th colspan="7">(GEO-Bench) Segmentation</th>
+  </tr>
+  <tr>
+    <th>Model</th>
+    <th>m-cashew</th>
+    <th>m-chesapeake</th>
+    <th>m-NeonTree</th>
+    <th>m-nz-cattle</th>
+    <th>m-pv4ger-seg</th>
+    <th>m-SA-crop</th>
+    <th>mean</th>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-L/16</td>
+    <td>94.2</td>
+    <td>75.6</td>
+    <td>61.8</td>
+    <td>83.7</td>
+    <td>95.2</td>
+    <td>36.8</td>
+    <td>74.5</td>
+  </tr>
+  <tr>
+    <td>DINOv3 ViT-7B/16</td>
+    <td>94.1</td>
+    <td>76.6</td>
+    <td>62.6</td>
+    <td>83.4</td>
+    <td>95.5</td>
+    <td>37.6</td>
+    <td>75.0</td>
+  </tr>
+</table>
+
+
+## Environmental Impact
+
+- **Hardware Type:** Nvidia H100
+- **Hours used:** 61,440 hours for ViT-7B model training
+- **Cloud Provider:** Private infrastructure
+- **Compute Region:** USA
+- **Carbon Emitted:** 18t CO2eq
+
+## Technical Specifications
+
+### Model Architecture and Objective
+
+Vision Transformer models:
+
+- ViT-S (21M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, MLP FFN, RoPE
+- ViT-S+ (29M parameters): patch size 16, embedding dimension 384, 4 register tokens, 6 heads, SwiGLU FFN, RoPE
+- ViT-B (86M parameters): patch size 16, embedding dimension 768, 4 register tokens, 12 heads, MLP FFN, RoPE
+- ViT-L (300M parameters): patch size 16, embedding dimension 1024, 4 register tokens, 16 heads, MLP FFN, RoPE
+- ViT-H+ (840M parameters): patch size 16, embedding dimension 1280, 4 register tokens, 20 heads, SwiGLU FFN, RoPE
+- ViT-7B (6716M parameters): patch size 16, embedding dimension 4096, 4 register tokens, 32 heads, SwiGLU FFN, RoPE
+
+ConvNeXt models:
+
+- ConvNeXt Tiny (29M parameters)
+- ConvNeXt Small (50M parameters)
+- ConvNeXt Base (89M parameters)
+- ConvNeXt Large (198M parameters)
+
+### Compute Infrastructure
+
+#### Hardware
+
+Nvidia H100 GPUs
+
+#### Software
+
+PyTorch 2.7
+
+## More Information
+
+See the [blog post](https://ai.meta.com/blog/dinov3-self-supervised-vision-model/) and the associated [website](https://ai.meta.com/dinov3/).
+
+## Citation
+
+**BibTeX**
+
+```
+@misc{simeoni2025dinov3,
+  title={{DINOv3}},
+  author={Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
+  year={2025},
+  eprint={2508.10104},
+  archivePrefix={arXiv},
+  primaryClass={cs.CV},
+  url={https://arxiv.org/abs/2508.10104},
+}
+```

PixelDiT2-H-16-512/encoder/config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "DINOv3ViTModel"
+  ],
+  "attention_dropout": 0.0,
+  "drop_path_rate": 0.0,
+  "hidden_act": "gelu",
+  "hidden_size": 768,
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "key_bias": false,
+  "layer_norm_eps": 1e-05,
+  "layerscale_value": 1.0,
+  "mlp_bias": true,
+  "model_type": "dinov3_vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "num_register_tokens": 4,
+  "patch_size": 16,
+  "pos_embed_jitter": null,
+  "pos_embed_rescale": 2.0,
+  "pos_embed_shift": null,
+  "proj_bias": true,
+  "query_bias": true,
+  "rope_theta": 100.0,
+  "torch_dtype": "float32",
+  "transformers_version": "4.56.0.dev0",
+  "use_gated_mlp": false,
+  "value_bias": true
+}

PixelDiT2-H-16-512/encoder/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9a21ac3df0c63839d62612dda6f454d816c25611cc7a52966ed5a5a94921dc8b
+size 342662192

PixelDiT2-H-16-512/encoder/preprocessor_config.json

@@ -0,0 +1,31 @@
+{
+  "crop_size": null,
+  "data_format": "channels_first",
+  "default_to_square": true,
+  "device": null,
+  "disable_grouping": null,
+  "do_center_crop": null,
+  "do_convert_rgb": null,
+  "do_normalize": true,
+  "do_rescale": true,
+  "do_resize": true,
+  "image_mean": [
+    0.485,
+    0.456,
+    0.406
+  ],
+  "image_processor_type": "DINOv3ViTImageProcessorFast",
+  "image_std": [
+    0.229,
+    0.224,
+    0.225
+  ],
+  "input_data_format": null,
+  "resample": 2,
+  "rescale_factor": 0.00392156862745098,
+  "return_tensors": null,
+  "size": {
+    "height": 224,
+    "width": 224
+  }
+}

PixelDiT2-H-16-512/model_index.json

@@ -0,0 +1,1021 @@
+{
+  "_class_name": [
+    "pipeline",
+    "PixelDiT2Pipeline"
+  ],
+  "_diffusers_version": "0.39.0",
+  "encoder": [
+    "transformers",
+    "DINOv3ViTModel"
+  ],
+  "scheduler": [
+    "scheduling_pixeldit2",
+    "PixelDiT2Scheduler"
+  ],
+  "transformer": [
+    "transformer_pixeldit2",
+    "PixelDiT2Transformer2DModel"
+  ],
+  "id2label": {
+    "0": "tench, Tinca tinca",
+    "1": "goldfish, Carassius auratus",
+    "2": "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias",
+    "3": "tiger shark, Galeocerdo cuvieri",
+    "4": "hammerhead, hammerhead shark",
+    "5": "electric ray, crampfish, numbfish, torpedo",
+    "6": "stingray",
+    "7": "cock",
+    "8": "hen",
+    "9": "ostrich, Struthio camelus",
+    "10": "brambling, Fringilla montifringilla",
+    "11": "goldfinch, Carduelis carduelis",
+    "12": "house finch, linnet, Carpodacus mexicanus",
+    "13": "junco, snowbird",
+    "14": "indigo bunting, indigo finch, indigo bird, Passerina cyanea",
+    "15": "robin, American robin, Turdus migratorius",
+    "16": "bulbul",
+    "17": "jay",
+    "18": "magpie",
+    "19": "chickadee",
+    "20": "water ouzel, dipper",
+    "21": "kite",
+    "22": "bald eagle, American eagle, Haliaeetus leucocephalus",
+    "23": "vulture",
+    "24": "great grey owl, great gray owl, Strix nebulosa",
+    "25": "European fire salamander, Salamandra salamandra",
+    "26": "common newt, Triturus vulgaris",
+    "27": "eft",
+    "28": "spotted salamander, Ambystoma maculatum",
+    "29": "axolotl, mud puppy, Ambystoma mexicanum",
+    "30": "bullfrog, Rana catesbeiana",
+    "31": "tree frog, tree-frog",
+    "32": "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui",
+    "33": "loggerhead, loggerhead turtle, Caretta caretta",
+    "34": "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea",
+    "35": "mud turtle",
+    "36": "terrapin",
+    "37": "box turtle, box tortoise",
+    "38": "banded gecko",
+    "39": "common iguana, iguana, Iguana iguana",
+    "40": "American chameleon, anole, Anolis carolinensis",
+    "41": "whiptail, whiptail lizard",
+    "42": "agama",
+    "43": "frilled lizard, Chlamydosaurus kingi",
+    "44": "alligator lizard",
+    "45": "Gila monster, Heloderma suspectum",
+    "46": "green lizard, Lacerta viridis",
+    "47": "African chameleon, Chamaeleo chamaeleon",
+    "48": "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis",
+    "49": "African crocodile, Nile crocodile, Crocodylus niloticus",
+    "50": "American alligator, Alligator mississipiensis",
+    "51": "triceratops",
+    "52": "thunder snake, worm snake, Carphophis amoenus",
+    "53": "ringneck snake, ring-necked snake, ring snake",
+    "54": "hognose snake, puff adder, sand viper",
+    "55": "green snake, grass snake",
+    "56": "king snake, kingsnake",
+    "57": "garter snake, grass snake",
+    "58": "water snake",
+    "59": "vine snake",
+    "60": "night snake, Hypsiglena torquata",
+    "61": "boa constrictor, Constrictor constrictor",
+    "62": "rock python, rock snake, Python sebae",
+    "63": "Indian cobra, Naja naja",
+    "64": "green mamba",
+    "65": "sea snake",
+    "66": "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus",
+    "67": "diamondback, diamondback rattlesnake, Crotalus adamanteus",
+    "68": "sidewinder, horned rattlesnake, Crotalus cerastes",
+    "69": "trilobite",
+    "70": "harvestman, daddy longlegs, Phalangium opilio",
+    "71": "scorpion",
+    "72": "black and gold garden spider, Argiope aurantia",
+    "73": "barn spider, Araneus cavaticus",
+    "74": "garden spider, Aranea diademata",
+    "75": "black widow, Latrodectus mactans",
+    "76": "tarantula",
+    "77": "wolf spider, hunting spider",
+    "78": "tick",
+    "79": "centipede",
+    "80": "black grouse",
+    "81": "ptarmigan",
+    "82": "ruffed grouse, partridge, Bonasa umbellus",
+    "83": "prairie chicken, prairie grouse, prairie fowl",
+    "84": "peacock",
+    "85": "quail",
+    "86": "partridge",
+    "87": "African grey, African gray, Psittacus erithacus",
+    "88": "macaw",
+    "89": "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita",
+    "90": "lorikeet",
+    "91": "coucal",
+    "92": "bee eater",
+    "93": "hornbill",
+    "94": "hummingbird",
+    "95": "jacamar",
+    "96": "toucan",
+    "97": "drake",
+    "98": "red-breasted merganser, Mergus serrator",
+    "99": "goose",
+    "100": "black swan, Cygnus atratus",
+    "101": "tusker",
+    "102": "echidna, spiny anteater, anteater",
+    "103": "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus",
+    "104": "wallaby, brush kangaroo",
+    "105": "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus",
+    "106": "wombat",
+    "107": "jellyfish",
+    "108": "sea anemone, anemone",
+    "109": "brain coral",
+    "110": "flatworm, platyhelminth",
+    "111": "nematode, nematode worm, roundworm",
+    "112": "conch",
+    "113": "snail",
+    "114": "slug",
+    "115": "sea slug, nudibranch",
+    "116": "chiton, coat-of-mail shell, sea cradle, polyplacophore",
+    "117": "chambered nautilus, pearly nautilus, nautilus",
+    "118": "Dungeness crab, Cancer magister",
+    "119": "rock crab, Cancer irroratus",
+    "120": "fiddler crab",
+    "121": "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica",
+    "122": "American lobster, Northern lobster, Maine lobster, Homarus americanus",
+    "123": "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
+    "124": "crayfish, crawfish, crawdad, crawdaddy",
+    "125": "hermit crab",
+    "126": "isopod",
+    "127": "white stork, Ciconia ciconia",
+    "128": "black stork, Ciconia nigra",
+    "129": "spoonbill",
+    "130": "flamingo",
+    "131": "little blue heron, Egretta caerulea",
+    "132": "American egret, great white heron, Egretta albus",
+    "133": "bittern",
+    "134": "crane",
+    "135": "limpkin, Aramus pictus",
+    "136": "European gallinule, Porphyrio porphyrio",
+    "137": "American coot, marsh hen, mud hen, water hen, Fulica americana",
+    "138": "bustard",
+    "139": "ruddy turnstone, Arenaria interpres",
+    "140": "red-backed sandpiper, dunlin, Erolia alpina",
+    "141": "redshank, Tringa totanus",
+    "142": "dowitcher",
+    "143": "oystercatcher, oyster catcher",
+    "144": "pelican",
+    "145": "king penguin, Aptenodytes patagonica",
+    "146": "albatross, mollymawk",
+    "147": "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus",
+    "148": "killer whale, killer, orca, grampus, sea wolf, Orcinus orca",
+    "149": "dugong, Dugong dugon",
+    "150": "sea lion",
+    "151": "Chihuahua",
+    "152": "Japanese spaniel",
+    "153": "Maltese dog, Maltese terrier, Maltese",
+    "154": "Pekinese, Pekingese, Peke",
+    "155": "Shih-Tzu",
+    "156": "Blenheim spaniel",
+    "157": "papillon",
+    "158": "toy terrier",
+    "159": "Rhodesian ridgeback",
+    "160": "Afghan hound, Afghan",
+    "161": "basset, basset hound",
+    "162": "beagle",
+    "163": "bloodhound, sleuthhound",
+    "164": "bluetick",
+    "165": "black-and-tan coonhound",
+    "166": "Walker hound, Walker foxhound",
+    "167": "English foxhound",
+    "168": "redbone",
+    "169": "borzoi, Russian wolfhound",
+    "170": "Irish wolfhound",
+    "171": "Italian greyhound",
+    "172": "whippet",
+    "173": "Ibizan hound, Ibizan Podenco",
+    "174": "Norwegian elkhound, elkhound",
+    "175": "otterhound, otter hound",
+    "176": "Saluki, gazelle hound",
+    "177": "Scottish deerhound, deerhound",
+    "178": "Weimaraner",
+    "179": "Staffordshire bullterrier, Staffordshire bull terrier",
+    "180": "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier",
+    "181": "Bedlington terrier",
+    "182": "Border terrier",
+    "183": "Kerry blue terrier",
+    "184": "Irish terrier",
+    "185": "Norfolk terrier",
+    "186": "Norwich terrier",
+    "187": "Yorkshire terrier",
+    "188": "wire-haired fox terrier",
+    "189": "Lakeland terrier",
+    "190": "Sealyham terrier, Sealyham",
+    "191": "Airedale, Airedale terrier",
+    "192": "cairn, cairn terrier",
+    "193": "Australian terrier",
+    "194": "Dandie Dinmont, Dandie Dinmont terrier",
+    "195": "Boston bull, Boston terrier",
+    "196": "miniature schnauzer",
+    "197": "giant schnauzer",
+    "198": "standard schnauzer",
+    "199": "Scotch terrier, Scottish terrier, Scottie",
+    "200": "Tibetan terrier, chrysanthemum dog",
+    "201": "silky terrier, Sydney silky",
+    "202": "soft-coated wheaten terrier",
+    "203": "West Highland white terrier",
+    "204": "Lhasa, Lhasa apso",
+    "205": "flat-coated retriever",
+    "206": "curly-coated retriever",
+    "207": "golden retriever",
+    "208": "Labrador retriever",
+    "209": "Chesapeake Bay retriever",
+    "210": "German short-haired pointer",
+    "211": "vizsla, Hungarian pointer",
+    "212": "English setter",
+    "213": "Irish setter, red setter",
+    "214": "Gordon setter",
+    "215": "Brittany spaniel",
+    "216": "clumber, clumber spaniel",
+    "217": "English springer, English springer spaniel",
+    "218": "Welsh springer spaniel",
+    "219": "cocker spaniel, English cocker spaniel, cocker",
+    "220": "Sussex spaniel",
+    "221": "Irish water spaniel",
+    "222": "kuvasz",
+    "223": "schipperke",
+    "224": "groenendael",
+    "225": "malinois",
+    "226": "briard",
+    "227": "kelpie",
+    "228": "komondor",
+    "229": "Old English sheepdog, bobtail",
+    "230": "Shetland sheepdog, Shetland sheep dog, Shetland",
+    "231": "collie",
+    "232": "Border collie",
+    "233": "Bouvier des Flandres, Bouviers des Flandres",
+    "234": "Rottweiler",
+    "235": "German shepherd, German shepherd dog, German police dog, alsatian",
+    "236": "Doberman, Doberman pinscher",
+    "237": "miniature pinscher",
+    "238": "Greater Swiss Mountain dog",
+    "239": "Bernese mountain dog",
+    "240": "Appenzeller",
+    "241": "EntleBucher",
+    "242": "boxer",
+    "243": "bull mastiff",
+    "244": "Tibetan mastiff",
+    "245": "French bulldog",
+    "246": "Great Dane",
+    "247": "Saint Bernard, St Bernard",
+    "248": "Eskimo dog, husky",
+    "249": "malamute, malemute, Alaskan malamute",
+    "250": "Siberian husky",
+    "251": "dalmatian, coach dog, carriage dog",
+    "252": "affenpinscher, monkey pinscher, monkey dog",
+    "253": "basenji",
+    "254": "pug, pug-dog",
+    "255": "Leonberg",
+    "256": "Newfoundland, Newfoundland dog",
+    "257": "Great Pyrenees",
+    "258": "Samoyed, Samoyede",
+    "259": "Pomeranian",
+    "260": "chow, chow chow",
+    "261": "keeshond",
+    "262": "Brabancon griffon",
+    "263": "Pembroke, Pembroke Welsh corgi",
+    "264": "Cardigan, Cardigan Welsh corgi",
+    "265": "toy poodle",
+    "266": "miniature poodle",
+    "267": "standard poodle",
+    "268": "Mexican hairless",
+    "269": "timber wolf, grey wolf, gray wolf, Canis lupus",
+    "270": "white wolf, Arctic wolf, Canis lupus tundrarum",
+    "271": "red wolf, maned wolf, Canis rufus, Canis niger",
+    "272": "coyote, prairie wolf, brush wolf, Canis latrans",
+    "273": "dingo, warrigal, warragal, Canis dingo",
+    "274": "dhole, Cuon alpinus",
+    "275": "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus",
+    "276": "hyena, hyaena",
+    "277": "red fox, Vulpes vulpes",
+    "278": "kit fox, Vulpes macrotis",
+    "279": "Arctic fox, white fox, Alopex lagopus",
+    "280": "grey fox, gray fox, Urocyon cinereoargenteus",
+    "281": "tabby, tabby cat",
+    "282": "tiger cat",
+    "283": "Persian cat",
+    "284": "Siamese cat, Siamese",
+    "285": "Egyptian cat",
+    "286": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
+    "287": "lynx, catamount",
+    "288": "leopard, Panthera pardus",
+    "289": "snow leopard, ounce, Panthera uncia",
+    "290": "jaguar, panther, Panthera onca, Felis onca",
+    "291": "lion, king of beasts, Panthera leo",
+    "292": "tiger, Panthera tigris",
+    "293": "cheetah, chetah, Acinonyx jubatus",
+    "294": "brown bear, bruin, Ursus arctos",
+    "295": "American black bear, black bear, Ursus americanus, Euarctos americanus",
+    "296": "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus",
+    "297": "sloth bear, Melursus ursinus, Ursus ursinus",
+    "298": "mongoose",
+    "299": "meerkat, mierkat",
+    "300": "tiger beetle",
+    "301": "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle",
+    "302": "ground beetle, carabid beetle",
+    "303": "long-horned beetle, longicorn, longicorn beetle",
+    "304": "leaf beetle, chrysomelid",
+    "305": "dung beetle",
+    "306": "rhinoceros beetle",
+    "307": "weevil",
+    "308": "fly",
+    "309": "bee",
+    "310": "ant, emmet, pismire",
+    "311": "grasshopper, hopper",
+    "312": "cricket",
+    "313": "walking stick, walkingstick, stick insect",
+    "314": "cockroach, roach",
+    "315": "mantis, mantid",
+    "316": "cicada, cicala",
+    "317": "leafhopper",
+    "318": "lacewing, lacewing fly",
+    "319": "dragonfly, darning needle, devils darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+    "320": "damselfly",
+    "321": "admiral",
+    "322": "ringlet, ringlet butterfly",
+    "323": "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus",
+    "324": "cabbage butterfly",
+    "325": "sulphur butterfly, sulfur butterfly",
+    "326": "lycaenid, lycaenid butterfly",
+    "327": "starfish, sea star",
+    "328": "sea urchin",
+    "329": "sea cucumber, holothurian",
+    "330": "wood rabbit, cottontail, cottontail rabbit",
+    "331": "hare",
+    "332": "Angora, Angora rabbit",
+    "333": "hamster",
+    "334": "porcupine, hedgehog",
+    "335": "fox squirrel, eastern fox squirrel, Sciurus niger",
+    "336": "marmot",
+    "337": "beaver",
+    "338": "guinea pig, Cavia cobaya",
+    "339": "sorrel",
+    "340": "zebra",
+    "341": "hog, pig, grunter, squealer, Sus scrofa",
+    "342": "wild boar, boar, Sus scrofa",
+    "343": "warthog",
+    "344": "hippopotamus, hippo, river horse, Hippopotamus amphibius",
+    "345": "ox",
+    "346": "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis",
+    "347": "bison",
+    "348": "ram, tup",
+    "349": "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis",
+    "350": "ibex, Capra ibex",
+    "351": "hartebeest",
+    "352": "impala, Aepyceros melampus",
+    "353": "gazelle",
+    "354": "Arabian camel, dromedary, Camelus dromedarius",
+    "355": "llama",
+    "356": "weasel",
+    "357": "mink",
+    "358": "polecat, fitch, foulmart, foumart, Mustela putorius",
+    "359": "black-footed ferret, ferret, Mustela nigripes",
+    "360": "otter",
+    "361": "skunk, polecat, wood pussy",
+    "362": "badger",
+    "363": "armadillo",
+    "364": "three-toed sloth, ai, Bradypus tridactylus",
+    "365": "orangutan, orang, orangutang, Pongo pygmaeus",
+    "366": "gorilla, Gorilla gorilla",
+    "367": "chimpanzee, chimp, Pan troglodytes",
+    "368": "gibbon, Hylobates lar",
+    "369": "siamang, Hylobates syndactylus, Symphalangus syndactylus",
+    "370": "guenon, guenon monkey",
+    "371": "patas, hussar monkey, Erythrocebus patas",
+    "372": "baboon",
+    "373": "macaque",
+    "374": "langur",
+    "375": "colobus, colobus monkey",
+    "376": "proboscis monkey, Nasalis larvatus",
+    "377": "marmoset",
+    "378": "capuchin, ringtail, Cebus capucinus",
+    "379": "howler monkey, howler",
+    "380": "titi, titi monkey",
+    "381": "spider monkey, Ateles geoffroyi",
+    "382": "squirrel monkey, Saimiri sciureus",
+    "383": "Madagascar cat, ring-tailed lemur, Lemur catta",
+    "384": "indri, indris, Indri indri, Indri brevicaudatus",
+    "385": "Indian elephant, Elephas maximus",
+    "386": "African elephant, Loxodonta africana",
+    "387": "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens",
+    "388": "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca",
+    "389": "barracouta, snoek",
+    "390": "eel",
+    "391": "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch",
+    "392": "rock beauty, Holocanthus tricolor",
+    "393": "anemone fish",
+    "394": "sturgeon",
+    "395": "gar, garfish, garpike, billfish, Lepisosteus osseus",
+    "396": "lionfish",
+    "397": "puffer, pufferfish, blowfish, globefish",
+    "398": "abacus",
+    "399": "abaya",
+    "400": "academic gown, academic robe, judge robe",
+    "401": "accordion, piano accordion, squeeze box",
+    "402": "acoustic guitar",
+    "403": "aircraft carrier, carrier, flattop, attack aircraft carrier",
+    "404": "airliner",
+    "405": "airship, dirigible",
+    "406": "altar",
+    "407": "ambulance",
+    "408": "amphibian, amphibious vehicle",
+    "409": "analog clock",
+    "410": "apiary, bee house",
+    "411": "apron",
+    "412": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
+    "413": "assault rifle, assault gun",
+    "414": "backpack, back pack, knapsack, packsack, rucksack, haversack",
+    "415": "bakery, bakeshop, bakehouse",
+    "416": "balance beam, beam",
+    "417": "balloon",
+    "418": "ballpoint, ballpoint pen, ballpen, Biro",
+    "419": "Band Aid",
+    "420": "banjo",
+    "421": "bannister, banister, balustrade, balusters, handrail",
+    "422": "barbell",
+    "423": "barber chair",
+    "424": "barbershop",
+    "425": "barn",
+    "426": "barometer",
+    "427": "barrel, cask",
+    "428": "barrow, garden cart, lawn cart, wheelbarrow",
+    "429": "baseball",
+    "430": "basketball",
+    "431": "bassinet",
+    "432": "bassoon",
+    "433": "bathing cap, swimming cap",
+    "434": "bath towel",
+    "435": "bathtub, bathing tub, bath, tub",
+    "436": "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon",
+    "437": "beacon, lighthouse, beacon light, pharos",
+    "438": "beaker",
+    "439": "bearskin, busby, shako",
+    "440": "beer bottle",
+    "441": "beer glass",
+    "442": "bell cote, bell cot",
+    "443": "bib",
+    "444": "bicycle-built-for-two, tandem bicycle, tandem",
+    "445": "bikini, two-piece",
+    "446": "binder, ring-binder",
+    "447": "binoculars, field glasses, opera glasses",
+    "448": "birdhouse",
+    "449": "boathouse",
+    "450": "bobsled, bobsleigh, bob",
+    "451": "bolo tie, bolo, bola tie, bola",
+    "452": "bonnet, poke bonnet",
+    "453": "bookcase",
+    "454": "bookshop, bookstore, bookstall",
+    "455": "bottlecap",
+    "456": "bow",
+    "457": "bow tie, bow-tie, bowtie",
+    "458": "brass, memorial tablet, plaque",
+    "459": "brassiere, bra, bandeau",
+    "460": "breakwater, groin, groyne, mole, bulwark, seawall, jetty",
+    "461": "breastplate, aegis, egis",
+    "462": "broom",
+    "463": "bucket, pail",
+    "464": "buckle",
+    "465": "bulletproof vest",
+    "466": "bullet train, bullet",
+    "467": "butcher shop, meat market",
+    "468": "cab, hack, taxi, taxicab",
+    "469": "caldron, cauldron",
+    "470": "candle, taper, wax light",
+    "471": "cannon",
+    "472": "canoe",
+    "473": "can opener, tin opener",
+    "474": "cardigan",
+    "475": "car mirror",
+    "476": "carousel, carrousel, merry-go-round, roundabout, whirligig",
+    "477": "carpenters kit, tool kit",
+    "478": "carton",
+    "479": "car wheel",
+    "480": "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
+    "481": "cassette",
+    "482": "cassette player",
+    "483": "castle",
+    "484": "catamaran",
+    "485": "CD player",
+    "486": "cello, violoncello",
+    "487": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
+    "488": "chain",
+    "489": "chainlink fence",
+    "490": "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour",
+    "491": "chain saw, chainsaw",
+    "492": "chest",
+    "493": "chiffonier, commode",
+    "494": "chime, bell, gong",
+    "495": "china cabinet, china closet",
+    "496": "Christmas stocking",
+    "497": "church, church building",
+    "498": "cinema, movie theater, movie theatre, movie house, picture palace",
+    "499": "cleaver, meat cleaver, chopper",
+    "500": "cliff dwelling",
+    "501": "cloak",
+    "502": "clog, geta, patten, sabot",
+    "503": "cocktail shaker",
+    "504": "coffee mug",
+    "505": "coffeepot",
+    "506": "coil, spiral, volute, whorl, helix",
+    "507": "combination lock",
+    "508": "computer keyboard, keypad",
+    "509": "confectionery, confectionary, candy store",
+    "510": "container ship, containership, container vessel",
+    "511": "convertible",
+    "512": "corkscrew, bottle screw",
+    "513": "cornet, horn, trumpet, trump",
+    "514": "cowboy boot",
+    "515": "cowboy hat, ten-gallon hat",
+    "516": "cradle",
+    "517": "crane",
+    "518": "crash helmet",
+    "519": "crate",
+    "520": "crib, cot",
+    "521": "Crock Pot",
+    "522": "croquet ball",
+    "523": "crutch",
+    "524": "cuirass",
+    "525": "dam, dike, dyke",
+    "526": "desk",
+    "527": "desktop computer",
+    "528": "dial telephone, dial phone",
+    "529": "diaper, nappy, napkin",
+    "530": "digital clock",
+    "531": "digital watch",
+    "532": "dining table, board",
+    "533": "dishrag, dishcloth",
+    "534": "dishwasher, dish washer, dishwashing machine",
+    "535": "disk brake, disc brake",
+    "536": "dock, dockage, docking facility",
+    "537": "dogsled, dog sled, dog sleigh",
+    "538": "dome",
+    "539": "doormat, welcome mat",
+    "540": "drilling platform, offshore rig",
+    "541": "drum, membranophone, tympan",
+    "542": "drumstick",
+    "543": "dumbbell",
+    "544": "Dutch oven",
+    "545": "electric fan, blower",
+    "546": "electric guitar",
+    "547": "electric locomotive",
+    "548": "entertainment center",
+    "549": "envelope",
+    "550": "espresso maker",
+    "551": "face powder",
+    "552": "feather boa, boa",
+    "553": "file, file cabinet, filing cabinet",
+    "554": "fireboat",
+    "555": "fire engine, fire truck",
+    "556": "fire screen, fireguard",
+    "557": "flagpole, flagstaff",
+    "558": "flute, transverse flute",
+    "559": "folding chair",
+    "560": "football helmet",
+    "561": "forklift",
+    "562": "fountain",
+    "563": "fountain pen",
+    "564": "four-poster",
+    "565": "freight car",
+    "566": "French horn, horn",
+    "567": "frying pan, frypan, skillet",
+    "568": "fur coat",
+    "569": "garbage truck, dustcart",
+    "570": "gasmask, respirator, gas helmet",
+    "571": "gas pump, gasoline pump, petrol pump, island dispenser",
+    "572": "goblet",
+    "573": "go-kart",
+    "574": "golf ball",
+    "575": "golfcart, golf cart",
+    "576": "gondola",
+    "577": "gong, tam-tam",
+    "578": "gown",
+    "579": "grand piano, grand",
+    "580": "greenhouse, nursery, glasshouse",
+    "581": "grille, radiator grille",
+    "582": "grocery store, grocery, food market, market",
+    "583": "guillotine",
+    "584": "hair slide",
+    "585": "hair spray",
+    "586": "half track",
+    "587": "hammer",
+    "588": "hamper",
+    "589": "hand blower, blow dryer, blow drier, hair dryer, hair drier",
+    "590": "hand-held computer, hand-held microcomputer",
+    "591": "handkerchief, hankie, hanky, hankey",
+    "592": "hard disc, hard disk, fixed disk",
+    "593": "harmonica, mouth organ, harp, mouth harp",
+    "594": "harp",
+    "595": "harvester, reaper",
+    "596": "hatchet",
+    "597": "holster",
+    "598": "home theater, home theatre",
+    "599": "honeycomb",
+    "600": "hook, claw",
+    "601": "hoopskirt, crinoline",
+    "602": "horizontal bar, high bar",
+    "603": "horse cart, horse-cart",
+    "604": "hourglass",
+    "605": "iPod",
+    "606": "iron, smoothing iron",
+    "607": "jack-o-lantern",
+    "608": "jean, blue jean, denim",
+    "609": "jeep, landrover",
+    "610": "jersey, T-shirt, tee shirt",
+    "611": "jigsaw puzzle",
+    "612": "jinrikisha, ricksha, rickshaw",
+    "613": "joystick",
+    "614": "kimono",
+    "615": "knee pad",
+    "616": "knot",
+    "617": "lab coat, laboratory coat",
+    "618": "ladle",
+    "619": "lampshade, lamp shade",
+    "620": "laptop, laptop computer",
+    "621": "lawn mower, mower",
+    "622": "lens cap, lens cover",
+    "623": "letter opener, paper knife, paperknife",
+    "624": "library",
+    "625": "lifeboat",
+    "626": "lighter, light, igniter, ignitor",
+    "627": "limousine, limo",
+    "628": "liner, ocean liner",
+    "629": "lipstick, lip rouge",
+    "630": "Loafer",
+    "631": "lotion",
+    "632": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
+    "633": "loupe, jewelers loupe",
+    "634": "lumbermill, sawmill",
+    "635": "magnetic compass",
+    "636": "mailbag, postbag",
+    "637": "mailbox, letter box",
+    "638": "maillot",
+    "639": "maillot, tank suit",
+    "640": "manhole cover",
+    "641": "maraca",
+    "642": "marimba, xylophone",
+    "643": "mask",
+    "644": "matchstick",
+    "645": "maypole",
+    "646": "maze, labyrinth",
+    "647": "measuring cup",
+    "648": "medicine chest, medicine cabinet",
+    "649": "megalith, megalithic structure",
+    "650": "microphone, mike",
+    "651": "microwave, microwave oven",
+    "652": "military uniform",
+    "653": "milk can",
+    "654": "minibus",
+    "655": "miniskirt, mini",
+    "656": "minivan",
+    "657": "missile",
+    "658": "mitten",
+    "659": "mixing bowl",
+    "660": "mobile home, manufactured home",
+    "661": "Model T",
+    "662": "modem",
+    "663": "monastery",
+    "664": "monitor",
+    "665": "moped",
+    "666": "mortar",
+    "667": "mortarboard",
+    "668": "mosque",
+    "669": "mosquito net",
+    "670": "motor scooter, scooter",
+    "671": "mountain bike, all-terrain bike, off-roader",
+    "672": "mountain tent",
+    "673": "mouse, computer mouse",
+    "674": "mousetrap",
+    "675": "moving van",
+    "676": "muzzle",
+    "677": "nail",
+    "678": "neck brace",
+    "679": "necklace",
+    "680": "nipple",
+    "681": "notebook, notebook computer",
+    "682": "obelisk",
+    "683": "oboe, hautboy, hautbois",
+    "684": "ocarina, sweet potato",
+    "685": "odometer, hodometer, mileometer, milometer",
+    "686": "oil filter",
+    "687": "organ, pipe organ",
+    "688": "oscilloscope, scope, cathode-ray oscilloscope, CRO",
+    "689": "overskirt",
+    "690": "oxcart",
+    "691": "oxygen mask",
+    "692": "packet",
+    "693": "paddle, boat paddle",
+    "694": "paddlewheel, paddle wheel",
+    "695": "padlock",
+    "696": "paintbrush",
+    "697": "pajama, pyjama, pjs, jammies",
+    "698": "palace",
+    "699": "panpipe, pandean pipe, syrinx",
+    "700": "paper towel",
+    "701": "parachute, chute",
+    "702": "parallel bars, bars",
+    "703": "park bench",
+    "704": "parking meter",
+    "705": "passenger car, coach, carriage",
+    "706": "patio, terrace",
+    "707": "pay-phone, pay-station",
+    "708": "pedestal, plinth, footstall",
+    "709": "pencil box, pencil case",
+    "710": "pencil sharpener",
+    "711": "perfume, essence",
+    "712": "Petri dish",
+    "713": "photocopier",
+    "714": "pick, plectrum, plectron",
+    "715": "pickelhaube",
+    "716": "picket fence, paling",
+    "717": "pickup, pickup truck",
+    "718": "pier",
+    "719": "piggy bank, penny bank",
+    "720": "pill bottle",
+    "721": "pillow",
+    "722": "ping-pong ball",
+    "723": "pinwheel",
+    "724": "pirate, pirate ship",
+    "725": "pitcher, ewer",
+    "726": "plane, carpenters plane, woodworking plane",
+    "727": "planetarium",
+    "728": "plastic bag",
+    "729": "plate rack",
+    "730": "plow, plough",
+    "731": "plunger, plumbers helper",
+    "732": "Polaroid camera, Polaroid Land camera",
+    "733": "pole",
+    "734": "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria",
+    "735": "poncho",
+    "736": "pool table, billiard table, snooker table",
+    "737": "pop bottle, soda bottle",
+    "738": "pot, flowerpot",
+    "739": "potters wheel",
+    "740": "power drill",
+    "741": "prayer rug, prayer mat",
+    "742": "printer",
+    "743": "prison, prison house",
+    "744": "projectile, missile",
+    "745": "projector",
+    "746": "puck, hockey puck",
+    "747": "punching bag, punch bag, punching ball, punchball",
+    "748": "purse",
+    "749": "quill, quill pen",
+    "750": "quilt, comforter, comfort, puff",
+    "751": "racer, race car, racing car",
+    "752": "racket, racquet",
+    "753": "radiator",
+    "754": "radio, wireless",
+    "755": "radio telescope, radio reflector",
+    "756": "rain barrel",
+    "757": "recreational vehicle, RV, R.V.",
+    "758": "reel",
+    "759": "reflex camera",
+    "760": "refrigerator, icebox",
+    "761": "remote control, remote",
+    "762": "restaurant, eating house, eating place, eatery",
+    "763": "revolver, six-gun, six-shooter",
+    "764": "rifle",
+    "765": "rocking chair, rocker",
+    "766": "rotisserie",
+    "767": "rubber eraser, rubber, pencil eraser",
+    "768": "rugby ball",
+    "769": "rule, ruler",
+    "770": "running shoe",
+    "771": "safe",
+    "772": "safety pin",
+    "773": "saltshaker, salt shaker",
+    "774": "sandal",
+    "775": "sarong",
+    "776": "sax, saxophone",
+    "777": "scabbard",
+    "778": "scale, weighing machine",
+    "779": "school bus",
+    "780": "schooner",
+    "781": "scoreboard",
+    "782": "screen, CRT screen",
+    "783": "screw",
+    "784": "screwdriver",
+    "785": "seat belt, seatbelt",
+    "786": "sewing machine",
+    "787": "shield, buckler",
+    "788": "shoe shop, shoe-shop, shoe store",
+    "789": "shoji",
+    "790": "shopping basket",
+    "791": "shopping cart",
+    "792": "shovel",
+    "793": "shower cap",
+    "794": "shower curtain",
+    "795": "ski",
+    "796": "ski mask",
+    "797": "sleeping bag",
+    "798": "slide rule, slipstick",
+    "799": "sliding door",
+    "800": "slot, one-armed bandit",
+    "801": "snorkel",
+    "802": "snowmobile",
+    "803": "snowplow, snowplough",
+    "804": "soap dispenser",
+    "805": "soccer ball",
+    "806": "sock",
+    "807": "solar dish, solar collector, solar furnace",
+    "808": "sombrero",
+    "809": "soup bowl",
+    "810": "space bar",
+    "811": "space heater",
+    "812": "space shuttle",
+    "813": "spatula",
+    "814": "speedboat",
+    "815": "spider web, spiders web",
+    "816": "spindle",
+    "817": "sports car, sport car",
+    "818": "spotlight, spot",
+    "819": "stage",
+    "820": "steam locomotive",
+    "821": "steel arch bridge",
+    "822": "steel drum",
+    "823": "stethoscope",
+    "824": "stole",
+    "825": "stone wall",
+    "826": "stopwatch, stop watch",
+    "827": "stove",
+    "828": "strainer",
+    "829": "streetcar, tram, tramcar, trolley, trolley car",
+    "830": "stretcher",
+    "831": "studio couch, day bed",
+    "832": "stupa, tope",
+    "833": "submarine, pigboat, sub, U-boat",
+    "834": "suit, suit of clothes",
+    "835": "sundial",
+    "836": "sunglass",
+    "837": "sunglasses, dark glasses, shades",
+    "838": "sunscreen, sunblock, sun blocker",
+    "839": "suspension bridge",
+    "840": "swab, swob, mop",
+    "841": "sweatshirt",
+    "842": "swimming trunks, bathing trunks",
+    "843": "swing",
+    "844": "switch, electric switch, electrical switch",
+    "845": "syringe",
+    "846": "table lamp",
+    "847": "tank, army tank, armored combat vehicle, armoured combat vehicle",
+    "848": "tape player",
+    "849": "teapot",
+    "850": "teddy, teddy bear",
+    "851": "television, television system",
+    "852": "tennis ball",
+    "853": "thatch, thatched roof",
+    "854": "theater curtain, theatre curtain",
+    "855": "thimble",
+    "856": "thresher, thrasher, threshing machine",
+    "857": "throne",
+    "858": "tile roof",
+    "859": "toaster",
+    "860": "tobacco shop, tobacconist shop, tobacconist",
+    "861": "toilet seat",
+    "862": "torch",
+    "863": "totem pole",
+    "864": "tow truck, tow car, wrecker",
+    "865": "toyshop",
+    "866": "tractor",
+    "867": "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi",
+    "868": "tray",
+    "869": "trench coat",
+    "870": "tricycle, trike, velocipede",
+    "871": "trimaran",
+    "872": "tripod",
+    "873": "triumphal arch",
+    "874": "trolleybus, trolley coach, trackless trolley",
+    "875": "trombone",
+    "876": "tub, vat",
+    "877": "turnstile",
+    "878": "typewriter keyboard",
+    "879": "umbrella",
+    "880": "unicycle, monocycle",
+    "881": "upright, upright piano",
+    "882": "vacuum, vacuum cleaner",
+    "883": "vase",
+    "884": "vault",
+    "885": "velvet",
+    "886": "vending machine",
+    "887": "vestment",
+    "888": "viaduct",
+    "889": "violin, fiddle",
+    "890": "volleyball",
+    "891": "waffle iron",
+    "892": "wall clock",
+    "893": "wallet, billfold, notecase, pocketbook",
+    "894": "wardrobe, closet, press",
+    "895": "warplane, military plane",
+    "896": "washbasin, handbasin, washbowl, lavabo, wash-hand basin",
+    "897": "washer, automatic washer, washing machine",
+    "898": "water bottle",
+    "899": "water jug",
+    "900": "water tower",
+    "901": "whiskey jug",
+    "902": "whistle",
+    "903": "wig",
+    "904": "window screen",
+    "905": "window shade",
+    "906": "Windsor tie",
+    "907": "wine bottle",
+    "908": "wing",
+    "909": "wok",
+    "910": "wooden spoon",
+    "911": "wool, woolen, woollen",
+    "912": "worm fence, snake fence, snake-rail fence, Virginia fence",
+    "913": "wreck",
+    "914": "yawl",
+    "915": "yurt",
+    "916": "web site, website, internet site, site",
+    "917": "comic book",
+    "918": "crossword puzzle, crossword",
+    "919": "street sign",
+    "920": "traffic light, traffic signal, stoplight",
+    "921": "book jacket, dust cover, dust jacket, dust wrapper",
+    "922": "menu",
+    "923": "plate",
+    "924": "guacamole",
+    "925": "consomme",
+    "926": "hot pot, hotpot",
+    "927": "trifle",
+    "928": "ice cream, icecream",
+    "929": "ice lolly, lolly, lollipop, popsicle",
+    "930": "French loaf",
+    "931": "bagel, beigel",
+    "932": "pretzel",
+    "933": "cheeseburger",
+    "934": "hotdog, hot dog, red hot",
+    "935": "mashed potato",
+    "936": "head cabbage",
+    "937": "broccoli",
+    "938": "cauliflower",
+    "939": "zucchini, courgette",
+    "940": "spaghetti squash",
+    "941": "acorn squash",
+    "942": "butternut squash",
+    "943": "cucumber, cuke",
+    "944": "artichoke, globe artichoke",
+    "945": "bell pepper",
+    "946": "cardoon",
+    "947": "mushroom",
+    "948": "Granny Smith",
+    "949": "strawberry",
+    "950": "orange",
+    "951": "lemon",
+    "952": "fig",
+    "953": "pineapple, ananas",
+    "954": "banana",
+    "955": "jackfruit, jak, jack",
+    "956": "custard apple",
+    "957": "pomegranate",
+    "958": "hay",
+    "959": "carbonara",
+    "960": "chocolate sauce, chocolate syrup",
+    "961": "dough",
+    "962": "meat loaf, meatloaf",
+    "963": "pizza, pizza pie",
+    "964": "potpie",
+    "965": "burrito",
+    "966": "red wine",
+    "967": "espresso",
+    "968": "cup",
+    "969": "eggnog",
+    "970": "alp",
+    "971": "bubble",
+    "972": "cliff, drop, drop-off",
+    "973": "coral reef",
+    "974": "geyser",
+    "975": "lakeside, lakeshore",
+    "976": "promontory, headland, head, foreland",
+    "977": "sandbar, sand bar",
+    "978": "seashore, coast, seacoast, sea-coast",
+    "979": "valley, vale",
+    "980": "volcano",
+    "981": "ballplayer, baseball player",
+    "982": "groom, bridegroom",
+    "983": "scuba diver",
+    "984": "rapeseed",
+    "985": "daisy",
+    "986": "yellow ladys slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+    "987": "corn",
+    "988": "acorn",
+    "989": "hip, rose hip, rosehip",
+    "990": "buckeye, horse chestnut, conker",
+    "991": "coral fungus",
+    "992": "agaric",
+    "993": "gyromitra",
+    "994": "stinkhorn, carrion fungus",
+    "995": "earthstar",
+    "996": "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa",
+    "997": "bolete",
+    "998": "ear, spike, capitulum",
+    "999": "toilet tissue, toilet paper, bathroom tissue"
+  }
+}

PixelDiT2-H-16-512/pipeline.py

@@ -0,0 +1,839 @@
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import importlib.util
+import inspect
+import json
+import sys
+from pathlib import Path
+from typing import Any, Callable
+
+import torch
+from diffusers.callbacks import MultiPipelineCallbacks, PipelineCallback
+from transformers import DINOv3ViTModel
+from diffusers.models.modeling_utils import ModelMixin
+from diffusers.pipelines.pipeline_utils import DiffusionPipeline, ImagePipelineOutput
+from diffusers.schedulers.scheduling_utils import SchedulerMixin
+from diffusers.utils import deprecate, is_torch_xla_available, logging, replace_example_docstring
+from diffusers.utils.torch_utils import randn_tensor
+from PIL import Image
+
+if is_torch_xla_available():
+    import torch_xla.core.xla_model as xm
+
+    XLA_AVAILABLE = True
+else:
+    XLA_AVAILABLE = False
+
+logger = logging.get_logger(__name__)
+
+EXAMPLE_DOC_STRING = """
+    Examples:
+        ```py
+        >>> from pathlib import Path
+        >>> import torch
+        >>> from diffusers import DiffusionPipeline
+
+        >>> model_dir = Path("pixeldit2-diffusers/PixelDiT2-H-16-256").resolve()
+        >>> pipe = DiffusionPipeline.from_pretrained(
+        ...     str(model_dir),
+        ...     local_files_only=True,
+        ...     custom_pipeline=str(model_dir / "pipeline.py"),
+        ...     trust_remote_code=True,
+        ...     torch_dtype=torch.bfloat16,
+        ... ).to("cuda:0")
+
+        >>> image = pipe(
+        ...     class_labels="golden retriever",
+        ...     num_inference_steps=50,
+        ...     guidance_scale=2.1,
+        ...     generator=torch.Generator(device="cuda:0").manual_seed(0),
+        ... ).images[0]
+        >>> image.save("sample.png")
+
+        >>> # Hugging Face Hub style model id: UserID/RepoID
+        >>> # RepoID is usually like "modelname-diffusers"
+        >>> # Example: "BiliSakura/PixelDiT2-diffusers" with subfolder "PixelDiT2-H-16-256"
+        ```
+"""
+
+
+class PixelDiT2Pipeline(DiffusionPipeline):
+    r"""
+    Class-conditional image pipeline for PixelDiT2.
+
+    PixelDiT2 denoises directly in RGB pixel space (no VAE). The transformer predicts the clean
+    image `x0`. [`PixelDiT2Scheduler`] integrates that prediction on `linspace(0, 1, steps + 1)`:
+    Heun on every interval except the last, which is Euler. `solver="euler"` uses the same knots
+    with one evaluation per interval.
+
+    Classifier-free guidance mixes a grounded class-conditional forward with an ungrounded null-class
+    forward. Guidance is applied only while the flow time is strictly greater than
+    `guidance_interval_min` and strictly less than `guidance_interval_max`.
+
+    The pipeline inherits from [`DiffusionPipeline`]. Check the superclass documentation for the
+    generic methods (download, save, device placement, CPU offload).
+
+    Args:
+        transformer ([`ModelMixin`]):
+            PixelDiT2 denoiser (`PixelDiT2Transformer2DModel`) that predicts `x0` in pixel space.
+        scheduler ([`PixelDiT2Scheduler`]):
+            Flow scheduler. The paper setting is Heun, `t_eps=0.05`. Loaded from `scheduler/`.
+        encoder (`DINOv3ViTModel`, *optional*):
+            Original DINOv3 checkpoint shipped in `encoder/`. The 256px variant includes DINOv3-S/16.
+            Sampling uses the mapped copy stored inside the transformer, so this module is not called.
+        id2label (`dict[int, str]`, *optional*):
+            ImageNet class id to English label mapping. Values may contain comma-separated synonyms.
+            Loaded from `model_index.json` when omitted.
+
+        Sampling defaults are not pipeline config. When `__call__` omits them, they are read from
+        `transformer.config`: `default_num_inference_steps`, `guidance_scale`, `guidance_interval_min`,
+        `guidance_interval_max`, `noise_scale`, and `t_eps`. Missing transformer entries fall back to
+        50 steps, CFG 2.4 on `(0.125, 0.9)`, `noise_scale` 1.0, and `t_eps` 0.05.
+    """
+
+    _optional_components = ["encoder"]
+    _callback_tensor_inputs = ["latents"]
+
+    def __init__(
+        self,
+        transformer: ModelMixin,
+        scheduler: SchedulerMixin,
+        encoder: DINOv3ViTModel | None = None,
+        id2label: dict[int, str] | dict[str, str] | None = None,
+    ):
+        super().__init__()
+        if scheduler is None:
+            scheduler = self._default_scheduler()
+
+        self.register_modules(transformer=transformer, scheduler=scheduler, encoder=encoder)
+        normalized_labels = self._normalize_id2label(id2label)
+        self.register_to_config(id2label={str(key): value for key, value in normalized_labels.items()})
+        self._id2label = normalized_labels
+        self.labels = self._build_label2id(self._id2label)
+        self._labels_loaded_from_model_index = bool(self._id2label)
+
+    def _sampling_default(self, name: str, fallback: float) -> float:
+        r"""
+        Read a sampling default from the transformer config.
+
+        Args:
+            name (`str`):
+                Key on `transformer.config`, such as `guidance_scale` or `noise_scale`.
+            fallback (`float`):
+                Value used when the transformer config does not define `name`.
+
+        Returns:
+            `float`: The configured value, or `fallback`.
+        """
+        value = getattr(self.transformer.config, name, None)
+        if value is None:
+            return fallback
+        return value
+
+    @staticmethod
+    def _default_scheduler() -> SchedulerMixin:
+        r"""
+        Build [`PixelDiT2Scheduler`] from the variant's scheduler module.
+
+        Returns:
+            [`PixelDiT2Scheduler`]: Heun on the paper linspace, `t_eps=0.05`.
+        """
+        module_path = Path(__file__).resolve().parent / "scheduler" / "scheduling_pixeldit2.py"
+        spec = importlib.util.spec_from_file_location("scheduling_pixeldit2", module_path)
+        if spec is None or spec.loader is None:
+            raise ImportError(f"Could not load the PixelDiT2 scheduler from {module_path}.")
+        module = importlib.util.module_from_spec(spec)
+        sys.modules["scheduling_pixeldit2"] = module
+        spec.loader.exec_module(module)
+        return module.PixelDiT2Scheduler()
+
+    def _ensure_labels_loaded(self) -> None:
+        r"""
+        Fill `id2label` from `model_index.json` when the constructor did not receive it.
+        """
+        if self._labels_loaded_from_model_index:
+            return
+        loaded = self._read_id2label_from_model_index(getattr(self.config, "_name_or_path", None))
+        if loaded:
+            self._id2label = loaded
+            self.labels = self._build_label2id(self._id2label)
+        self._labels_loaded_from_model_index = True
+
+    @staticmethod
+    def _normalize_id2label(id2label: dict[int, str] | dict[str, str] | None) -> dict[int, str]:
+        r"""
+        Cast label-map keys to integers.
+
+        Args:
+            id2label (`dict`, *optional*):
+                Class id to English label. Keys may be strings, which is how JSON stores them.
+
+        Returns:
+            `dict[int, str]`: Integer-keyed map. Empty when `id2label` is missing.
+        """
+        if not id2label:
+            return {}
+        return {int(key): value for key, value in id2label.items()}
+
+    @staticmethod
+    def _read_id2label_from_model_index(variant_path: str | None) -> dict[int, str]:
+        r"""
+        Read `id2label` from a local variant directory.
+
+        Args:
+            variant_path (`str`, *optional*):
+                Directory that contains `model_index.json`.
+
+        Returns:
+            `dict[int, str]`: Loaded map, or an empty dict when the file or key is absent.
+        """
+        if not variant_path:
+            return {}
+        model_index_path = Path(variant_path).resolve() / "model_index.json"
+        if not model_index_path.exists():
+            return {}
+        raw = json.loads(model_index_path.read_text(encoding="utf-8"))
+        id2label = raw.get("id2label")
+        if not isinstance(id2label, dict):
+            return {}
+        return {int(key): value for key, value in id2label.items()}
+
+    @staticmethod
+    def _build_label2id(id2label: dict[int, str]) -> dict[str, int]:
+        r"""
+        Invert `id2label`, splitting comma-separated synonyms.
+
+        Args:
+            id2label (`dict[int, str]`):
+                Class id to one or more English names.
+
+        Returns:
+            `dict[str, int]`: Synonym to class id, sorted by synonym.
+        """
+        label2id: dict[str, int] = {}
+        for class_id, value in id2label.items():
+            for synonym in value.split(","):
+                synonym = synonym.strip()
+                if synonym:
+                    label2id[synonym] = int(class_id)
+        return dict(sorted(label2id.items()))
+
+    @property
+    def id2label(self) -> dict[int, str]:
+        r"""
+        ImageNet id to English label map.
+
+        Returns:
+            `dict[int, str]`: The map loaded from the constructor or `model_index.json`.
+        """
+        self._ensure_labels_loaded()
+        return self._id2label
+
+    def get_label_ids(self, label: str | list[str]) -> list[int]:
+        r"""
+        Map ImageNet English labels to class ids.
+
+        Args:
+            label (`str` or `list[str]`):
+                One label or a list of labels. Each string must match a synonym in `id2label`
+                after stripping whitespace. Synonyms are the comma-separated pieces of each value.
+
+        Returns:
+            `list[int]`: Class ids in the same order as `label`.
+        """
+        self._ensure_labels_loaded()
+        label2id = self.labels
+        if not label2id:
+            raise ValueError("No English labels loaded. Ensure `id2label` exists in model_index.json.")
+        if isinstance(label, str):
+            label = [label]
+        missing = [item for item in label if item not in label2id]
+        if missing:
+            preview = ", ".join(list(label2id.keys())[:8])
+            raise ValueError(f"Unknown English label(s): {missing}. Example valid labels: {preview}, ...")
+        return [label2id[item] for item in label]
+
+    def _normalize_class_labels(self, class_labels: int | str | list[int | str]) -> list[int]:
+        r"""
+        Turn a class-label argument into a list of integer ids.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                One id, one English label, or a list of either. A list must not mix types.
+
+        Returns:
+            `list[int]`: One id per requested image.
+        """
+        if isinstance(class_labels, int):
+            return [class_labels]
+        if isinstance(class_labels, str):
+            return self.get_label_ids(class_labels)
+        if len(class_labels) == 0:
+            raise ValueError("`class_labels` must contain at least one class id or label.")
+        if isinstance(class_labels[0], str):
+            if not all(isinstance(item, str) for item in class_labels):
+                raise TypeError("`class_labels` must be all strings or all integers.")
+            return self.get_label_ids(class_labels)
+        if not all(isinstance(item, int) for item in class_labels):
+            raise TypeError("`class_labels` must be all strings or all integers.")
+        return list(class_labels)
+
+    def prepare_extra_step_kwargs(
+        self,
+        generator: torch.Generator | list[torch.Generator] | None,
+        eta: float,
+    ) -> dict[str, Any]:
+        r"""
+        Extra arguments forwarded to `scheduler.step`, depending on what that method accepts.
+
+        Args:
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                Generator passed through when the scheduler step samples noise.
+            eta (`float`):
+                DDIM eta in `[0, 1]`. Ignored by schedulers whose `step` has no `eta` argument.
+
+        Returns:
+            `dict`: Keyword arguments for `scheduler.step`.
+        """
+        extra_step_kwargs: dict[str, Any] = {}
+        step_params = set(inspect.signature(self.scheduler.step).parameters.keys())
+        if "eta" in step_params:
+            extra_step_kwargs["eta"] = eta
+        if "generator" in step_params:
+            extra_step_kwargs["generator"] = generator
+        return extra_step_kwargs
+
+    def check_inputs(
+        self,
+        class_labels: int | str | list[int | str],
+        height: int,
+        width: int,
+        num_inference_steps: int,
+        guidance_scale: float,
+        guidance_interval_min: float,
+        guidance_interval_max: float,
+        noise_scale: float,
+        t_eps: float,
+        output_type: str,
+        callback_steps: int | None = None,
+        callback_on_step_end_tensor_inputs: list[str] | None = None,
+    ) -> None:
+        r"""
+        Reject arguments that the denoiser or the sampler cannot run.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                Class ids or English labels. A list must be non-empty.
+            height (`int`):
+                Output height in pixels. Must be positive and divisible by the patch size.
+            width (`int`):
+                Output width in pixels. Must be positive and divisible by the patch size.
+            num_inference_steps (`int`):
+                Number of integration intervals. Must be >= 1.
+            guidance_scale (`float`):
+                CFG scale. Must be finite.
+            guidance_interval_min (`float`):
+                Exclusive lower flow-time bound of CFG.
+            guidance_interval_max (`float`):
+                Exclusive upper flow-time bound of CFG. Must be greater than the minimum and at most 1.
+            noise_scale (`float`):
+                Initial Gaussian scale. Must be positive.
+            t_eps (`float`):
+                Velocity denominator floor. Must be positive.
+            output_type (`str`):
+                One of `"pil"`, `"np"`, `"pt"`, `"latent"`.
+            callback_steps (`int`, *optional*):
+                Deprecated callback period. When set, must be a positive integer.
+            callback_on_step_end_tensor_inputs (`list[str]`, *optional*):
+                Tensor names the step callback may read. Each name must be in `_callback_tensor_inputs`.
+        """
+        if isinstance(class_labels, list) and len(class_labels) == 0:
+            raise ValueError("`class_labels` must contain at least one class id or label.")
+        patch_size = int(self.transformer.config.patch_size)
+        if height <= 0 or width <= 0:
+            raise ValueError(f"`height` and `width` must be positive, got {(height, width)}.")
+        if height % patch_size != 0 or width % patch_size != 0:
+            raise ValueError(f"`height` and `width` must be divisible by patch_size={patch_size}. Got {(height, width)}.")
+        if num_inference_steps < 1:
+            raise ValueError(f"`num_inference_steps` must be >= 1, got {num_inference_steps}.")
+        if not torch.isfinite(torch.tensor(guidance_scale)):
+            raise ValueError(f"`guidance_scale` must be finite, got {guidance_scale}.")
+        if not 0.0 <= guidance_interval_min < guidance_interval_max <= 1.0:
+            raise ValueError(
+                "Guidance interval must satisfy 0 <= min < max <= 1, got "
+                f"[{guidance_interval_min}, {guidance_interval_max}]."
+            )
+        if noise_scale <= 0:
+            raise ValueError(f"`noise_scale` must be positive, got {noise_scale}.")
+        if t_eps <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {t_eps}.")
+        if output_type not in {"pil", "np", "pt", "latent"}:
+            raise ValueError(f"Unsupported `output_type` {output_type!r}. Choose from 'pil', 'np', 'pt', 'latent'.")
+        if callback_steps is not None and (not isinstance(callback_steps, int) or callback_steps < 1):
+            raise ValueError(f"`callback_steps` must be a positive integer, got {callback_steps}.")
+        if callback_on_step_end_tensor_inputs is not None:
+            unknown = [name for name in callback_on_step_end_tensor_inputs if name not in self._callback_tensor_inputs]
+            if unknown:
+                raise ValueError(
+                    f"`callback_on_step_end_tensor_inputs` contains {unknown}, which are not in "
+                    f"{self._callback_tensor_inputs}."
+                )
+
+    def prepare_latents(
+        self,
+        batch_size: int,
+        num_channels: int,
+        height: int,
+        width: int,
+        dtype: torch.dtype,
+        device: torch.device,
+        generator: torch.Generator | list[torch.Generator] | None,
+        latents: torch.Tensor | None,
+        noise_scale: float,
+    ) -> torch.Tensor:
+        r"""
+        Sample the initial pixel-space noise, or validate a tensor the caller already sampled.
+
+        PixelDiT2 has no VAE, so these "latents" are RGB images. Fresh noise is multiplied by
+        `noise_scale`. A provided `latents` tensor is not rescaled.
+
+        Args:
+            batch_size (`int`):
+                Number of images.
+            num_channels (`int`):
+                Channel count. 3 for RGB.
+            height (`int`):
+                Image height in pixels.
+            width (`int`):
+                Image width in pixels.
+            dtype (`torch.dtype`):
+                Dtype of freshly sampled noise. The integration itself is accumulated in float32.
+            device (`torch.device`):
+                Device of the returned tensor.
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                Per-call RNG. A list must have length `batch_size`.
+            latents (`torch.Tensor`, *optional*):
+                Starting noise of shape `(batch_size, num_channels, height, width)`.
+            noise_scale (`float`):
+                Scale applied only when `latents` is `None`.
+
+        Returns:
+            `torch.Tensor`: Starting noise of shape `(batch_size, num_channels, height, width)`.
+        """
+        shape = (batch_size, num_channels, height, width)
+        if isinstance(generator, list) and len(generator) != batch_size:
+            raise ValueError(
+                f"You passed a list of {len(generator)} generators for a batch of {batch_size}. "
+                "The two lengths must match."
+            )
+        if latents is None:
+            latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
+            latents = latents * float(noise_scale)
+        else:
+            latents = latents.to(device=device)
+            if tuple(latents.shape) != shape:
+                raise ValueError(f"`latents` shape {tuple(latents.shape)} does not match the expected {shape}.")
+        return latents
+
+    def _prepare_scheduler(
+        self,
+        num_inference_steps: int,
+        device: torch.device,
+        timesteps: list[float] | None,
+        sigmas: list[float] | None,
+    ) -> torch.Tensor:
+        r"""
+        Install the paper linspace on `self.scheduler` and return its flow times.
+
+        [`PixelDiT2Scheduler`] owns the grid. `timesteps` and `sigmas` are rejected. Use
+        `solver="euler"` for one model evaluation per interval.
+
+        Args:
+            num_inference_steps (`int`):
+                Number of flow-time intervals.
+            device (`torch.device`):
+                Device of the scheduler tensors.
+            timesteps (`list[float]`, *optional*):
+                Not used. The scheduler builds `linspace(0, 1, steps + 1)`.
+            sigmas (`list[float]`, *optional*):
+                Not used. PixelDiT2 integrates flow time, not a sigma schedule.
+
+        Returns:
+            `torch.Tensor`: Flow times at which the denoiser is evaluated, in `[0, 1]`.
+        """
+        if timesteps is not None or sigmas is not None:
+            raise ValueError(
+                "PixelDiT2Scheduler builds its own linspace and does not accept custom `timesteps` or "
+                '`sigmas`. Use PixelDiT2Scheduler(solver="euler") for one evaluation per step.'
+            )
+        self.scheduler.set_timesteps(num_inference_steps, device=device)
+        return self.scheduler.timesteps
+
+    def _predict_x0(
+        self,
+        latents: torch.Tensor,
+        flow_time: torch.Tensor,
+        class_labels: torch.Tensor,
+        skip_grounding: bool,
+    ) -> torch.Tensor:
+        r"""
+        Run the denoiser and return an x0 prediction in float32.
+
+        Args:
+            latents (`torch.Tensor`):
+                Current sample `(batch, channels, height, width)`.
+            flow_time (`torch.Tensor`):
+                Scalar flow time in `[0, 1]`, broadcast inside the denoiser.
+            class_labels (`torch.Tensor`):
+                Class ids `(batch,)`.
+            skip_grounding (`bool`):
+                When `True`, skip the DINOv3 encoder. Used for the unconditional branch.
+
+        Returns:
+            `torch.Tensor`: x0 prediction `(batch, channels, height, width)` in float32.
+        """
+        model_input = latents
+        parameter_dtype = self.transformer.dtype
+        device_type = model_input.device.type
+        use_amp = device_type in {"cuda", "xpu"} and parameter_dtype in (torch.float16, torch.bfloat16)
+        if not use_amp:
+            model_input = model_input.to(dtype=parameter_dtype)
+        timesteps = flow_time.to(device=model_input.device, dtype=parameter_dtype).expand(model_input.shape[0])
+        with torch.autocast(device_type if device_type != "mps" else "cpu", dtype=parameter_dtype, enabled=use_amp):
+            prediction = self.transformer(
+                model_input,
+                timestep=timesteps,
+                class_labels=class_labels,
+                skip_grounding=skip_grounding,
+            ).sample
+        return prediction.float()
+
+    def _guided_x0(
+        self,
+        latents: torch.Tensor,
+        flow_time: float,
+        class_labels: torch.Tensor,
+        class_null: torch.Tensor,
+        guidance_scale: float,
+        guidance_interval_min: float,
+        guidance_interval_max: float,
+    ) -> torch.Tensor:
+        r"""
+        Classifier-free guidance on x0, restricted to the paper's time interval.
+
+        Outside the interval the scale is 1, which is exactly the conditional prediction, so the
+        ungrounded forward is skipped.
+
+        Args:
+            latents (`torch.Tensor`):
+                Current sample.
+            flow_time (`float`):
+                Scalar flow time of this evaluation.
+            class_labels (`torch.Tensor`):
+                Conditional class ids.
+            class_null (`torch.Tensor`):
+                Null class ids, one per batch row.
+            guidance_scale (`float`):
+                CFG scale used inside the interval.
+            guidance_interval_min (`float`):
+                Exclusive lower bound.
+            guidance_interval_max (`float`):
+                Exclusive upper bound.
+
+        Returns:
+            `torch.Tensor`: Guided x0, same shape as `latents`, in float32.
+        """
+        flow = torch.tensor(flow_time, device=latents.device, dtype=torch.float32)
+        conditional = self._predict_x0(latents, flow, class_labels, skip_grounding=False)
+        inside = guidance_interval_min < flow_time < guidance_interval_max
+        if guidance_scale == 1.0 or not inside:
+            return conditional
+        unconditional = self._predict_x0(latents, flow, class_null, skip_grounding=True)
+        return unconditional + guidance_scale * (conditional - unconditional)
+
+    @property
+    def guidance_scale(self) -> float:
+        r"""
+        Classifier-free guidance scale of the call that is currently running.
+
+        Returns:
+            `float`: The scale set by the active `__call__`.
+        """
+        return self._guidance_scale
+
+    @property
+    def do_classifier_free_guidance(self) -> bool:
+        r"""
+        Whether the active call can run an unconditional forward.
+
+        The unconditional forward is still skipped at timesteps outside the guidance interval.
+
+        Returns:
+            `bool`: True when `guidance_scale > 1`.
+        """
+        return self._guidance_scale > 1.0
+
+    @property
+    def num_timesteps(self) -> int:
+        r"""
+        Number of scheduler timesteps in the active call.
+
+        Returns:
+            `int`: Length of the timestep schedule, including Heun's repeated states.
+        """
+        return self._num_timesteps
+
+    @property
+    def interrupt(self) -> bool:
+        r"""
+        Whether the active denoising loop should skip remaining steps.
+
+        Returns:
+            `bool`: True after the caller sets `pipeline._interrupt = True`.
+        """
+        return self._interrupt
+
+    def _images_from_latents(self, latents: torch.Tensor, output_type: str) -> torch.Tensor | list[Image.Image]:
+        r"""
+        Convert pixel-space samples to the requested output type.
+
+        Quantization matches the original sampler: `uint8(clip((x + 1) * 127.5 + 0.5, 0, 255))`.
+
+        Args:
+            latents (`torch.Tensor`):
+                Samples of shape `(batch, channels, height, width)` in model range around `[-1, 1]`.
+            output_type (`str`):
+                `"latent"` returns `latents` unchanged. `"pt"` is float RGB in `[0, 1]` after that
+                quantization. `"np"` is `uint8` HWC arrays. `"pil"` is a list of `PIL.Image.Image`.
+
+        Returns:
+            Images in the requested type.
+        """
+        if output_type == "latent":
+            return latents
+        images = torch.clamp((latents.float() + 1.0) * 127.5 + 0.5, 0, 255).to(torch.uint8)
+        if output_type == "pt":
+            return images.float() / 255.0
+        arrays = images.permute(0, 2, 3, 1).cpu().numpy()
+        if output_type == "np":
+            return arrays
+        return [Image.fromarray(image) for image in arrays]
+
+    @torch.no_grad()
+    @replace_example_docstring(EXAMPLE_DOC_STRING)
+    def __call__(
+        self,
+        class_labels: int | str | list[int | str],
+        height: int | None = None,
+        width: int | None = None,
+        num_inference_steps: int | None = None,
+        timesteps: list[float] | None = None,
+        sigmas: list[float] | None = None,
+        guidance_scale: float | None = None,
+        guidance_interval_min: float | None = None,
+        guidance_interval_max: float | None = None,
+        noise_scale: float | None = None,
+        t_eps: float | None = None,
+        eta: float = 0.0,
+        generator: torch.Generator | list[torch.Generator] | None = None,
+        latents: torch.Tensor | None = None,
+        output_type: str = "pil",
+        return_dict: bool = True,
+        callback_on_step_end: Callable[[int, int, dict], dict] | PipelineCallback | MultiPipelineCallbacks | None = None,
+        callback_on_step_end_tensor_inputs: list[str] | None = None,
+        **kwargs,
+    ) -> ImagePipelineOutput | tuple:
+        r"""
+        Generate images from ImageNet class labels.
+
+        Args:
+            class_labels (`int`, `str`, or `list`):
+                One ImageNet class id, one English label, or a list of either. English labels are
+                resolved with [`get_label_ids`]. The list length is the batch size.
+            height (`int`, *optional*):
+                Image height in pixels. Defaults to `transformer.config.sample_size`. Must be
+                divisible by the patch size. Other sizes recompute rotary positions for that grid.
+            width (`int`, *optional*):
+                Image width in pixels. Defaults to `transformer.config.sample_size`.
+            num_inference_steps (`int`, *optional*):
+                Denoising intervals. Defaults to `transformer.config.default_num_inference_steps` (50).
+            timesteps (`list[float]`, *optional*):
+                Not accepted. [`PixelDiT2Scheduler`] builds `linspace(0, 1, steps + 1)`.
+            sigmas (`list[float]`, *optional*):
+                Not accepted. PixelDiT2 integrates flow time. Use `solver="euler"` for one evaluation per step.
+            guidance_scale (`float`, *optional*):
+                Classifier-free guidance scale. Defaults to `transformer.config.guidance_scale`.
+                Guidance is on when this is greater than 1 and the flow time is inside the interval.
+            guidance_interval_min (`float`, *optional*):
+                Exclusive lower flow-time bound. Defaults to `transformer.config.guidance_interval_min`.
+            guidance_interval_max (`float`, *optional*):
+                Exclusive upper flow-time bound. Defaults to `transformer.config.guidance_interval_max`.
+            noise_scale (`float`, *optional*):
+                Initial noise scale. Defaults to `transformer.config.noise_scale`. Ignored when `latents` is passed.
+            t_eps (`float`, *optional*):
+                Floor of the velocity denominator `1 - t`. Defaults to `transformer.config.t_eps` (0.05).
+            eta (`float`, defaults to 0.0):
+                DDIM eta. Ignored by [`PixelDiT2Scheduler`].
+            generator (`torch.Generator` or `list[torch.Generator]`, *optional*):
+                RNG for the initial noise.
+            latents (`torch.Tensor`, *optional*):
+                Initial noise `(batch, 3, height, width)`. Not multiplied by `noise_scale`.
+            output_type (`str`, defaults to `"pil"`):
+                `"pil"`, `"np"`, `"pt"` (float RGB in `[0, 1]`), or `"latent"` (pixels in model range).
+            return_dict (`bool`, defaults to `True`):
+                Return [`ImagePipelineOutput`] when `True`, otherwise a one-tuple of images.
+            callback_on_step_end (`Callable` or `PipelineCallback`, *optional*):
+                Called as `callback_on_step_end(pipeline, step, timestep, callback_kwargs)` after each
+                scheduler step. Return a dict to replace tensors listed in
+                `callback_on_step_end_tensor_inputs`.
+            callback_on_step_end_tensor_inputs (`list[str]`, *optional*):
+                Tensor names passed to the step callback. Defaults to `["latents"]`.
+
+        Examples:
+
+        Returns:
+            [`ImagePipelineOutput`] or `tuple`:
+                When `return_dict` is `True`, [`ImagePipelineOutput`] with the images. Otherwise a tuple
+                whose first element is the images.
+        """
+        callback = kwargs.pop("callback", None)
+        callback_steps = kwargs.pop("callback_steps", None)
+        if kwargs:
+            raise TypeError(f"Unexpected arguments: {sorted(kwargs)}.")
+        if callback_on_step_end_tensor_inputs is None:
+            callback_on_step_end_tensor_inputs = ["latents"]
+
+        if callback is not None:
+            deprecate(
+                "callback",
+                "1.0.0",
+                "Passing `callback` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
+            )
+        if callback_steps is not None:
+            deprecate(
+                "callback_steps",
+                "1.0.0",
+                "Passing `callback_steps` as an input argument to `__call__` is deprecated, consider using `callback_on_step_end`",
+            )
+        if isinstance(callback_on_step_end, (PipelineCallback, MultiPipelineCallbacks)):
+            callback_on_step_end_tensor_inputs = callback_on_step_end.tensor_inputs
+
+        image_size = int(self.transformer.config.sample_size)
+        height = image_size if height is None else int(height)
+        width = image_size if width is None else int(width)
+        if num_inference_steps is None:
+            num_inference_steps = int(self._sampling_default("default_num_inference_steps", 50))
+        if guidance_scale is None:
+            guidance_scale = float(self._sampling_default("guidance_scale", 2.4))
+        if guidance_interval_min is None:
+            guidance_interval_min = float(self._sampling_default("guidance_interval_min", 0.125))
+        if guidance_interval_max is None:
+            guidance_interval_max = float(self._sampling_default("guidance_interval_max", 0.9))
+        if noise_scale is None:
+            noise_scale = float(self._sampling_default("noise_scale", 1.0))
+        if t_eps is None:
+            t_eps = float(self._sampling_default("t_eps", 0.05))
+
+        # 1. Check inputs.
+        self.check_inputs(
+            class_labels,
+            height,
+            width,
+            num_inference_steps,
+            guidance_scale,
+            guidance_interval_min,
+            guidance_interval_max,
+            noise_scale,
+            t_eps,
+            output_type,
+            callback_steps,
+            callback_on_step_end_tensor_inputs,
+        )
+
+        self._guidance_scale = float(guidance_scale)
+        self._interrupt = False
+
+        # 2. Define call parameters.
+        class_label_ids = self._normalize_class_labels(class_labels)
+        batch_size = len(class_label_ids)
+        device = self._execution_device
+        null_class = int(self.transformer.config.num_classes)
+        class_labels_t = torch.tensor(class_label_ids, device=device, dtype=torch.long)
+        if torch.any(class_labels_t < 0) or torch.any(class_labels_t > null_class):
+            raise ValueError(f"Class ids must lie in [0, {null_class}], got {class_label_ids}.")
+        class_null = torch.full_like(class_labels_t, null_class)
+
+        # 3. Encode input condition (class ids are already the condition; nothing else is encoded).
+
+        # 4. Prepare timesteps.
+        scheduled_timesteps = self._prepare_scheduler(num_inference_steps, device, timesteps, sigmas)
+
+        # 5. Prepare latent variables (pixel-space noise; there is no VAE).
+        latents = self.prepare_latents(
+            batch_size,
+            int(self.transformer.config.in_channels),
+            height,
+            width,
+            torch.float32,
+            device,
+            generator,
+            latents,
+            noise_scale,
+        ).float()
+
+        # 6. Prepare extra step kwargs.
+        extra_step_kwargs = self.prepare_extra_step_kwargs(generator, eta)
+        num_warmup_steps = len(scheduled_timesteps) - num_inference_steps * self.scheduler.order
+        self._num_timesteps = len(scheduled_timesteps)
+
+        # 7. Denoising loop.
+        with self.progress_bar(total=num_inference_steps) as progress_bar:
+            for i, timestep in enumerate(scheduled_timesteps):
+                if self.interrupt:
+                    continue
+
+                flow_time = float(timestep.to(dtype=torch.float32).reshape(-1)[0])
+                x0 = self._guided_x0(
+                    latents,
+                    flow_time,
+                    class_labels_t,
+                    class_null,
+                    self.guidance_scale,
+                    guidance_interval_min,
+                    guidance_interval_max,
+                )
+                latents = self.scheduler.step(
+                    x0, timestep, latents, t_eps=t_eps, **extra_step_kwargs, return_dict=False
+                )[0]
+
+                if callback_on_step_end is not None:
+                    callback_kwargs = {key: locals()[key] for key in callback_on_step_end_tensor_inputs}
+                    callback_outputs = callback_on_step_end(self, i, timestep, callback_kwargs)
+                    latents = callback_outputs.pop("latents", latents)
+
+                if i == len(scheduled_timesteps) - 1 or ((i + 1) > num_warmup_steps and (i + 1) % self.scheduler.order == 0):
+                    progress_bar.update()
+                    if callback is not None and i % callback_steps == 0:
+                        callback(i, timestep, latents)
+
+                if XLA_AVAILABLE:
+                    xm.mark_step()
+
+        images = self._images_from_latents(latents, output_type)
+        self.maybe_free_model_hooks()
+
+        if not return_dict:
+            return (images,)
+        return ImagePipelineOutput(images=images)

PixelDiT2-H-16-512/scheduler/scheduler_config.json

@@ -0,0 +1,8 @@
+{
+  "_class_name": "PixelDiT2Scheduler",
+  "_diffusers_version": "0.39.0",
+  "num_train_timesteps": 1000,
+  "prediction_type": "sample",
+  "solver": "heun",
+  "t_eps": 0.05
+}

PixelDiT2-H-16-512/scheduler/scheduling_pixeldit2.py

@@ -0,0 +1,216 @@
+"""PixelDiT2 flow scheduler.
+
+Time runs from 0 (noise) to 1 (data). The transformer predicts the clean sample x0.
+Velocity is `(x0 - x) / max(1 - t, t_eps)`. Heun is used on every interval except the
+last, which is Euler. This matches `GroundedHeunSampler` rather than the sigma grid
+inside `FlowMatchHeunDiscreteScheduler`.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Optional, Union
+
+import torch
+from diffusers.configuration_utils import ConfigMixin, register_to_config
+from diffusers.schedulers.scheduling_utils import SchedulerMixin
+from diffusers.utils import BaseOutput
+
+
+@dataclass
+class PixelDiT2SchedulerOutput(BaseOutput):
+    prev_sample: torch.Tensor
+
+
+class PixelDiT2Scheduler(SchedulerMixin, ConfigMixin):
+    """Uniform flow-time scheduler for PixelDiT2.
+
+    `solver="heun"` evaluates the model twice on each interval except the last.
+    `solver="euler"` evaluates it once per interval on the same linspace.
+    """
+
+    config_name = "scheduler_config.json"
+    _compatibles: list[str] = []
+    order = 2
+
+    @register_to_config
+    def __init__(
+        self,
+        num_train_timesteps: int = 1000,
+        solver: str = "heun",
+        prediction_type: str = "sample",
+        t_eps: float = 0.05,
+    ):
+        if solver not in {"heun", "euler"}:
+            raise ValueError(f'`solver` must be "heun" or "euler", got {solver}.')
+        if prediction_type not in {"sample", "velocity"}:
+            raise ValueError(f'`prediction_type` must be "sample" or "velocity", got {prediction_type}.')
+        if num_train_timesteps < 1:
+            raise ValueError(f"`num_train_timesteps` must be >= 1, got {num_train_timesteps}.")
+        if t_eps <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {t_eps}.")
+
+        self.timesteps: Optional[torch.Tensor] = None
+        self.schedule: Optional[torch.Tensor] = None
+        self.num_inference_steps: Optional[int] = None
+        self._step_plan: list[dict] = []
+        self._step_index: Optional[int] = None
+        self._begin_index: Optional[int] = None
+        self._heun_sample: Optional[torch.Tensor] = None
+        self._heun_velocity: Optional[torch.Tensor] = None
+
+    @property
+    def init_noise_sigma(self) -> float:
+        return 1.0
+
+    @property
+    def step_index(self) -> Optional[int]:
+        return self._step_index
+
+    @property
+    def begin_index(self) -> Optional[int]:
+        return self._begin_index
+
+    def set_begin_index(self, begin_index: int = 0) -> None:
+        self._begin_index = int(begin_index)
+
+    def set_timesteps(
+        self,
+        num_inference_steps: int,
+        device: Union[str, torch.device, None] = None,
+        **kwargs,
+    ) -> None:
+        """Install `linspace(0, 1, steps + 1)` and the Heun or Euler step plan.
+
+        Args:
+            num_inference_steps (`int`):
+                Number of intervals from noise to data.
+            device (`str` or `torch.device`, *optional*):
+                Device of `timesteps`.
+        """
+        if kwargs:
+            raise ValueError(f"PixelDiT2Scheduler.set_timesteps does not accept {sorted(kwargs)}.")
+        if num_inference_steps < 1:
+            raise ValueError(f"`num_inference_steps` must be >= 1, got {num_inference_steps}.")
+
+        knots = torch.linspace(0.0, 1.0, num_inference_steps + 1, device=device, dtype=torch.float32)
+        plan: list[dict] = []
+        use_heun = self.config.solver == "heun" and num_inference_steps > 1
+        if use_heun:
+            for index in range(num_inference_steps - 1):
+                current = float(knots[index])
+                nxt = float(knots[index + 1])
+                plan.append({"kind": "heun_predict", "t_model": current, "t_cur": current, "t_next": nxt})
+                plan.append({"kind": "heun_correct", "t_model": nxt, "t_cur": current, "t_next": nxt})
+            last = num_inference_steps - 1
+            plan.append(
+                {
+                    "kind": "euler",
+                    "t_model": float(knots[last]),
+                    "t_cur": float(knots[last]),
+                    "t_next": float(knots[last + 1]),
+                }
+            )
+        else:
+            for index in range(num_inference_steps):
+                plan.append(
+                    {
+                        "kind": "euler",
+                        "t_model": float(knots[index]),
+                        "t_cur": float(knots[index]),
+                        "t_next": float(knots[index + 1]),
+                    }
+                )
+
+        self.num_inference_steps = int(num_inference_steps)
+        self.schedule = knots
+        self._step_plan = plan
+        self.timesteps = torch.tensor([item["t_model"] for item in plan], device=device, dtype=torch.float32)
+        self.order = 2 if use_heun else 1
+        self._step_index = 0 if self._begin_index is None else self._begin_index
+        self._heun_sample = None
+        self._heun_velocity = None
+
+    def scale_model_input(self, sample: torch.Tensor, timestep: Union[float, torch.Tensor]) -> torch.Tensor:
+        del timestep
+        return sample
+
+    def _velocity(
+        self,
+        sample: torch.Tensor,
+        model_output: torch.Tensor,
+        flow_time: float,
+        t_eps: float,
+    ) -> torch.Tensor:
+        if self.config.prediction_type == "velocity":
+            return model_output.float()
+        denominator = max(1.0 - float(flow_time), float(t_eps))
+        return (model_output.float() - sample.float()) / denominator
+
+    def step(
+        self,
+        model_output: torch.Tensor,
+        timestep: Union[float, torch.Tensor],
+        sample: torch.Tensor,
+        return_dict: bool = True,
+        generator: Optional[Union[torch.Generator, list[torch.Generator]]] = None,
+        t_eps: Optional[float] = None,
+    ) -> Union[PixelDiT2SchedulerOutput, tuple]:
+        """Advance one planned step.
+
+        Args:
+            model_output (`torch.Tensor`):
+                x0 prediction when `prediction_type` is `"sample"`.
+            timestep (`float` or `torch.Tensor`):
+                Flow time at which `model_output` was evaluated. The step plan is the
+                source of the update; this value is checked against that plan.
+            sample (`torch.Tensor`):
+                Current sample. For a Heun corrector this is the predictor state.
+            return_dict (`bool`, defaults to `True`):
+                Return [`PixelDiT2SchedulerOutput`] when `True`.
+            generator (`torch.Generator`, *optional*):
+                Unused. The integrator is deterministic.
+            t_eps (`float`, *optional*):
+                Floor on `1 - t`. Defaults to the scheduler config.
+
+        Returns:
+            [`PixelDiT2SchedulerOutput`] or `tuple`:
+                The updated sample in float32.
+        """
+        del generator
+        if not self._step_plan or self._step_index is None:
+            raise ValueError("Call `set_timesteps` before `step`.")
+        if self._step_index >= len(self._step_plan):
+            raise ValueError("PixelDiT2Scheduler has already reached the final timestep.")
+
+        plan = self._step_plan[self._step_index]
+        sample = sample.to(dtype=torch.float32)
+        floor = float(self.config.t_eps if t_eps is None else t_eps)
+        if floor <= 0:
+            raise ValueError(f"`t_eps` must be positive, got {floor}.")
+        velocity = self._velocity(sample, model_output, plan["t_model"], floor)
+        dt = plan["t_next"] - plan["t_cur"]
+
+        kind = plan["kind"]
+        if kind == "heun_predict":
+            self._heun_sample = sample
+            self._heun_velocity = velocity
+            prev_sample = sample + dt * velocity
+        elif kind == "heun_correct":
+            if self._heun_sample is None or self._heun_velocity is None:
+                raise RuntimeError("Heun corrector was called before its predictor.")
+            prev_sample = self._heun_sample + dt * 0.5 * (self._heun_velocity + velocity)
+            self._heun_sample = None
+            self._heun_velocity = None
+        else:
+            prev_sample = sample + dt * velocity
+            self._heun_sample = None
+            self._heun_velocity = None
+
+        self._step_index += 1
+        if not return_dict:
+            return (prev_sample,)
+        return PixelDiT2SchedulerOutput(prev_sample=prev_sample)
+
+    def __len__(self) -> int:
+        return int(self.config.num_train_timesteps)

PixelDiT2-H-16-512/transformer/config.json

@@ -0,0 +1,26 @@
+{
+  "_class_name": "PixelDiT2Transformer2DModel",
+  "_diffusers_version": "0.39.0",
+  "attn_dropout": 0.0,
+  "default_num_inference_steps": 50,
+  "depth": 32,
+  "grounding_encoder": "dinov3_vitb16",
+  "grounding_proj_blocks": 1,
+  "grounding_proj_heads": 16,
+  "guidance_interval_max": 0.925,
+  "guidance_interval_min": 0.1,
+  "guidance_scale": 2.1,
+  "hidden_size": 1280,
+  "in_channels": 3,
+  "in_context_len": 32,
+  "in_context_start": 8,
+  "mlp_hidden_multiple": 32,
+  "noise_scale": 2.0,
+  "num_classes": 1000,
+  "num_heads": 16,
+  "patch_size": 16,
+  "proj_dropout": 0.2,
+  "sample_size": 512,
+  "single_silu_cond": true,
+  "t_eps": 0.05
+}

PixelDiT2-H-16-512/transformer/diffusion_pytorch_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b096606fa532ffda15ebc313a50a42d6e6c2121b8195217942266c457b2b4567
+size 2147459616

PixelDiT2-H-16-512/transformer/transformer_pixeldit2.py

@@ -0,0 +1,1094 @@
+# Copyright 2026 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""PixelDiT2 denoiser as a diffusers transformer.
+
+PixelDiT2 predicts the clean image x0 directly in RGB pixel space. A frozen
+DINOv3 encoder grounds every patch token; the unconditional branch of
+classifier-free guidance skips that encoder (`skip_grounding=True`) and uses
+the null class id `num_classes`.
+"""
+
+from __future__ import annotations
+
+import math
+from dataclasses import dataclass
+
+import timm
+import torch
+import torch.nn.functional as F
+from diffusers.configuration_utils import ConfigMixin, register_to_config
+from diffusers.models.modeling_outputs import Transformer2DModelOutput
+from diffusers.models.modeling_utils import ModelMixin
+from torch import nn
+
+
+@dataclass(frozen=True)
+class EncoderSpec:
+    r"""
+    Frozen grounding encoder used by PixelDiT2.
+
+    Args:
+        timm_name (`str`):
+            `timm` model id, including the pretrained tag when one exists.
+        feature_dim (`int`):
+            Channel count of the encoder patch tokens.
+        patch_size (`int`):
+            Encoder patch size in pixels.
+    """
+
+    timm_name: str
+    feature_dim: int
+    patch_size: int
+
+
+GROUNDING_ENCODERS: dict[str, EncoderSpec] = {
+    "dinov3_vits16": EncoderSpec("vit_small_patch16_dinov3_qkvb.lvd1689m", 384, 16),
+    "dinov3_vitb16": EncoderSpec("vit_base_patch16_dinov3_qkvb.lvd1689m", 768, 16),
+    "dinov3_vitl16": EncoderSpec("vit_large_patch16_dinov3_qkvb.lvd1689m", 1024, 16),
+}
+
+IMAGENET_MEAN = (0.485, 0.456, 0.406)
+IMAGENET_STD = (0.229, 0.224, 0.225)
+
+
+def apply_adaln(hidden_states: torch.Tensor, shift: torch.Tensor, scale: torch.Tensor) -> torch.Tensor:
+    r"""
+    Apply adaptive layer-norm modulation.
+
+    Args:
+        hidden_states (`torch.Tensor`):
+            Token states of shape `(batch, tokens, hidden)`.
+        shift (`torch.Tensor`):
+            Shift term broadcastable to `hidden_states`.
+        scale (`torch.Tensor`):
+            Scale term broadcastable to `hidden_states`. The applied factor is `1 + scale`.
+
+    Returns:
+        `torch.Tensor`: Modulated tokens, same shape as `hidden_states`.
+    """
+    return hidden_states * (1 + scale) + shift
+
+
+def precompute_freqs_cis_2d(
+    dim: int,
+    height: int,
+    width: int,
+    theta: float = 10000.0,
+    scale: float = 16.0,
+) -> torch.Tensor:
+    r"""
+    Build 2D rotary frequencies for a patch grid.
+
+    Positions are `linspace(0, scale, length)` on each axis, matching the PixelDiT2 training code.
+    A different grid size recomputes the table, which is how non-native resolutions are handled.
+
+    Args:
+        dim (`int`):
+            Head dimension. Must be divisible by 4.
+        height (`int`):
+            Grid height in patches.
+        width (`int`):
+            Grid width in patches.
+        theta (`float`, defaults to 10000.0):
+            RoPE base period.
+        scale (`float`, defaults to 16.0):
+            Coordinate of the last patch on each axis.
+
+    Returns:
+        `torch.Tensor`: Complex frequencies of shape `(height * width, dim / 2)`.
+    """
+    x_pos = torch.linspace(0, scale, width)
+    y_pos = torch.linspace(0, scale, height)
+    y_pos, x_pos = torch.meshgrid(y_pos, x_pos, indexing="ij")
+    y_pos = y_pos.reshape(-1)
+    x_pos = x_pos.reshape(-1)
+    freqs = 1.0 / (theta ** (torch.arange(0, dim, 4)[: (dim // 4)].float() / dim))
+    x_freqs = torch.outer(x_pos, freqs).float()
+    y_freqs = torch.outer(y_pos, freqs).float()
+    x_cis = torch.polar(torch.ones_like(x_freqs), x_freqs)
+    y_cis = torch.polar(torch.ones_like(y_freqs), y_freqs)
+    freqs_cis = torch.cat([x_cis.unsqueeze(dim=-1), y_cis.unsqueeze(dim=-1)], dim=-1)
+    return freqs_cis.reshape(height * width, -1)
+
+
+def apply_rotary_emb(
+    query: torch.Tensor,
+    key: torch.Tensor,
+    freqs_cis: torch.Tensor,
+) -> tuple[torch.Tensor, torch.Tensor]:
+    r"""
+    Apply rotary embeddings to query and key.
+
+    Args:
+        query (`torch.Tensor`):
+            Queries of shape `(batch, tokens, heads, head_dim)`.
+        key (`torch.Tensor`):
+            Keys of shape `(batch, tokens, heads, head_dim)`.
+        freqs_cis (`torch.Tensor`):
+            Complex frequencies of shape `(tokens, head_dim / 2)`.
+
+    Returns:
+        `tuple[torch.Tensor, torch.Tensor]`: Rotated query and key, same shapes as the inputs.
+    """
+    freqs_cis = freqs_cis[None, :, None, :]
+    query_complex = torch.view_as_complex(query.float().reshape(*query.shape[:-1], -1, 2))
+    key_complex = torch.view_as_complex(key.float().reshape(*key.shape[:-1], -1, 2))
+    query_out = torch.view_as_real(query_complex * freqs_cis).flatten(3)
+    key_out = torch.view_as_real(key_complex * freqs_cis).flatten(3)
+    return query_out.type_as(query), key_out.type_as(key)
+
+
+class TimestepConditioner(nn.Module):
+    r"""
+    Sinusoidal timestep embedding followed by a two-layer MLP.
+
+    PixelDiT2 uses `max_period=10`, not the more common 10000.
+
+    Args:
+        hidden_size (`int`):
+            Width of the MLP output.
+        frequency_embedding_size (`int`, defaults to 256):
+            Width of the sinusoidal embedding.
+    """
+
+    def __init__(self, hidden_size: int, frequency_embedding_size: int = 256):
+        super().__init__()
+        self.mlp = nn.Sequential(
+            nn.Linear(frequency_embedding_size, hidden_size, bias=True),
+            nn.SiLU(),
+            nn.Linear(hidden_size, hidden_size, bias=True),
+        )
+        self.frequency_embedding_size = frequency_embedding_size
+
+    @staticmethod
+    def timestep_embedding(timesteps: torch.Tensor, dim: int, max_period: int = 10) -> torch.Tensor:
+        r"""
+        Build a sinusoidal embedding of scalar timesteps.
+
+        Args:
+            timesteps (`torch.Tensor`):
+                Timesteps of shape `(batch,)`.
+            dim (`int`):
+                Embedding width.
+            max_period (`int`, defaults to 10):
+                Maximum period of the sinusoid.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, dim)`.
+        """
+        half = dim // 2
+        freqs = torch.exp(
+            -math.log(max_period) * torch.arange(start=0, end=half, dtype=torch.float32, device=timesteps.device) / half
+        )
+        args = timesteps[..., None].float() * freqs[None, ...]
+        embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)
+        if dim % 2:
+            embedding = torch.cat([embedding, torch.zeros_like(embedding[:, :1])], dim=-1)
+        return embedding
+
+    def forward(self, timesteps: torch.Tensor) -> torch.Tensor:
+        r"""
+        Embed timesteps.
+
+        Args:
+            timesteps (`torch.Tensor`):
+                Flow time `t` of shape `(batch,)`, in `[0, 1]`.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, hidden_size)`.
+        """
+        timestep_freq = self.timestep_embedding(timesteps, self.frequency_embedding_size)
+        mlp_dtype = self.mlp[0].weight.dtype
+        if timestep_freq.dtype != mlp_dtype:
+            timestep_freq = timestep_freq.to(mlp_dtype)
+        return self.mlp(timestep_freq)
+
+
+class ClassEmbedder(nn.Module):
+    r"""
+    Embedding table for ImageNet class ids, including the null class.
+
+    Args:
+        num_classes (`int`):
+            Number of rows. PixelDiT2 uses `num_classes + 1` so index `num_classes` is unconditional.
+        hidden_size (`int`):
+            Embedding width.
+    """
+
+    def __init__(self, num_classes: int, hidden_size: int):
+        super().__init__()
+        self.embedding_table = nn.Embedding(num_classes, hidden_size)
+        self.num_classes = num_classes
+
+    def forward(self, labels: torch.Tensor) -> torch.Tensor:
+        r"""
+        Look up class embeddings.
+
+        Args:
+            labels (`torch.Tensor`):
+                Integer class ids of shape `(batch,)`.
+
+        Returns:
+            `torch.Tensor`: Embeddings of shape `(batch, hidden_size)`.
+        """
+        return self.embedding_table(labels)
+
+
+class RMSNorm(nn.Module):
+    r"""
+    Root-mean-square normalization.
+
+    Args:
+        hidden_size (`int`):
+            Channel width.
+        eps (`float`, defaults to 1e-6):
+            Numerical stability term.
+    """
+
+    def __init__(self, hidden_size: int, eps: float = 1e-6):
+        super().__init__()
+        self.weight = nn.Parameter(torch.ones(hidden_size))
+        self.variance_epsilon = eps
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Normalize the last dimension.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Input of shape `(..., hidden_size)`.
+
+        Returns:
+            `torch.Tensor`: Normalized tensor, same shape as the input.
+        """
+        input_dtype = hidden_states.dtype
+        hidden_states = hidden_states.to(torch.float32)
+        variance = hidden_states.pow(2).mean(-1, keepdim=True)
+        hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
+        return self.weight * hidden_states.to(input_dtype)
+
+
+class FeedForward(nn.Module):
+    r"""
+    SwiGLU feed-forward block.
+
+    Args:
+        dim (`int`):
+            Input and output width.
+        hidden_dim (`int`):
+            Width before the SwiGLU 2/3 reduction. The inner width is `int(2 * hidden_dim / 3)`,
+            then rounded up to `hidden_multiple` when that value is positive.
+        drop (`float`, defaults to 0.0):
+            Dropout on the hidden activation. Disabled by `eval()`.
+        hidden_multiple (`int`, defaults to 0):
+            Round the inner width up to a multiple of this value. `0` disables rounding.
+            The 512px PixelDiT2 checkpoint uses 32.
+    """
+
+    def __init__(self, dim: int, hidden_dim: int, drop: float = 0.0, hidden_multiple: int = 0):
+        super().__init__()
+        hidden_dim = int(2 * hidden_dim / 3)
+        hidden_multiple = int(hidden_multiple)
+        if hidden_multiple > 0:
+            hidden_dim = ((hidden_dim + hidden_multiple - 1) // hidden_multiple) * hidden_multiple
+        self.w1 = nn.Linear(dim, hidden_dim, bias=False)
+        self.w3 = nn.Linear(dim, hidden_dim, bias=False)
+        self.w2 = nn.Linear(hidden_dim, dim, bias=False)
+        self.ffn_dropout = nn.Dropout(drop)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Apply SwiGLU.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(..., dim)`.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(..., dim)`.
+        """
+        hidden = F.silu(self.w1(hidden_states)) * self.w3(hidden_states)
+        return self.w2(self.ffn_dropout(hidden))
+
+
+class RotaryAttention(nn.Module):
+    r"""
+    Multi-head self-attention with QK RMSNorm and 2D rotary embeddings.
+
+    Args:
+        dim (`int`):
+            Token width. Must be divisible by `num_heads`.
+        num_heads (`int`, defaults to 8):
+            Number of attention heads.
+        qkv_bias (`bool`, defaults to `False`):
+            Whether the fused QKV projection has a bias. PixelDiT2 checkpoints use `False`.
+        qk_norm (`bool`, defaults to `True`):
+            Apply RMSNorm to query and key per head.
+        attn_drop (`float`, defaults to 0.0):
+            Attention dropout. The released models use 0.
+        proj_drop (`float`, defaults to 0.0):
+            Dropout after the output projection.
+    """
+
+    def __init__(
+        self,
+        dim: int,
+        num_heads: int = 8,
+        qkv_bias: bool = False,
+        qk_norm: bool = True,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+    ):
+        super().__init__()
+        if dim % num_heads != 0:
+            raise ValueError(f"dim={dim} must be divisible by num_heads={num_heads}.")
+        self.dim = dim
+        self.num_heads = num_heads
+        self.head_dim = dim // num_heads
+        self.qkv = nn.Linear(dim, dim * 3, bias=qkv_bias)
+        self.q_norm = RMSNorm(self.head_dim) if qk_norm else nn.Identity()
+        self.k_norm = RMSNorm(self.head_dim) if qk_norm else nn.Identity()
+        self.attn_drop = nn.Dropout(attn_drop)
+        self.proj = nn.Linear(dim, dim)
+        self.proj_drop = nn.Dropout(proj_drop)
+
+    def forward(self, hidden_states: torch.Tensor, pos: torch.Tensor, mask: torch.Tensor | None = None) -> torch.Tensor:
+        r"""
+        Run self-attention.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, tokens, dim)`.
+            pos (`torch.Tensor`):
+                Complex RoPE table of shape `(tokens, head_dim / 2)`.
+            mask (`torch.Tensor`, *optional*):
+                Attention mask forwarded to scaled-dot-product attention. `None` attends to every token.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, dim)`.
+        """
+        batch, tokens, channels = hidden_states.shape
+        qkv = self.qkv(hidden_states).reshape(batch, tokens, 3, self.num_heads, channels // self.num_heads)
+        qkv = qkv.permute(2, 0, 1, 3, 4)
+        query, key, value = qkv[0], qkv[1], qkv[2]
+        query = self.q_norm(query)
+        key = self.k_norm(key)
+        query, key = apply_rotary_emb(query, key, freqs_cis=pos)
+        query = query.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2)
+        key = key.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2).contiguous()
+        value = value.view(batch, -1, self.num_heads, channels // self.num_heads).transpose(1, 2).contiguous()
+        # The training forward always uses dropout_p=0 and ignores `attn_drop`.
+        hidden_states = F.scaled_dot_product_attention(query, key, value, attn_mask=mask, dropout_p=0.0)
+        hidden_states = hidden_states.transpose(1, 2).reshape(batch, tokens, channels)
+        hidden_states = self.proj(hidden_states)
+        hidden_states = self.proj_drop(hidden_states)
+        return hidden_states
+
+
+class PatchTokenEmbedder(nn.Module):
+    r"""
+    Linear embedding of flattened pixel patches.
+
+    Args:
+        in_channels (`int`):
+            Flattened patch width, `in_channels * patch_size ** 2` at the call site.
+        embed_dim (`int`):
+            Token width.
+        bias (`bool`, defaults to `True`):
+            Bias on the linear projection. PixelDiT2 uses `True`.
+    """
+
+    def __init__(self, in_channels: int, embed_dim: int, bias: bool = True):
+        super().__init__()
+        self.in_channels = in_channels
+        self.embed_dim = embed_dim
+        self.proj = nn.Linear(in_channels, embed_dim, bias=bias)
+        self.norm = nn.Identity()
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        r"""
+        Embed patch tokens.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Flattened patches of shape `(batch, tokens, in_channels)`.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, embed_dim)`.
+        """
+        return self.norm(self.proj(hidden_states))
+
+
+class ModulatedDiTBlock(nn.Module):
+    r"""
+    AdaLN-Zero transformer block with rotary attention and SwiGLU.
+
+    Args:
+        hidden_size (`int`):
+            Token width.
+        num_heads (`int`):
+            Attention heads.
+        mlp_ratio (`float`, defaults to 4.0):
+            MLP expansion before the SwiGLU 2/3 reduction.
+        attn_drop (`float`, defaults to 0.0):
+            Attention dropout.
+        proj_drop (`float`, defaults to 0.0):
+            Dropout on the attention projection and the MLP.
+        mlp_hidden_multiple (`int`, defaults to 0):
+            Forwarded to [`FeedForward`].
+    """
+
+    def __init__(
+        self,
+        hidden_size: int,
+        num_heads: int,
+        mlp_ratio: float = 4.0,
+        attn_drop: float = 0.0,
+        proj_drop: float = 0.0,
+        mlp_hidden_multiple: int = 0,
+    ):
+        super().__init__()
+        self.norm1 = RMSNorm(hidden_size, eps=1e-6)
+        self.attn = RotaryAttention(
+            hidden_size,
+            num_heads=num_heads,
+            qkv_bias=False,
+            attn_drop=attn_drop,
+            proj_drop=proj_drop,
+        )
+        self.norm2 = RMSNorm(hidden_size, eps=1e-6)
+        self.mlp = FeedForward(
+            hidden_size,
+            int(hidden_size * mlp_ratio),
+            drop=proj_drop,
+            hidden_multiple=mlp_hidden_multiple,
+        )
+        self.adaLN_modulation = nn.Sequential(
+            nn.SiLU(),
+            nn.Linear(hidden_size, 6 * hidden_size, bias=True),
+        )
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        conditioning: torch.Tensor,
+        pos: torch.Tensor,
+        mask: torch.Tensor | None = None,
+    ) -> torch.Tensor:
+        r"""
+        Run one modulated block.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, tokens, hidden)`.
+            conditioning (`torch.Tensor`):
+                AdaLN conditioning of shape `(batch, tokens, hidden)` or `(batch, 1, hidden)`.
+            pos (`torch.Tensor`):
+                RoPE table aligned with `tokens`.
+            mask (`torch.Tensor`, *optional*):
+                Attention mask. `None` uses full attention.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, tokens, hidden)`.
+        """
+        shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp = self.adaLN_modulation(conditioning).chunk(6, dim=-1)
+        hidden_states = hidden_states + gate_msa * self.attn(
+            apply_adaln(self.norm1(hidden_states), shift_msa, scale_msa), pos, mask=mask
+        )
+        hidden_states = hidden_states + gate_mlp * self.mlp(apply_adaln(self.norm2(hidden_states), shift_mlp, scale_mlp))
+        return hidden_states
+
+
+class PatchFinalLayer(nn.Module):
+    r"""
+    AdaLN-modulated projection from tokens back to pixel patches.
+
+    Args:
+        hidden_size (`int`):
+            Token width.
+        patch_size (`int`):
+            Spatial patch size.
+        out_channels (`int`):
+            Image channels, 3 for RGB.
+    """
+
+    def __init__(self, hidden_size: int, patch_size: int, out_channels: int):
+        super().__init__()
+        self.norm = RMSNorm(hidden_size, eps=1e-6)
+        self.linear = nn.Linear(hidden_size, patch_size * patch_size * out_channels, bias=True)
+        self.adaLN_modulation = nn.Sequential(
+            nn.SiLU(),
+            nn.Linear(hidden_size, 2 * hidden_size, bias=True),
+        )
+
+    def forward(self, hidden_states: torch.Tensor, conditioning: torch.Tensor) -> torch.Tensor:
+        r"""
+        Project tokens to flattened patches.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Tokens of shape `(batch, patches, hidden)`.
+            conditioning (`torch.Tensor`):
+                Conditioning of shape `(batch, patches, hidden)`. A 2D tensor is unsqueezed.
+
+        Returns:
+            `torch.Tensor`: Patch pixels of shape `(batch, patches, patch_size ** 2 * out_channels)`.
+        """
+        shift, scale = self.adaLN_modulation(conditioning).chunk(2, dim=-1)
+        if shift.dim() == 2:
+            shift = shift.unsqueeze(1)
+            scale = scale.unsqueeze(1)
+        return self.linear(apply_adaln(self.norm(hidden_states), shift, scale))
+
+
+class TimestepProjection(nn.Module):
+    r"""
+    Projects DINOv3 tokens into the denoiser width with timestep modulation.
+
+    Args:
+        in_dim (`int`):
+            Encoder feature width.
+        out_dim (`int`):
+            Denoiser hidden size.
+        hidden_size (`int`):
+            Width of the internal transformer blocks. PixelDiT2 uses the denoiser width.
+        num_blocks (`int`, defaults to 1):
+            Number of modulated blocks. Released checkpoints use 1.
+        num_heads (`int`, defaults to 16):
+            Heads inside those blocks. `hidden_size` must be divisible by this value.
+        mlp_ratio (`float`, defaults to 4.0):
+            MLP expansion of the internal blocks.
+    """
+
+    def __init__(
+        self,
+        in_dim: int,
+        out_dim: int,
+        hidden_size: int,
+        num_blocks: int = 1,
+        num_heads: int = 16,
+        mlp_ratio: float = 4.0,
+    ):
+        super().__init__()
+        if hidden_size % num_heads != 0:
+            raise ValueError(f"TimestepProjection hidden_size={hidden_size} must be divisible by num_heads={num_heads}.")
+        self.head_dim = hidden_size // num_heads
+        self.input_proj = nn.Linear(in_dim, hidden_size, bias=True)
+        self.t_embedder = TimestepConditioner(hidden_size)
+        self.blocks = nn.ModuleList(
+            [ModulatedDiTBlock(hidden_size, num_heads, mlp_ratio=mlp_ratio) for _ in range(num_blocks)]
+        )
+        self.output_proj = nn.Linear(hidden_size, out_dim, bias=True)
+        for block in self.blocks:
+            nn.init.zeros_(block.adaLN_modulation[-1].weight)
+            nn.init.zeros_(block.adaLN_modulation[-1].bias)
+        self._pos_cache: dict[tuple[int, int], torch.Tensor] = {}
+
+    def fetch_pos(self, height: int, width: int, device: torch.device) -> torch.Tensor:
+        r"""
+        Return cached RoPE frequencies for a patch grid.
+
+        Args:
+            height (`int`):
+                Grid height in patches.
+            width (`int`):
+                Grid width in patches.
+            device (`torch.device`):
+                Device of the returned table.
+
+        Returns:
+            `torch.Tensor`: Complex frequencies of shape `(height * width, head_dim / 2)`.
+        """
+        key = (height, width)
+        if key not in self._pos_cache:
+            self._pos_cache[key] = precompute_freqs_cis_2d(self.head_dim, height, width)
+        return self._pos_cache[key].to(device)
+
+    def forward(self, tokens: torch.Tensor, timesteps: torch.Tensor, grid_hw: tuple[int, int]) -> torch.Tensor:
+        r"""
+        Project grounding tokens.
+
+        Args:
+            tokens (`torch.Tensor`):
+                Encoder tokens of shape `(batch, patches, in_dim)`.
+            timesteps (`torch.Tensor`):
+                Flow time of shape `(batch,)`.
+            grid_hw (`tuple[int, int]`):
+                `(height, width)` of the token grid in patches.
+
+        Returns:
+            `torch.Tensor`: Tokens of shape `(batch, patches, out_dim)`.
+        """
+        hidden_states = self.input_proj(tokens)
+        conditioning = self.t_embedder(timesteps).unsqueeze(1).to(dtype=hidden_states.dtype)
+        pos = self.fetch_pos(grid_hw[0], grid_hw[1], hidden_states.device)
+        for block in self.blocks:
+            hidden_states = block(hidden_states, conditioning, pos)
+        return self.output_proj(hidden_states)
+
+
+class GroundingConditioner(nn.Module):
+    r"""
+    Frozen DINOv3 encoder plus the trainable projection `P_g`.
+
+    The encoder is built with `pretrained=False`. Conversion copies either the
+    checkpoint's encoder tensors or a `timm` pretrained checkpoint into this module
+    so inference does not download weights.
+
+    Args:
+        encoder_name (`str`):
+            Key of [`GROUNDING_ENCODERS`], for example `dinov3_vits16`.
+        hidden_size (`int`):
+            Denoiser width.
+        target_patch_size (`int`):
+            Denoiser patch size. The encoder grid is resized to this patch grid.
+        proj_num_blocks (`int`, defaults to 1):
+            Blocks inside the projection.
+        proj_num_heads (`int`, defaults to 16):
+            Heads inside the projection.
+    """
+
+    def __init__(
+        self,
+        encoder_name: str,
+        hidden_size: int,
+        target_patch_size: int,
+        proj_num_blocks: int = 1,
+        proj_num_heads: int = 16,
+    ):
+        super().__init__()
+        if encoder_name not in GROUNDING_ENCODERS:
+            raise ValueError(f"Unknown grounding encoder {encoder_name!r}. Available: {sorted(GROUNDING_ENCODERS)}.")
+        self.spec = GROUNDING_ENCODERS[encoder_name]
+        self.encoder_name = encoder_name
+        self.target_patch_size = int(target_patch_size)
+        self.encoder = timm.create_model(self.spec.timm_name, pretrained=False, dynamic_img_size=True)
+        self.encoder.requires_grad_(False)
+        self.dino_proj = TimestepProjection(
+            in_dim=self.spec.feature_dim,
+            out_dim=hidden_size,
+            hidden_size=hidden_size,
+            num_blocks=proj_num_blocks,
+            num_heads=proj_num_heads,
+        )
+        self.register_buffer("image_mean", torch.tensor(IMAGENET_MEAN).view(1, 3, 1, 1), persistent=False)
+        self.register_buffer("image_std", torch.tensor(IMAGENET_STD).view(1, 3, 1, 1), persistent=False)
+
+    def train(self, mode: bool = True) -> "GroundingConditioner":
+        r"""
+        Keep the frozen encoder in eval mode.
+
+        Args:
+            mode (`bool`, defaults to `True`):
+                Whether the projection should train.
+
+        Returns:
+            `GroundingConditioner`: This module.
+        """
+        super().train(mode)
+        self.encoder.eval()
+        return self
+
+    def preprocess(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, tuple[int, int]]:
+        r"""
+        Map pixels from `[-1, 1]` to the encoder's ImageNet normalization.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)` in model range `[-1, 1]`.
+
+        Returns:
+            `tuple[torch.Tensor, tuple[int, int]]`:
+                Normalized encoder input and the denoiser patch grid `(grid_h, grid_w)`.
+        """
+        height, width = hidden_states.shape[-2:]
+        grid_hw = (height // self.target_patch_size, width // self.target_patch_size)
+        encoder_hw = (grid_hw[0] * self.spec.patch_size, grid_hw[1] * self.spec.patch_size)
+        hidden_states = (hidden_states + 1.0) * 0.5
+        hidden_states = (hidden_states - self.image_mean.to(hidden_states.dtype)) / self.image_std.to(hidden_states.dtype)
+        if encoder_hw != (height, width):
+            hidden_states = F.interpolate(hidden_states, size=encoder_hw, mode="bicubic", align_corners=False)
+        return hidden_states, grid_hw
+
+    def _zero_key_bias(self) -> None:
+        """Keep DINOv3 key bias at zero.
+
+        timm's qkvb attention stores a zero key bias as a non-persistent buffer, so it
+        is absent from the checkpoint. `from_pretrained` builds the encoder on the meta
+        device and materializes missing buffers as uninitialized memory. A non-zero key
+        bias makes attention NaN.
+        """
+        for module in self.encoder.modules():
+            bias = getattr(module, "k_bias", None)
+            if isinstance(bias, torch.Tensor) and bias.device.type != "meta":
+                bias.zero_()
+
+    def encode(self, hidden_states: torch.Tensor) -> tuple[torch.Tensor, tuple[int, int]]:
+        r"""
+        Encode an image into patch tokens aligned with the denoiser grid.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)` in `[-1, 1]`.
+
+        Returns:
+            `tuple[torch.Tensor, tuple[int, int]]`:
+                Tokens `(batch, grid_h * grid_w, feature_dim)` and the grid size.
+        """
+        self._zero_key_bias()
+        encoder_input, grid_hw = self.preprocess(hidden_states)
+        with torch.no_grad():
+            features = self.encoder.forward_features(encoder_input)
+        prefix = int(getattr(self.encoder, "num_prefix_tokens", 0))
+        tokens = features[:, prefix:, :]
+        encoder_grid = (
+            encoder_input.shape[-2] // self.spec.patch_size,
+            encoder_input.shape[-1] // self.spec.patch_size,
+        )
+        if encoder_grid != grid_hw:
+            batch, _, channels = tokens.shape
+            tokens = tokens.transpose(1, 2).reshape(batch, channels, *encoder_grid)
+            tokens = F.interpolate(tokens, size=grid_hw, mode="bilinear", align_corners=False)
+            tokens = tokens.flatten(2).transpose(1, 2)
+        return tokens, grid_hw
+
+    def forward(self, hidden_states: torch.Tensor, timesteps: torch.Tensor) -> torch.Tensor:
+        r"""
+        Encode and project grounding tokens.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Images of shape `(batch, 3, height, width)`.
+            timesteps (`torch.Tensor`):
+                Flow time of shape `(batch,)`.
+
+        Returns:
+            `torch.Tensor`: Grounding of shape `(batch, patches, hidden_size)`.
+        """
+        tokens, grid_hw = self.encode(hidden_states)
+        return self.dino_proj(tokens.to(dtype=self.dino_proj.input_proj.weight.dtype), timesteps, grid_hw)
+
+
+class PixelDiT2Transformer2DModel(ModelMixin, ConfigMixin):
+    r"""
+    PixelDiT2 class-conditional denoiser.
+
+    The module names match the training checkpoint (`t_embedder`, `y_embedder`, `s_embedder`,
+    `patch_blocks`, `final_layer`, `dino_conditioner`) so an EMA state dict loads without renaming.
+
+    Args:
+        sample_size (`int`, defaults to 256):
+            Native square resolution in pixels. Other multiples of `patch_size` are valid at inference
+            because rotary positions are recomputed for the requested grid.
+        in_channels (`int`, defaults to 3):
+            Image channels.
+        patch_size (`int`, defaults to 16):
+            Pixel patch size. `height` and `width` must be divisible by this value.
+        hidden_size (`int`, defaults to 1280):
+            Token width of PixelDiT2-H.
+        num_heads (`int`, defaults to 16):
+            Attention heads.
+        depth (`int`, defaults to 32):
+            Number of patch blocks.
+        num_classes (`int`, defaults to 1000):
+            ImageNet classes. The embedding table has `num_classes + 1` rows; the last row is the null class.
+        in_context_len (`int`, defaults to 32):
+            Number of in-context class tokens inserted at `in_context_start`.
+        in_context_start (`int`, defaults to 8):
+            Block index at which the in-context tokens are inserted. Must lie in `[0, depth)`.
+        attn_dropout (`float`, defaults to 0.0):
+            Attention dropout on the middle half of the blocks.
+        proj_dropout (`float`, defaults to 0.2):
+            Projection dropout on the middle half of the blocks. Inactive in `eval()`.
+        mlp_hidden_multiple (`int`, defaults to 0):
+            SwiGLU width rounding. `0` for 256px, `32` for 512px.
+        grounding_encoder (`str`, defaults to `"dinov3_vits16"`):
+            Frozen encoder key. 256px uses `dinov3_vits16`; 512px uses `dinov3_vitb16`.
+        grounding_proj_blocks (`int`, defaults to 1):
+            Blocks in the grounding projection.
+        grounding_proj_heads (`int`, defaults to 16):
+            Heads in the grounding projection.
+        single_silu_cond (`bool`, defaults to `False`):
+            When `False` (256px), SiLU is applied to the summed condition. When `True` (512px), it is not.
+        guidance_scale (`float`, defaults to 2.4):
+            Recommended classifier-free guidance scale. Read by the pipeline when the caller omits it.
+        guidance_interval_min (`float`, defaults to 0.125):
+            Lower flow-time bound of the guidance interval, exclusive.
+        guidance_interval_max (`float`, defaults to 0.9):
+            Upper flow-time bound of the guidance interval, exclusive.
+        noise_scale (`float`, defaults to 1.0):
+            Standard deviation of the initial Gaussian. 1.0 at 256px and 2.0 at 512px.
+        t_eps (`float`, defaults to 0.05):
+            Minimum denominator when converting an x0 prediction into a velocity.
+        default_num_inference_steps (`int`, defaults to 50):
+            Heun steps used when the pipeline caller omits `num_inference_steps`.
+    """
+
+    @register_to_config
+    def __init__(
+        self,
+        sample_size: int = 256,
+        in_channels: int = 3,
+        patch_size: int = 16,
+        hidden_size: int = 1280,
+        num_heads: int = 16,
+        depth: int = 32,
+        num_classes: int = 1000,
+        in_context_len: int = 32,
+        in_context_start: int = 8,
+        attn_dropout: float = 0.0,
+        proj_dropout: float = 0.2,
+        mlp_hidden_multiple: int = 0,
+        grounding_encoder: str = "dinov3_vits16",
+        grounding_proj_blocks: int = 1,
+        grounding_proj_heads: int = 16,
+        single_silu_cond: bool = False,
+        guidance_scale: float = 2.4,
+        guidance_interval_min: float = 0.125,
+        guidance_interval_max: float = 0.9,
+        noise_scale: float = 1.0,
+        t_eps: float = 0.05,
+        default_num_inference_steps: int = 50,
+    ):
+        super().__init__()
+        if not 0 <= in_context_start < depth:
+            raise ValueError(f"in_context_start must be in [0, {depth - 1}], got {in_context_start}.")
+        if hidden_size % num_heads != 0:
+            raise ValueError(f"hidden_size={hidden_size} must be divisible by num_heads={num_heads}.")
+
+        self.in_channels = in_channels
+        self.out_channels = in_channels
+        self.hidden_size = hidden_size
+        self.num_heads = num_heads
+        self.depth = depth
+        self.patch_size = patch_size
+        self.num_classes = num_classes
+        self.in_context_len = in_context_len
+        self.in_context_start = in_context_start
+        self.single_silu_cond = single_silu_cond
+
+        self.t_embedder = TimestepConditioner(hidden_size)
+        self.y_embedder = ClassEmbedder(num_classes + 1, hidden_size)
+        self.s_embedder = PatchTokenEmbedder(in_channels * patch_size**2, hidden_size, bias=True)
+        self.in_context_posemb = nn.Parameter(torch.zeros(1, in_context_len, hidden_size)) if in_context_len > 0 else None
+        self.patch_blocks = nn.ModuleList(
+            [
+                ModulatedDiTBlock(
+                    hidden_size,
+                    num_heads,
+                    attn_drop=attn_dropout if depth // 4 <= i < depth // 4 * 3 else 0.0,
+                    proj_drop=proj_dropout if depth // 4 <= i < depth // 4 * 3 else 0.0,
+                    mlp_hidden_multiple=mlp_hidden_multiple,
+                )
+                for i in range(depth)
+            ]
+        )
+        self.final_layer = PatchFinalLayer(hidden_size, patch_size, self.out_channels)
+        self._init_denoiser_weights()
+        self.dino_conditioner = GroundingConditioner(
+            encoder_name=grounding_encoder,
+            hidden_size=hidden_size,
+            target_patch_size=patch_size,
+            proj_num_blocks=grounding_proj_blocks,
+            proj_num_heads=grounding_proj_heads,
+        )
+        self._pos_cache: dict[tuple[int, int, int], torch.Tensor] = {}
+
+    def _init_denoiser_weights(self) -> None:
+        r"""
+        Initialize the trainable denoiser the same way as the training code.
+
+        The grounding encoder is created afterwards and is not touched here.
+        """
+        weight = self.s_embedder.proj.weight.data
+        nn.init.xavier_uniform_(weight.view([weight.shape[0], -1]))
+        nn.init.constant_(self.s_embedder.proj.bias, 0)
+        nn.init.normal_(self.y_embedder.embedding_table.weight, std=0.02)
+        nn.init.normal_(self.t_embedder.mlp[0].weight, std=0.02)
+        nn.init.normal_(self.t_embedder.mlp[2].weight, std=0.02)
+        if self.in_context_posemb is not None:
+            nn.init.normal_(self.in_context_posemb, std=0.02)
+        for block in self.patch_blocks:
+            nn.init.zeros_(block.adaLN_modulation[-1].weight)
+            nn.init.zeros_(block.adaLN_modulation[-1].bias)
+        nn.init.zeros_(self.final_layer.linear.weight)
+        nn.init.zeros_(self.final_layer.linear.bias)
+        nn.init.zeros_(self.final_layer.adaLN_modulation[-1].weight)
+        nn.init.zeros_(self.final_layer.adaLN_modulation[-1].bias)
+
+    def fetch_pos(self, height: int, width: int, device: torch.device, num_context_tokens: int = 0) -> torch.Tensor:
+        r"""
+        RoPE table for a patch grid, with an identity prefix when context tokens are present.
+
+        Args:
+            height (`int`):
+                Grid height in patches.
+            width (`int`):
+                Grid width in patches.
+            device (`torch.device`):
+                Device of the returned table.
+            num_context_tokens (`int`, defaults to 0):
+                Leading tokens that receive an identity rotation (real part 1, imaginary part 0).
+
+        Returns:
+            `torch.Tensor`: Complex frequencies of shape `(num_context_tokens + height * width, head_dim / 2)`.
+        """
+        key = (height, width, num_context_tokens)
+        if key not in self._pos_cache:
+            pos = precompute_freqs_cis_2d(self.hidden_size // self.num_heads, height, width)
+            if num_context_tokens > 0:
+                identity = torch.ones(num_context_tokens, pos.shape[-1], dtype=pos.dtype)
+                pos = torch.cat([identity, pos], dim=0)
+            self._pos_cache[key] = pos
+        return self._pos_cache[key].to(device)
+
+    def _grounded_condition(
+        self,
+        hidden_states: torch.Tensor,
+        timesteps: torch.Tensor,
+        timestep_embedding: torch.Tensor,
+        class_embedding: torch.Tensor,
+    ) -> torch.Tensor:
+        r"""
+        Conditioning used by the class-conditional branch.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Noisy images `(batch, channels, height, width)`.
+            timesteps (`torch.Tensor`):
+                Flow time `(batch,)`.
+            timestep_embedding (`torch.Tensor`):
+                Timestep embedding `(batch, hidden)`.
+            class_embedding (`torch.Tensor`):
+                Class embedding `(batch, 1, hidden)`.
+
+        Returns:
+            `torch.Tensor`: Per-patch conditioning `(batch, patches, hidden)`.
+        """
+        grounding = self.dino_conditioner(hidden_states, timesteps)
+        conditioning = grounding + timestep_embedding.unsqueeze(1) + class_embedding
+        if self.single_silu_cond:
+            return conditioning
+        return F.silu(conditioning)
+
+    def _null_condition(
+        self,
+        timestep_embedding: torch.Tensor,
+        class_embedding: torch.Tensor,
+        num_patches: int,
+    ) -> torch.Tensor:
+        r"""
+        Conditioning used when grounding is dropped.
+
+        Args:
+            timestep_embedding (`torch.Tensor`):
+                Timestep embedding `(batch, hidden)`.
+            class_embedding (`torch.Tensor`):
+                Class embedding `(batch, 1, hidden)`.
+            num_patches (`int`):
+                Number of spatial patches to broadcast over.
+
+        Returns:
+            `torch.Tensor`: Conditioning `(batch, num_patches, hidden)`.
+        """
+        conditioning = timestep_embedding.unsqueeze(1) + class_embedding
+        if not self.single_silu_cond:
+            conditioning = F.silu(conditioning)
+        return conditioning.expand(-1, num_patches, -1)
+
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        timestep: torch.Tensor,
+        class_labels: torch.Tensor,
+        skip_grounding: bool = False,
+        skip_grounding_mask: torch.Tensor | None = None,
+        return_dict: bool = True,
+    ) -> Transformer2DModelOutput | tuple:
+        r"""
+        Predict the clean image x0.
+
+        Args:
+            hidden_states (`torch.Tensor`):
+                Noisy RGB sample of shape `(batch, in_channels, height, width)`, in model range around `[-1, 1]`.
+                `height` and `width` must be divisible by `patch_size`.
+            timestep (`torch.Tensor`):
+                Flow-matching time `t` in `[0, 1]`, shape `(batch,)` or a scalar broadcast to the batch.
+                `t = 0` is noise and `t = 1` is data.
+            class_labels (`torch.Tensor`):
+                Integer ImageNet ids of shape `(batch,)`. The null class is `num_classes`.
+            skip_grounding (`bool`, defaults to `False`):
+                When `True`, do not run the DINOv3 encoder. The unconditional CFG branch sets this.
+            skip_grounding_mask (`torch.Tensor`, *optional*):
+                Boolean mask of shape `(batch,)`. `True` rows use the null grounding condition.
+                Ignored when `skip_grounding` is `True`.
+            return_dict (`bool`, defaults to `True`):
+                Return [`Transformer2DModelOutput`] when `True`, otherwise a one-tuple of the sample.
+
+        Returns:
+            [`Transformer2DModelOutput`] or `tuple`:
+                The x0 prediction of shape `(batch, in_channels, height, width)`.
+        """
+        batch, _, height, width = hidden_states.shape
+        if height % self.patch_size or width % self.patch_size:
+            raise ValueError(
+                f"Input resolution ({height}, {width}) must be divisible by patch_size={self.patch_size}."
+            )
+        timestep = timestep.reshape(batch)
+        class_labels = class_labels.reshape(batch).long()
+        grid_h, grid_w = height // self.patch_size, width // self.patch_size
+        num_patches = grid_h * grid_w
+
+        timestep_embedding = self.t_embedder(timestep)
+        class_embedding = self.y_embedder(class_labels).view(batch, 1, self.hidden_size)
+
+        if skip_grounding:
+            conditioning = self._null_condition(timestep_embedding, class_embedding, num_patches)
+        else:
+            conditioning = self._grounded_condition(hidden_states, timestep, timestep_embedding, class_embedding)
+            if skip_grounding_mask is not None:
+                null_conditioning = self._null_condition(timestep_embedding, class_embedding, num_patches)
+                conditioning = torch.where(skip_grounding_mask.view(batch, 1, 1), null_conditioning, conditioning)
+
+        tokens = self.s_embedder(F.unfold(hidden_states, kernel_size=self.patch_size, stride=self.patch_size).transpose(1, 2))
+        use_context = self.in_context_len > 0
+        pos = self.fetch_pos(grid_h, grid_w, hidden_states.device)
+        pos_with_context = self.fetch_pos(grid_h, grid_w, hidden_states.device, self.in_context_len) if use_context else None
+        if use_context:
+            context_conditioning = conditioning.mean(dim=1, keepdim=True).expand(-1, self.in_context_len, -1)
+            conditioning_extended = torch.cat([context_conditioning, conditioning], dim=1)
+
+        context_inserted = False
+        for index, block in enumerate(self.patch_blocks):
+            if use_context and not context_inserted and index == self.in_context_start:
+                context_tokens = class_embedding.expand(-1, self.in_context_len, -1)
+                tokens = torch.cat([context_tokens + self.in_context_posemb.to(tokens.dtype), tokens], dim=1)
+                context_inserted = True
+            tokens = block(
+                tokens,
+                conditioning_extended if context_inserted else conditioning,
+                pos_with_context if context_inserted else pos,
+            )
+        if context_inserted:
+            tokens = tokens[:, self.in_context_len :, :]
+
+        output = self.final_layer(tokens, conditioning)
+        output = output.view(batch, num_patches, self.patch_size**2, self.out_channels)
+        output = output.permute(0, 3, 2, 1).reshape(batch, self.out_channels * self.patch_size**2, num_patches)
+        output = F.fold(output, (height, width), kernel_size=self.patch_size, stride=self.patch_size)
+
+        if not return_dict:
+            return (output,)
+        return Transformer2DModelOutput(sample=output)

README.md

@@ -0,0 +1,153 @@
+---
+license: other
+license_name: nsclv1
+license_link: https://huggingface.co/nvidia/PixelDiT2-ImageNet
+library_name: diffusers
+tags:
+  - diffusers
+  - pixeldit2
+  - image-generation
+  - class-conditional
+  - imagenet
+  - flow-matching
+  - pixel-space
+inference: true
+widget:
+  - text: golden retriever
+    output:
+      url: PixelDiT2-H-16-256/demo.png
+  - text: golden retriever
+    output:
+      url: PixelDiT2-H-16-512/demo.png
+language:
+  - en
+---
+
+# BiliSakura/PixelDiT2-diffusers
+
+Self-contained PixelDiT2 class-conditional checkpoints for Hugging Face diffusers. PixelDiT2 denoises in RGB pixel space (no VAE). Each variant ships its original DINOv3 checkpoint in `encoder/` and the mapped copy inside the transformer weights, so inference does not download it. 256px uses DINOv3-S/16. 512px uses DINOv3-B/16.
+
+This repository is PixelDiT2 only. The earlier PixelDiT family is [BiliSakura/PixelDiT-diffusers](https://huggingface.co/BiliSakura/PixelDiT-diffusers).
+
+## Demo
+
+![PixelDiT2-H/16 256 demo](PixelDiT2-H-16-256/demo.png)
+
+Class: *golden retriever* (ImageNet 207). PixelDiT2-H/16 at 256×256, 50 Heun steps, `guidance_scale=2.4`, interval `(0.125, 0.9)`, `noise_scale=1.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`.
+
+![PixelDiT2-H/16 512 demo](PixelDiT2-H-16-512/demo.png)
+
+Same class at 512×512, 50 Heun steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`, `torch_dtype=bfloat16`, seed 1, on `cuda:0`.
+
+## Available checkpoints
+
+| Subfolder | Resolution | Source checkpoint | Encoder | Steps | CFG | CFG interval | Noise scale |
+| --- | --- | --- | --- | ---: | ---: | --- | ---: |
+| [`PixelDiT2-H-16-256/`](PixelDiT2-H-16-256/) | 256×256 | `pixeldit2_h16_in256_ep600.ckpt` | DINOv3-S/16 | 50 | 2.4 | (0.125, 0.9) | 1.0 |
+| [`PixelDiT2-H-16-512/`](PixelDiT2-H-16-512/) | 512×512 | `pixeldit2_h16_in512_ep680.ckpt` | DINOv3-B/16 | 50 | 2.1 | (0.1, 0.925) | 2.0 |
+
+Reported ImageNet FID is 1.46 at 256×256 and 1.48 at 512×512 (50k samples, Heun, 50 steps).
+
+## Repo layout
+
+```text
+BiliSakura/PixelDiT2-diffusers/
+├── README.md
+├── .gitattributes
+├── PixelDiT2-H-16-256/
+│   ├── pipeline.py
+│   ├── model_index.json
+│   ├── demo.png
+│   ├── encoder/
+│   ├── scheduler/scheduling_pixeldit2.py
+│   └── transformer/
+└── PixelDiT2-H-16-512/
+    ├── pipeline.py
+    ├── model_index.json
+    ├── demo.png
+    ├── encoder/
+    ├── scheduler/scheduling_pixeldit2.py
+    └── transformer/
+```
+
+`scheduler/scheduling_pixeldit2.py` is `PixelDiT2Scheduler`. It integrates flow time from 0 to 1 with Heun, and uses Euler on the last interval. Each variant is self-contained.
+
+## Load from Hugging Face
+
+```python
+import torch
+from diffusers import DiffusionPipeline
+
+pipe = DiffusionPipeline.from_pretrained(
+    "BiliSakura/PixelDiT2-diffusers",
+    subfolder="PixelDiT2-H-16-256",
+    custom_pipeline="pipeline.py",
+    trust_remote_code=True,
+    torch_dtype=torch.bfloat16,
+).to("cuda:0")
+
+generator = torch.Generator(device="cuda:0").manual_seed(1)
+image = pipe(
+    class_labels="golden retriever",
+    num_inference_steps=50,
+    guidance_scale=2.4,
+    guidance_interval_min=0.125,
+    guidance_interval_max=0.9,
+    generator=generator,
+).images[0]
+image.save("demo.png")
+```
+
+## Load from a local clone
+
+```python
+from pathlib import Path
+import torch
+from diffusers import DiffusionPipeline
+
+model_dir = Path("PixelDiT2-H-16-256").resolve()
+pipe = DiffusionPipeline.from_pretrained(
+    str(model_dir),
+    local_files_only=True,
+    custom_pipeline=str(model_dir / "pipeline.py"),
+    trust_remote_code=True,
+    torch_dtype=torch.bfloat16,
+).to("cuda:0")
+
+print(pipe.id2label[207])
+print(pipe.get_label_ids("golden retriever"))
+
+generator = torch.Generator(device="cuda:0").manual_seed(1)
+image = pipe(class_labels="golden retriever", generator=generator).images[0]
+image.save("demo.png")
+```
+
+Place the model on one GPU with `.to("cuda:0")`. Do not call `enable_model_cpu_offload()`.
+
+Point `model_dir` at `PixelDiT2-H-16-512` for the 512px checkpoint. Omitting the sampling arguments uses that folder's config: 50 steps, `guidance_scale=2.1`, interval `(0.1, 0.925)`, `noise_scale=2.0`.
+
+`height` and `width` must be divisible by 16. Other sizes recompute rotary positions for that grid. One evaluation per step uses the same linspace: `pipe.scheduler = pipe.scheduler.__class__.from_config(pipe.scheduler.config, solver="euler")`.
+
+## ImageNet class labels
+
+`id2label` is in each variant's `model_index.json`.
+
+- `pipe.id2label` — id to English label
+- `pipe.labels` — synonym to id
+- `pipe.get_label_ids("golden retriever")`
+- `pipe(class_labels="golden retriever")` — strings resolve automatically
+
+## Citation
+
+```bibtex
+@inproceedings{yu2026pixeldit2,
+      title={PixelDiT2: Representation-Grounded Pixel Diffusion Transformers},
+      author={Yongsheng Yu and Wei Xiong and Yichen Sheng and Shiqiu Liu and Jiebo Luo},
+      booktitle={Conference on Neural Information Processing Systems (NeurIPS)},
+      year={2026},
+}
+```
+
+## License
+
+Pipeline code in this repository is Apache-2.0. Converted weights remain under the NVIDIA NSCLv1 non-commercial research license of [nvidia/PixelDiT2-ImageNet](https://huggingface.co/nvidia/PixelDiT2-ImageNet).