---
license: mit
pipeline_tag: text-to-image
library_name: pixeldense
base_model: zehongma/PixelGen
base_model_relation: finetune
datasets:
  - BLIP3o/BLIP3o-60k
language:
  - en
tags:
  - text-to-image
  - pixel-diffusion
  - flow-matching
  - representation-alignment
  - neurips-2026
---

# PixelDense-XXL T2I

**PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion** (NeurIPS 2026)

[📄 Paper](https://arxiv.org/abs/2610.00483) | [🌐 Project Page](https://bio.lehanyang.info/PixelDense/) | [💻 Code](https://github.com/Hansxsourse/PixelDense)

![PixelDense samples at 512×512](https://bio.lehanyang.info/PixelDense/static/images/teaser.jpg)

PixelDense uses frozen dense-prediction models as representation-alignment targets for pixel-space diffusion. DINOv2 and SAM2 supervise a semantic projection stream, Depth Anything v2 and Metric3D v2 supervise a geometric projection stream, and a weight-space orthogonality penalty keeps the two streams in disjoint subspaces of the denoiser feature. The teachers and alignment projections are used only during training, so this checkpoint is a plain 1.1B pixel-space text-to-image denoiser with the same inference cost as PixelGen.

## Model details

| | |
|---|---|
| Architecture | JiT-T2I pixel-space denoiser (PixelGen-XXL) |
| Parameters | 1.1B |
| Resolution | 512 × 512 |
| Text encoder | [Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B) (not included; downloaded automatically) |
| Sampler | Adam LM solver, 25 steps, classifier-free guidance 4.0 |
| Base model | [PixelGen-XXL T2I](https://huggingface.co/zehongma/PixelGen) |
| Fine-tuning data | [BLIP3o-60k](https://huggingface.co/datasets/BLIP3o/BLIP3o-60k) |

See the paper for the full training recipe.

## Files

| File | Size | Contents |
|---|---|---|
| `PixelDense_XXL_T2I.ckpt` | 4.5 GB | EMA denoiser weights, fp32 |

The checkpoint holds weights only: `{"state_dict": {"ema_denoiser.<param>": tensor}}`. It has no optimizer state or training metadata and loads with `torch.load(path, weights_only=True)`. Use it with the [PixelDense code](https://github.com/Hansxsourse/PixelDense): the denoiser has one learned embedding (`visual_type_embedding`) that upstream PixelGen does not have.

## Usage

```bash
git clone https://github.com/Hansxsourse/PixelDense
cd PixelDense
pip install -r requirements.txt

hf download xsourse/PixelDense PixelDense_XXL_T2I.ckpt --local-dir ./ckpts
python main.py predict -c ./configs_t2i/pixeldense_inference_res512.yaml
```

Images are written to `./workdirs/exp_pixeldense_inference/images/predict/`. To use your own prompts, pass a text file with one prompt per line:

```bash
python main.py predict -c ./configs_t2i/pixeldense_inference_res512.yaml \
    --data.pred_dataset.init_args.conditions ./prompts.txt
```

Inference needs only the denoiser and the Qwen3 text encoder; the dense teachers are not loaded.

## Results

512 × 512, from Table 1 of the paper. Higher is better for every metric.

| Model | GenEval Overall | DPG-Bench | HPS v2.1 |
|---|---|---|---|
| PixelGen (released) | 0.79 | – | 0.281 |
| PixelGen + fine-tune (DINOv2 REPA only) | 0.7927 | 78.7 | 0.280 |
| **PixelDense** | **0.8093** | **78.9** | **0.282** |

"PixelGen + fine-tune" is the matched baseline: the same fine-tuning run with only the original DINOv2 alignment.

## Limitations

This is a research checkpoint. It generates at 512 × 512 only, and its outputs can reflect biases in the base model's training data and in BLIP3o-60k.

## License

The PixelDense weights are released under the MIT License. They are fine-tuned from [PixelGen-XXL T2I](https://huggingface.co/zehongma/PixelGen), which is released under Apache-2.0.

## Citation

```bibtex
@inproceedings{yang2026pixeldense,
  title={PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion},
  author={Yang, Lehan and Qi, Daiqing and Zhang, Wenhao and Li, Avery and Yang, Yiqing and Li, Yifan and Kong, Yu and Zheng, Haitian and Zhang, Zhifei and Lin, Zhe and Jampani, Varun and Li, Sheng},
  booktitle={Advances in Neural Information Processing Systems},
  year={2026}
}
```

## Acknowledgement

PixelDense is built on [PixelGen](https://github.com/Zehong-Ma/PixelGen) and [DeCo](https://github.com/Zehong-Ma/DeCo). The teachers come from [DINOv2](https://github.com/facebookresearch/dinov2), [SAM 2](https://github.com/facebookresearch/sam2), [Depth Anything V2](https://github.com/DepthAnything/Depth-Anything-V2) and [Metric3D](https://github.com/YvanYin/Metric3D).