| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - image-generation |
| - pixel-art |
| - rectified-flow |
| - onnx |
| - webgpu |
| pipeline_tag: unconditional-image-generation |
| library_name: onnxruntime |
| --- |
| # Sprite diffusion |
|
|
| A 24.2M-parameter, category-conditioned rectified-flow transformer for **32×32 RGBA pixel-art sprites**. This repository distributes inference-only EMA weights for the original 100k model and its 150k continuation, plus ONNX exports for a browser generator. It is not a text-to-image model: it accepts eight category labels, not natural-language prompts. |
|
|
| [Code and training write-up](https://github.com/Cyronius/model-experiments/tree/main/sprite-diffusion) · [Browser generator](https://cyronius.github.io/model-experiments/sprite-diffusion/) |
|
|
| ## Files and use |
|
|
| - `sprite-100k.safetensors`, `sprite-150k.safetensors`: float32 EMA state dictionaries, no optimizer, training arguments, sampler state or datasets. |
| - `sprite-100k.onnx`, `sprite-150k.onnx`: float32 velocity networks, opset 17, fixed batch two for conditional/unconditional classifier-free guidance. |
| - Matching JSON files record architecture, artifact hashes and local PyTorch/ONNX parity checks. |
| - `dit.py` defines the PyTorch architecture (Apache-2.0; see `LICENSE-CODE`). Use `DiT(384, 10, 2)`, load a safetensors state dictionary, then run Euler integration from Gaussian noise at t=1 to t=0. |
|
|
| ONNX inputs: `x` float32 `[2,4,32,32]`, `t` float32 `[2]`, `category` int64 `[2]`, `footprint` int64 `[2]`. The output is `velocity` float32 `[2,4,32,32]`. For a category c, duplicate the latent into both batch rows, use category `[c,8]`, footprint `[3,3]`, and combine velocities as `unconditional + 3 * (conditional - unconditional)`. Take a negative Euler step of `1 / steps`. Scale the final clamped [-1,1] RGBA to [0,255], with alpha threshold 128. |
|
|
| Categories, in order: creature, item, weapon, food, effect, terrain, structure, nature. The food category includes drinks and potions. |
|
|
| **Sampling:** 32 steps is the owner's preferred default; 16 is balanced and 8 is fast. 1–2 steps produce underdeveloped outputs. 64 is not reliably better. Keep both checkpoints: a qualitative same-seed comparison did not establish a universal winner. The 150k model showed more nature variety, while the original retained more varied characters in the inspected seeds. |
|
|
| ## Training and evaluation |
|
|
| Hidden width 384, 10 blocks, three 128-dimensional attention heads, patch size 2 and 256 tokens. The original checkpoint ran for 100k updates. The continuation used an expanded 383,660-image/frame dataset, 3e-5 learning rate, category/subtype-balanced sampling and palette augmentation: a 10k-update pilot reset the optimizer, then another 40k resumed that pilot. Creature subtypes affect exposure only; they are not output labels. |
|
|
| The comparison used 100k/110k/150k checkpoints at 1,2,4,8,12,16,32,64 steps, 64 sprites per setting, two fixed seed sets, CFG 3, EMA weights and no palette snapping. One conversational vision review inspected 1,536 generated sprites. It was not an independent blinded evaluation. Unseen validation and a matched uniform control remain outstanding; no new formal copying or diversity evaluation is claimed. |
|
|
| ## License |
|
|
| Model weights and code are licensed under [Apache 2.0](LICENSE). |
|
|
