sprite-gpt: rectified-flow diffusion for emoji + sprites (32px uncond, 64px class-cond)

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+samples_grid.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,101 @@
 ---
-license: mit
+license: cc-by-sa-4.0
+tags:
+- diffusion
+- rectified-flow
+- flow-matching
+- pixel-art
+- emoji
+- sprites
+- image-generation
+- pytorch
+library_name: pytorch
+pipeline_tag: unconditional-image-generation
 ---
+
+# sprite-gpt
+
+Small pixel-space diffusion models for emoji and game sprites, trained with
+rectified flow (flow matching) on a single RTX 5090. Code:
+[github.com/gmmeyer/sprite-diffusion](https://github.com/gmmeyer/sprite-diffusion)
+
+| file | model | params | training |
+|---|---|---|---|
+| `sprite-gpt-cond64.pt` | class-conditional 64px, 31 classes | 55.8M | 95k steps @ bs128 on 24,134 emoji+sprites |
+| `sprite-gpt-emoji32.pt` | unconditional 32px emoji | 29.1M | 60k steps @ bs256 on 8,504 emoji |
+
+Both are ADM-style U-Nets (scale-shift GroupNorm conditioning, self-attention
+at 16x16 and 8x8) trained with the rectified-flow objective
+(`x_t = (1-t)·x0 + t·ε`, predict `v = ε − x0`, logit-normal timestep
+sampling). Weights are fp32 EMA (decay 0.9995). Sampling: 50-step Euler ODE;
+the conditional model uses classifier-free guidance (null class = index 31,
+condition dropout 10% at train time; guidance 3 works well).
+
+![conditional samples](samples_grid.png)
+
+## Usage
+
+```python
+# git clone https://github.com/gmmeyer/sprite-diffusion && cd sprite-diffusion && uv sync
+import torch
+from spritegpt.unet import UNet
+from spritegpt.flow import sample
+from spritegpt.utils import save_image_grid
+
+ck = torch.load("sprite-gpt-cond64.pt", map_location="cuda", weights_only=False)
+cfg = ck["config"]
+model = UNet(
+    img_size=cfg["img_size"],
+    base=cfg["base"],
+    ch_mult=tuple(int(c) for c in cfg["ch_mult"].split(",")),
+    num_res=cfg["num_res"],
+    attn_res=tuple(int(r) for r in cfg["attn_res"].split(",")),
+    num_classes=cfg["num_classes"],
+).cuda()
+model.load_state_dict(ck["ema_state_dict"])
+model.eval()
+
+y = torch.full((16,), 24, device="cuda", dtype=torch.long)  # sprite/dcss-mon
+with torch.autocast("cuda", dtype=torch.bfloat16):
+    imgs = sample(model, 16, 64, steps=50, y=y, guidance=3.0,
+                  num_classes=cfg["num_classes"], device="cuda")
+save_image_grid(imgs.float(), "monsters.png", nrow=4)
+```
+
+## Classes (conditional model)
+
+Class ids are the alphabetical index into:
+
+`emoji/activities, emoji/animals-nature, emoji/component, emoji/extras-openmoji,
+emoji/extras-unicode, emoji/flags, emoji/food-drink, emoji/objects,
+emoji/people-body, emoji/smileys-emotion, emoji/symbols, emoji/travel-places,
+emoji/unknown, sprite/dawnlike, sprite/dcss-dungeon, sprite/dcss-effect,
+sprite/dcss-item, sprite/dcss-misc, sprite/dcss-mon, sprite/dcss-player,
+sprite/kenney-1bit, sprite/kenney-caves, sprite/kenney-characters,
+sprite/kenney-micro-roguelike, sprite/kenney-modern-city,
+sprite/kenney-pixel-platformer, sprite/kenney-pixel-shmup, sprite/kenney-rpg,
+sprite/kenney-tiny-battle, sprite/kenney-tiny-dungeon, sprite/kenney-tiny-town`
+
+## Training data and attribution
+
+Images composited onto white backgrounds at 32/64px:
+
+- [OpenMoji](https://openmoji.org) — CC-BY-SA 4.0 (the reason this model card
+  is CC-BY-SA)
+- [Twemoji](https://github.com/jdecked/twemoji) — CC-BY 4.0, © Twitter/X and
+  contributors
+- [Dungeon Crawl Stone Soup tiles](https://github.com/crawl/crawl) — public domain
+- [Kenney asset packs](https://kenney.nl) — CC0
+- [DawnLike tileset](https://opengameart.org/content/dawnlike-16x16-universal-rogue-like-tileset-v181)
+  — CC-BY 4.0, credit DragonDePlatino and DawnBringer
+
+## Limitations
+
+- The unconditional 32px emoji model partially memorizes its 8.5k-image
+  training set (~1/3 of samples are near-duplicates of training images).
+- Rare classes (a few hundred images) produce less diverse samples.
+- `sprite/kenney-characters` is trained on paper-doll layers (bodies,
+  clothing, hats as separate sprites), so it generates those parts rather
+  than assembled characters, and retains some flat-green artifacts from a
+  data-contamination issue fixed partway through training (see repo history).
+- Outputs are white-background RGB; no alpha channel.

sprite-gpt-cond64.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45ca5c1d11c0f6520e0367dbd718f60d35de78ff702d5b3b1378bfb86c55ac03
+size 223277955

sprite-gpt-emoji32.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09f0d2b99c1c3a6460ba51d49e9a851bbb1fe11cb7a9d6cd473f1a38f91d33a4
+size 116330147