Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- diffusion
|
| 5 |
+
- ddpm
|
| 6 |
+
- pixel-art
|
| 7 |
+
- pokemon
|
| 8 |
+
- text-to-image
|
| 9 |
+
library_name: pytorch
|
| 10 |
+
pipeline_tag: unconditional-image-generation
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# Pokémon Pixel-Art Diffusion — checkpoints
|
| 14 |
+
|
| 15 |
+
Trained checkpoints for a **from-scratch DDPM** that generates 96×96 Pokémon pixel art
|
| 16 |
+
(by name, by free-text description, or by fusing two Pokémon).
|
| 17 |
+
|
| 18 |
+
**Code, model definitions & demo notebook:** https://github.com/weiee666/Pokemon_Combination
|
| 19 |
+
|
| 20 |
+
## Checkpoints
|
| 21 |
+
|
| 22 |
+
| file (`experiments/.../ckpt_ep300.pt`) | conditioning | attention | classes | params |
|
| 23 |
+
|---|---|---|---:|---:|
|
| 24 |
+
| `exp01_pokemon1070front_condUNet-CFG` | name embedding | – | 1070 | 4.2M |
|
| 25 |
+
| `exp02_pokemon20aug_condUNet-CFG-TB` | name embedding | – | 20 | 3.9M |
|
| 26 |
+
| `exp03_pokemon20aug_condUNet-CFG-attn-TB` | name embedding | self-attn | 20 | 5.8M |
|
| 27 |
+
| `exp04_pokemon20clean_bigUNet-CFG-attn-EMA` | name embedding | self-attn + EMA | 20 | 30.3M |
|
| 28 |
+
| `exp05_pokemon20aug_bigUNet-CFG-attn-EMA` | name embedding | self-attn + EMA | 20 | 30.3M |
|
| 29 |
+
| `exp06_stage1_pokemonALL_clipText` | CLIP pooled text | – | 988 | 30.5M |
|
| 30 |
+
| `exp07_stage1_pokemonALL_xattn` | CLIP per-token | self + cross-attn @24²/12² | 988 | 33.5M |
|
| 31 |
+
| `exp08_stage1_pokemonALL_xattn48` | CLIP per-token | self + cross-attn @48²/24²/12² | 988 | 34.0M |
|
| 32 |
+
| `exp09_stage1_pokemonALLcentered_xattn48` | CLIP per-token | self + cross-attn @48²/24²/12² | 988 | 34.0M |
|
| 33 |
+
|
| 34 |
+
`exp06_.../clip_table.pt` = pre-computed CLIP pooled vectors per class (needed by the exp06 model).
|
| 35 |
+
|
| 36 |
+
The checkpoints store **EMA weights** (for the experiments that use EMA). CLIP-conditioned
|
| 37 |
+
models (exp06–09) expect the CLIP ViT-B-32 (`laion2b_s34b_b79k`) text encoder.
|
| 38 |
+
|
| 39 |
+
## Load
|
| 40 |
+
|
| 41 |
+
```python
|
| 42 |
+
from huggingface_hub import hf_hub_download
|
| 43 |
+
import torch
|
| 44 |
+
|
| 45 |
+
ckpt = hf_hub_download(
|
| 46 |
+
"WEIEE/pokemon-diffusion",
|
| 47 |
+
"experiments/exp09_stage1_pokemonALLcentered_xattn48/checkpoints/ckpt_ep300.pt",
|
| 48 |
+
)
|
| 49 |
+
state = torch.load(ckpt, map_location="cpu")
|
| 50 |
+
# the matching UNet / Diffusion class definitions live in the GitHub notebooks
|
| 51 |
+
```
|
| 52 |
+
|
| 53 |
+
Educational / research project. Pokémon and all sprites are © Nintendo / Game Freak.
|