---
license: mit
library_name: pytorch
pipeline_tag: text-to-image
tags:
  - pixel-art
  - weight-space-programming
  - hand-written-weights
  - no-training
  - tiny-model
  - pico-8
  - toy
language:
  - en
---

<div align="center">

# 🕹️ PixelGen

**A text-to-pixel-art model with 1,257 parameters. Zero of them were trained.**

Every weight was typed in by hand. It still works.

<img src="showcase.png" alt="11 sprite classes × 9 colour ramps, all generated by PixelGen" width="100%"/>

<sub>Every sprite above: `"<colour> <class>"` → PixelGen → 16×16 PICO-8 sprite. No gradients were harmed.</sub>

<br/>

![params](https://img.shields.io/badge/parameters-1%2C257-ff004d?style=flat-square)
![weights](https://img.shields.io/badge/weights-23.5%20KB-29adff?style=flat-square)
![training](https://img.shields.io/badge/training%20steps-0-00e436?style=flat-square)
![palette](https://img.shields.io/badge/palette-PICO--8-ffec27?style=flat-square)
![license](https://img.shields.io/badge/license-MIT-83769c?style=flat-square)

</div>

---

## What this is

A tiny PyTorch `nn.Module` that turns a text prompt like `"red potion"` or `"blue invader"` into a 16×16 pixel-art sprite in the PICO-8 palette, upscaled to 128×128.

It's an experiment in **weight-space programming**, inspired by [this post](https://rikiyat.github.io/blog/weight-space-programming/): instead of training a network, you write a program and *compile it into weights*. Linear layers become lookup tables, argmax becomes a decision, a 1×1 conv becomes a palette. The model is a real `nn.Module` — you can call `.forward()`, save a `state_dict`, load it back — but no optimizer ever touched it.

The fun part is that an image generator with the parameter count of a single small MLP layer produces consistently cute, correctly-coloured, correctly-shaped sprites, and you can read the entire "model" as source code.

## Numbers (measured, not estimated)

| | |
|---|---|
| Registered parameters | **1,257** |
| Non-trainable buffers (shape/shade/ramp banks) | 3,611 |
| Total values in the checkpoint | 4,868 (≈19.5 KB fp32) |
| `pixelgen.pt` on disk | 23.5 KB |
| Training steps | **0** |
| Optimizers, backward passes, `requires_grad=True` in the repo | 0 (grep-verified) |
| Sprite classes | 11 |
| Colour ramps | 9 |
| Output | 16×16 → 128×128 RGB, every pixel ∈ PICO-8 (16 colours) |
| Build reproducibility | `build.py` twice → identical SHA-256 |
| `pixelgen.pt` SHA-256 | `175cc5249fba8c756ea019f418f4aa0a9633017ffad7f238b863865758c49b80` |

<details>
<summary>Parameter breakdown</summary>

| Tensor | Shape | Count | What it is |
|---|---|---|---|
| `word_to_class.weight` | 11 × 59 | 649 | vocab → class scores (hand-set synonym table) |
| `word_to_class.bias` | 11 | 11 | default class bias (potion) |
| `word_to_ramp.weight` | 9 × 59 | 531 | vocab → colour-ramp scores |
| `word_to_ramp.bias` | 9 | 9 | default ramp bias (green) |
| `outline.weight` | 1×1×3×3 | 9 | 3×3 dilation kernel |
| `palette_conv.weight` | 3×16×1×1 | 48 | one-hot colour → RGB (exact PICO-8 values ÷ 255) |
| **Parameters** | | **1,257** | |
| `on_bank` / `free_bank` | 11×16×8 ×2 | 2,816 | half-sprite templates (`#` solid, `?` noise-controlled) |
| `shade_bank` | 3×16×16 | 768 | light / mid / dark row masks |
| `ramp_table` | 9×3 | 27 | palette indices per ramp |
| **Buffers** | | **3,611** | |

</details>

## How it works

The forward pass is nine steps. Each one is a normal tensor op; together they're a program.

```
"red potion"
    │
    ▼
 1. bag of words      prompt → count vector over a 59-word vocab
 2. class WTA         Linear(59→11) → argmax → one-hot class          [potion]
 3. shape bank        one-hot selects a 16×8 half-template;
                      '?' cells flip on when seed-noise ≥ 0.5
 4. mirror            half + horizontal flip → 16×16 body            (always symmetric)
 5. outline           3×3 dilate(body) − body → outline pixels       (palette index 1)
 6. shading           rows 0–6 light, 7–10 mid, 11–15 dark
 7. ramp WTA          Linear(59→9) → argmax → colour ramp             [red]
 8. one-hot palette   every pixel → one-hot over 16 PICO-8 colours
 9. palette conv      Conv2d(16→3, 1×1) → RGB
    │
    ▼
 16×16 sprite → nearest-neighbour ×8 → 128×128 PNG
```

The "generative" part is step 3: templates have `?` cells that are gated by per-seed noise, so the same prompt gives small variations across seeds:

<div align="center">
<img src="seeds.png" alt="blue invader, seeds 0–7" width="80%"/>
<sub><code>"blue invader"</code>, seeds 0 through 7</sub>
</div>

## How to use

```bash
git clone https://huggingface.co/DJLougen/pixelgen && cd pixelgen
pip install torch numpy pillow            # that's the whole dependency list
python generate.py "red potion" --seed 0 --out out/red_potion.png
python generate.py "golden key" --seed 3 --scale 16
```

Or from Python:

```python
from pixelgen.model import PixelGen

model = PixelGen()                        # weights are built in __init__, no file needed
img = model.sprite("purple slime", seed=4, scale=8)   # PIL.Image, 128×128
img.save("slime.png")
```

The `pixelgen.pt` checkpoint is included for completeness (`pixelgen.weights.load(path)`), but it's redundant — `PixelGen()` reconstructs identical weights from source every time.

**Prompt vocabulary.** Class words (and synonyms):

| class | words |
|---|---|
| potion | potion, bottle, flask, elixir, vial |
| sword | sword, blade, dagger |
| shield | shield, buckler |
| ship | ship, rocket, spaceship, starship |
| invader | invader, alien, monster |
| slime | slime, blob, jelly |
| skull | skull, bone, skeleton |
| tree | tree, pine, forest |
| key | key |
| heart | heart, love, life |
| star | star, sparkle |

Colour words: `red blue green yellow purple orange pink brown white`, plus synonyms (`crimson ruby → red`, `azure sapphire → blue`, `emerald lime → green`, `gold golden → yellow`, `violet amethyst → purple`, `amber → orange`, `rose → pink`, `wood wooden → brown`, `grey gray silver bone → white`). Anything else is ignored. Fun side effect: `"bone"` is both a skull synonym and a white synonym, so `"bone"` alone gives a white skull.

## Limitations

Being upfront, because this is the part that matters:

- **It only knows 11 things.** Prompt for a dragon and you get a potion (the default class). There is no generalisation — zero — because there is no learned representation. The "model" is a lookup table with a tokenizer in front of it.
- **Every sprite is horizontally symmetric.** The mirror step is what makes 16×8 templates cheap, and it's also why the key had to be drawn as a top-down key rather than a side-view one.
- **Variation is cosmetic.** Seeds only toggle the hand-marked `?` cells (a few pixels per sprite). Two seeds of the same prompt are the same sprite with slightly different speckles.
- **Shading is positional, not structural.** Light/mid/dark bands are fixed rows, so a tall sprite and a short sprite get lit differently for no good reason.
- **Multi-word prompts resolve by word count, not meaning.** `"sword shield"` is a tie broken by index order. `"not red"` is red.
- **Colour choice is global.** One ramp per sprite; no two-tone sprites.
- **It's PyTorch for the bit, not for the need.** You could rewrite this in 100 lines of NumPy. Using `nn.Linear` and `nn.Conv2d` is the point of the exercise, not a requirement.
- **Art quality is "programmer art".** The templates were hand-drawn in a text editor by someone who is not a pixel artist. `invader` and `key` went through a few iterations; the rest are first drafts.

## Why bother?

Because it's a nice demonstration that a lot of what we call "a model" is just *structure*, and structure can be written directly. A linear layer with the right weights *is* a dictionary. A conv with the right kernel *is* an edge detector. Winner-take-all *is* an `if`. The blog post argues this is a useful lens for understanding what trained networks are doing; this repo is the version where you skip the training and just write the weights.

Also it's cool that it worked.

## Repo contents

```
pixelgen/
  model.py       PixelGen nn.Module — all weights set in __init__
  templates.py   11 half-sprite templates ('#' solid, '?' noise, '.' empty)
  palette.py     PICO-8 palette + 9 colour ramps
  tokenizer.py   vocab + synonyms
  weights.py     save/load helpers
build.py         writes weights/pixelgen.pt (deterministic)
generate.py      CLI
tests/           10 property tests (shape, palette membership, determinism, symmetry, …)
weights/pixelgen.pt
showcase.png     the grid at the top
seeds.png        seed variation strip
```

Tests: `python -m pytest tests -q` → 10 passed.

## Citation

```bibtex
@misc{pixelgen2025,
  title  = {PixelGen: a hand-weighted pixel-art generator},
  author = {Lougen, D. J.},
  year   = {2025},
  url    = {https://huggingface.co/DJLougen/pixelgen},
  note   = {Weight-space programming experiment; no parameters were trained.}
}
```

Inspired by *[Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/)*.