license: mit
library_name: pytorch
pipeline_tag: text-to-image
tags:
- pixel-art
- weight-space-programming
- hand-written-weights
- no-training
- tiny-model
- pico-8
- toy
language:
- en
🕹️ PixelGen2
A text-to-pixel-art model with 6,747 parameters. Zero of them were trained.
Every weight was typed in by hand. It draws 21,866 distinct sprites from 28 templates.
Every sprite above is one prompt: "<colour> <body> with <colour> <accessory>". No gradients were harmed.
What this is
A tiny PyTorch nn.Module that turns a prompt like "red potion" or "purple ghost with gold crown" into a 16×16 pixel-art sprite in the PICO-8 palette, upscaled to 128×128.
It's an experiment in weight-space programming, inspired by this post: instead of training a network, you write a program and compile it into weights. Linear layers become lookup tables, argmax becomes a decision, a 1×1 conv becomes a palette. The model is a real nn.Module — you can call .forward(), save a state_dict, load it back — but no optimizer ever touched it.
There are two models in the repo:
PixelGen (v1) |
PixelGen2 (v2) |
|
|---|---|---|
| prompt | "<colour> <body>" |
"[small] <colour> <body> [with <colour> <accessory>]" |
| parameters | 4,227 | 6,747 |
| hand-drawn templates | 21 bodies | 21 bodies + 7 accessories |
| distinct sprites (seed 0) | 189 | 21,866 |
| shading | fixed row bands | structural rim light (two 3×3 convs) |
v1 is the minimal version, published separately as DJLougen/pixelgen; v2 is the same thing plus four compositional layers, and is what the CLI uses by default. This repo contains both. Everything below was measured, not estimated — scripts in experiments/.
How it works
Nine steps for v1. Each is an ordinary tensor op; together they're a program.
"red potion"
│
▼
1. bag of words prompt → count vector over a 138-word vocab
2. class WTA Linear(138→21) → argmax → one-hot body [potion]
3. shape bank one-hot selects a 16×8 half-template;
'?' cells flip on when seed-noise ≥ 0.5
4. mirror half + horizontal flip → 16×16 body (always symmetric)
5. outline 3×3 dilate(body) − body → outline pixels (palette index 1)
6. shading rows 0–6 light, 7–10 mid, 11–15 dark
7. ramp WTA Linear(138→9) → argmax → colour ramp [red]
8. one-hot palette every pixel → one-hot over 16 PICO-8 colours
9. palette conv Conv2d(16→3, 1×1) → RGB
│
▼
16×16 sprite → nearest-neighbour ×8 → 128×128 PNG
v2 inserts four more fixed-weight layers (pixelgen/compose.py):
| layer | mechanism | params |
|---|---|---|
| accessory | Linear(138→8) WTA selects a 16×8 part (crown, halo, horns, helmet, antenna, wings, torch) overlaid on the body with max. none wins ties. |
1,112 |
| second colour | the prompt is split at with/wearing/holding/and; a second Linear(138→9) colours the accessory from the words after it. No colour word there → body colour wins via a 0.5 one-hot bonus. |
1,251 |
| structural shading | two 3×3 convs with a single tap each: a pixel is light if its up-left neighbour is empty, dark if its down-right neighbour is. Light from the top-left, for every shape, for 18 parameters. | 18 |
| size | small/tiny/mini → nearest-neighbour 0.75× resample of the body map, bottom-centred. |
139 |
The "generative" part is step 3: templates have ? cells gated by per-seed noise, so the same prompt gives small variations across seeds:
"blue invader", seeds 0 through 7
Numbers
| v1 | v2 | |
|---|---|---|
| Registered parameters | 4,227 | 6,747 |
| Non-trainable buffers (template / shade / ramp / part banks) | 6,171 | 7,333 |
| Total values | 10,398 (≈41.6 KB fp32) | 14,080 (≈56.3 KB fp32) |
| Training steps | 0 | 0 |
Optimizers, backward passes, requires_grad=True in the repo |
0 (grep-verified) | 0 |
| Bodies / accessories / colour ramps | 21 / – / 9 | 21 / 7 / 9 |
| Distinct images at seed 0 | 189 of 189 prompts | 21,866 of 24,192 prompts |
| Output | 16×16 → 128×128 RGB, every pixel ∈ PICO-8 | same |
weights/pixelgen.pt (v1) |
45.7 KB, SHA-256 4c5dffd1…540eb, identical across rebuilds |
– |
Parameter breakdown (v2; v1 is the first six rows)
| Tensor | Shape | Count | What it is |
|---|---|---|---|
word_to_class.weight |
21 × 138 | 2,898 | vocab → body scores (synonym table) |
word_to_class.bias |
21 | 21 | default body (potion) |
word_to_ramp.weight |
9 × 138 | 1,242 | vocab → colour-ramp scores |
word_to_ramp.bias |
9 | 9 | default ramp (green) |
outline.weight |
1×1×3×3 | 9 | dilation kernel |
palette_conv.weight |
3×16×1×1 | 48 | one-hot colour → RGB (exact PICO-8 ÷ 255) |
word_to_acc.weight / .bias |
8 × 138, 8 | 1,112 | vocab → accessory scores, none biased |
word_to_ramp2.weight / .bias |
9 × 138, 9 | 1,251 | accessory-half vocab → ramp |
lit.weight, shadow.weight |
1×1×3×3 ×2 | 18 | up-left tap, down-right tap |
word_to_size.weight / .bias |
1 × 138, 1 | 139 | size words → small flag |
| Parameters | 6,747 | ||
on_bank / free_bank |
21×16×8 ×2 | 5,376 | body templates (# solid, ? noise-controlled) |
acc_bank |
8×16×8 | 1,024 | accessory templates |
shade_bank |
3×16×16 | 768 | row-band base shading |
ramp_table |
9×3 | 27 | palette indices per ramp |
acc_mask_a |
138 | 138 | body-class words never mean a hat in the body half |
| Buffers | 7,333 |
How to use
git clone https://huggingface.co/DJLougen/pixelgen2 && cd pixelgen2
pip install torch numpy pillow # that's the whole dependency list
python generate.py "red potion" --out out/red_potion.png
python generate.py "green slime with gold crown" --seed 3 --scale 16
python generate.py "small angel skull" --out out/skull.png
python generate.py "blue invader" --v1 # original single-part model
Or from Python:
from pixelgen.compose import PixelGen2
model = PixelGen2() # weights are built in __init__, no file needed
img = model.sprite("purple ghost with horns", seed=4, scale=8) # PIL.Image, 128×128
img.save("ghost.png")
Prompt grammar. [small] [colour] <body> [with [colour] <accessory>]. Words are matched by a bag-of-words, so order doesn't matter within each half.
Vocabulary
Bodies: potion (bottle, flask, elixir, vial) · sword (blade, dagger) · shield (buckler) · ship (rocket, spaceship, starship) · invader (alien, monster) · slime (blob, jelly) · skull (bone, skeleton) · tree (pine, forest) · key · heart (love, life) · star (sparkle) · mushroom (fungus, toadstool, shroom) · ghost (spirit, phantom, spooky) · bomb (grenade, explosive) · coin (token, money, medal) · gem (diamond, jewel, crystal) · cherry (cherries, berry, fruit) · crown · bat (vampire) · cactus (desert, succulent) · robot (droid, android, mech)
Accessories: crown (king, queen, royal, crowned, prince, princess) · halo (angel, angelic, holy, saint) · horns (horned, devil, demon, evil) · helmet (knight, armor, armour, armored) · antenna (antennae, bug, martian) · wings (winged, fly, flying, bird, fairy, butterfly) · torch (torches, fire, flame, burning)
Colours: red (crimson, ruby) · blue (azure, sapphire) · green (emerald, lime) · yellow (gold, golden) · purple (violet, amethyst) · orange (amber) · pink (rose) · brown (wood, wooden) · white (grey, gray, silver, bone)
Size: small, tiny, mini, little. Separators: with, wearing, holding, and, plus.
"crown" in the body half is the crown object; after a separator it's a hat. "bone" is both a skull synonym and a white synonym, so "bone" alone is a white skull. "alien" is an invader body and "martian" is an antenna accessory, so "martian alien" is what you'd hope.
How far can you push it?
Four experiments, all in experiments/ with the scripts that produced them. Full notes in experiments/README.md.
1. Adding a class is a pure append. Freeze the 11-class model, add mushroom, diff. +400 values. Diffed by index, 84 old values changed; diffed by meaning (class name × word), zero — the raw diff was entirely a column permutation from a sorted() vocab. 17/17 old prompts bit-identical over 3 seeds. The same check still passes after +10 classes and the whole accessory vocabulary.
2. Capacity is unbounded; addressing is quadratic. Synthetic random classes, rebuilt and probed at each size:
| classes | params | dense layer share | isolated |
|---|---|---|---|
| 21 | 2,937 | 22% | ✓ |
| 121 | 51,537 | 57% | ✓ |
| 1,021 | 3.2M | 92% | ✓ |
| 10,021 | 302M | 99.1% | ✓ |
It never breaks — the original 21 classes stay bit-identical at 10k. But vocab grows with classes, so word_to_class is ≈3C² and 99.997% zeros. Under 100k params the dense design tops out near 170 classes.
3. Compressing the class layer costs semantics. A binary code (d = ⌈log₂C⌉, words emit ±1 codes, score = d − 2·hamming) shrinks the class layer 528× at 10k classes with no loss of isolation — but 61% of two-body-word prompts ("sword shield") now resolve to a third class, vs 0% dense. The additive bag-of-words semantics were the thing that got compressed away.
4. Composition is the lever. v1 → v2: 1.6× params, 115× distinct outputs. 781 distinct images per hand-drawn template instead of 9.
Top: v1 row bands. Bottom: v2 structural lighting, 18 params.
Quality, honestly: structural lighting helps blobby shapes (potion, heart, mushroom, tree, gem) and hurts thin ones (invader, bat, robot) — when every pixel is an edge, rim light becomes speckle. A fixed 3×3 kernel has no way to know a shape is too thin to light. That's the limit of hand-written rules without a learned prior, and exactly the kind of thing a few hundred trained parameters would fix.
Limitations
- It only knows what was drawn. 21 bodies, 7 accessories. Prompt for a dragon and you get a potion (the default). Capacity is unbounded; content is a human with a text editor.
- Every shape is horizontally symmetric. The mirror step makes 16×8 templates cheap; it also means
"torch"is two torches and the key is a top-down key. Lighting (v2) is deliberately not symmetric. - Accessories sit at fixed rows. A crown floats above short bodies; a helmet merges into tall ones. Fixing that needs per-body anchor points — more drawing, not more mechanism.
- One accessory per sprite.
"devil knight"picks one by index order. - Variation is cosmetic. Seeds only toggle the
?cells — a few pixels per sprite. - Bag of words.
"sword shield"is a tie broken by index;"not red"is red; within each half, order is ignored. - Structural shading speckles thin shapes. See above.
- It's PyTorch for the bit, not for the need. You could rewrite this in NumPy. Using
nn.Linearandnn.Conv2dis the point of the exercise. - Art quality is programmer art, drawn in a text editor by someone who is not a pixel artist.
Why bother?
Because it's a nice demonstration that a lot of what we call "a model" is just structure, and structure can be written directly. A linear layer with the right weights is a dictionary. A conv with the right kernel is an edge detector. Winner-take-all is an if. The blog post argues this is a useful lens for understanding what trained networks are doing; this repo is the version where you skip the training and just write the weights — and then push on it to see where hand-writing stops being enough.
Where it stops: addressing (quadratic), compression (breaks additivity), and quality rules at the edges (shading thin shapes). The natural next step is the hybrid — hand-write the ~95% that's structure, train only the small word→part map.
Also it's cool that it worked.
Repo contents
pixelgen/
model.py PixelGen (v1) — all weights set in __init__
compose.py PixelGen2 (v2) — v1 + accessory, second colour, structural shading, size
templates.py 21 body half-templates ('#' solid, '?' noise, '.' empty)
parts.py 7 accessory half-templates
palette.py PICO-8 palette + 9 colour ramps
tokenizer.py vocab, synonyms, prompt splitting
weights.py save/load helpers
build.py writes weights/pixelgen.pt (v1, deterministic)
generate.py CLI (v2 default, --v1 for the original)
tests/ 16 property tests
experiments/ the four experiments above: scripts, frozen baselines, README
showcase2.png the grid at the top
seeds.png seed variation strip
Tests: python -m pytest tests -q → 16 passed.
Citation
@misc{pixelgen2_2025,
title = {PixelGen2: a compositional hand-weighted pixel-art generator},
author = {Lougen, D. J.},
year = {2025},
url = {https://huggingface.co/DJLougen/pixelgen2},
note = {Weight-space programming experiment; no parameters were trained.}
}
Inspired by Weight-Space Programming.