How to Generate Intentionally
WVY-Image-Composition-101 · by Starpower
This isn't an image model you use. It's one you learn from.
People say you need millions of images to make a model generate anything. That's true for a model that's supposed to do everything. For a model built for one specific job, you don't need a pile of data. You need intentional data: a few clean pieces, designed so they can be swapped and recombined.
This repo proves it with a 7.8M-parameter text-to-image model written from scratch in numpy. It was trained on 6 transparent PNGs and 8 background colors, and it correctly draws scenes it was never shown.
The idea in one picture
Each animal is one transparent PNG. Scenes get stacked like layers (background → bottom animal → top animal), and every image gets an exact caption:
a dog on top of a cat on a green background
6 animals × 5 partners × 2 layouts × 8 colors = 480 possible scenes, all made from 14 ingredients.
The test: does it build, or does it copy?
The model only studies 361 of those scenes. The other 119 are held back:
| Split | Count | What it tests |
|---|---|---|
| Seen in training | 361 | memory |
| Never seen · new color | 87 | It knows this arrangement, just never on this background. Can it move it onto a new color? |
| Never seen · flipped stack | 32 | "cat on top of dog" was hidden in every color. It only saw "dog on top of cat." Can it flip the stack from word order alone? |
Results
| Split | Prompts | Pixel error (MSE) | Drew the right scene* | Closer to the correct answer than to anything it studied |
|---|---|---|---|---|
| Seen in training | 361 | 0.000002 | 100% | 100% |
| Never seen · new color | 87 | 0.00019 | 100% | 100% |
| Never seen · flipped stack | 32 | 0.00057 | 100% | 100% |
*Out of all 480 possible scenes, the generated image is closest to the one the prompt asked for.
The hardest test. Left: what it generated. Middle: the correct answer, which it never saw. Right: the closest image it actually studied. The right column always has the wrong animal, so it isn't copying.
"a cat on top of a dog" in all 8 colors. Every one of these is a prompt it never saw:
How it works
prompt → words → each word = 24 numbers + a tag for its position in the sentence
→ layer 1 (512 neurons) → layer 2 (1024 neurons) → 48×48 RGB pixels
- The position tag matters. "cat … dog" and "dog … cat" use the exact same words, so word order is the only thing telling the model which animal goes on top.
- Forward pass, backprop and the Adam optimizer are all hand-written in numpy. Gradients were checked against numeric estimates before training.
- Nothing is pretrained. All 7.8M weights start random.
- Training took 9,000 steps on 2 CPU cores, about 21 minutes.
The never-seen lines drop along with the training line, which means it's learning the parts, not memorizing the pictures.
What it gets wrong (look for this)
On never-seen scenes you can spot faint ghosts: blurry outlines of other animals leaking into the background. That's the model blending in "the average of everything it ever saw in that spot." It's the seam between memorization and generalization, and it's exactly the kind of mush intentional data is meant to squeeze out.
Honest limits
- The animals are simple drawings made in code, not photos. Flat drawings skip lighting interactions, like a red wall casting red light on fur.
- It's deterministic. The same prompt always gives the same image, so there's no variety.
- It only reads two sentence shapes. Anything else won't parse.
- It can't draw anything outside its 14 ingredients. Inside them, it can draw combinations it never saw.
Try it
Needs only numpy and Pillow.
python3 generate.py "a cat on top of a dog on a green background" -o cat_dog.png
python3 generate.py "a fish next to a pig on a purple background" -o fish_pig.png
Prompt shapes (the captions always say "a", even before orange):
a <animal> on top of a <animal> on a <color> background
a <animal> next to a <animal> on a <color> background
animals: dog cat bird fish pig rabbit
colors: white black red orange yellow green blue purple
Retrain from zero and re-run the tests:
python3 model.py # builds the dataset, trains, saves weights (~20 min on CPU)
python3 evaluate.py # scores never-seen prompts, makes the figures
Files
| File | What it is |
|---|---|
data.py |
draws the 6 sprites, stacks the scenes, writes captions, makes the train / never-seen split |
model.py |
the numpy model, hand-written backprop, Adam, training loop |
evaluate.py |
metrics + figures |
generate.py |
prompt → image |
wvy_compose_weights.npz |
trained weights |
sprites/ |
the 6 ingredient PNGs |
train_log.json, eval.json |
training curve and scores |
media/ |
the explainer video and figures |
Don't pile up data. Sculpt it. · WVY · Starpower