README.md

---
license: mit
pipeline_tag: text-to-image
language:
- en
tags:
- image
- t2i
- text-to-image
- custom-code
- tiny
model-index:
- name: PixelModel-v1
  results:
  - task:
      type: text-to-image
    dataset:
      name: fid
      type: fid
    metrics:
    - name: fid
      type: fid
      value: 439.46
  - task:
      type: text-to-image
    dataset:
      name: clip
      type: clip
    metrics:
    - name: clip
      type: clip
      value: 0.2002
new_version: bench-labs/PixelModel-v2
---

<table align="center">
<tr>
<td align="center">

<a href="https://huggingface.co/spaces/bench-labs/BenchLabs-Leaderboard">🏆 BenchLabs Leaderboard</a> •
<a href="https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard">🌍 Tiny-T2I Leaderboard</a> •
<a href="https://huggingface.co/spaces/bench-labs/blog?post=pixelmodel-v1.html">📝 Blog Post</a>

</td>
</tr>
</table>

# PixelModel v1 🖼️

<img src="model.png" alt="The entire PixelModel v1: a 160x149 PNG containing all 23,747 weights" style="image-rendering: pixelated; width: 320px; max-width: 100%;">


A neural network where the weights **are** the image. Now the model is a thumbnail.

## 📌 What is this?

`model.png` is not a picture — it *is* the model.

Every pixel encodes neural network weights. At inference, the PNG is decoded
into weight matrices, the prompt is hashed into an embedding, and a
coordinate-conditioned decoder paints an image — at **any resolution**.

[v0](https://huggingface.co/bench-labs/pixelmodel) was a 202,752-parameter MLP
welded to 32×32 output, trained on 6 color swatches. v1 is **8.5× smaller
(23,747 parameters)**, renders at any resolution (native 64×64), and is trained
on ~20K real MS-COCO caption/image pairs.

# cherry on top 🍒
The model generates 600 images (cpu) in 5 (five) seconds.
Thats 5000 images in 24 seconds on cpu.
The model trained on cpu for just 30 minutes.

## 🆕 v0 → v1

| | v0 | v1 |
|---|---|---|
| Parameters | 202,752 | **23,747** |
| `model.png` | 64×3200 px | **160×149 px** (a thumbnail) |
| Output head | 1 weight row per pixel (196K params, 97% of model) | CPPN decoder on (x, y) — resolution-free |
| Resolution | fixed 32×32 | any; native 64×64 |
| Prompt embedding | char-sum (order-blind, collision-heavy) | hashed trigrams + words (FNV-1a), still 0 params |
| Biases | none | yes |
| Weight precision | 8-bit (G channel wasted) | **16-bit** (R=high byte, G=low byte) |
| Training data | 6 solid-color swatches | ~20K MS-COCO caption/image pairs |

<img src="pixelmodel-v1-loss.png" alt="Training loss curve crossing below both naive-predictor baselines" style="width: 100%; max-width: 720px;">

## 🎨 Weight Encoding

Each pixel stores one weight at 16-bit precision, mapped from [-2, 2]:

- **R channel** → high byte
- **G channel** → low byte
- **B channel** → reserved

Round-trip quantization error ≈ 3×10⁻⁵ per weight.

## 🧠 Architecture

```text
prompt string
  → hashed char-trigram + word embedding (64-dim, deterministic, 0 params)
  → T1 (80×64)+b → tanh
  → T2 (64×80)+b → tanh          = latent z (64)

for every pixel (x, y):
  concat(z, fourier features of (x, y))   ← 18 coord dims, freqs 1/2/4/8
  → D1 (80×82)+b → tanh
  → D2 (80×80)+b → tanh
  → D3 (3×80)+b  → sigmoid  = RGB
```

Because pixels are decoded from coordinates, parameter count is independent of
resolution — `--res 256` works with the same 23,747 weights.

All weights live inside `model.png` (160×149 px).

## 📦 Standard weights (safetensors)

`model.png` is the canonical model — training writes to it directly. For
tooling that expects standard weight files, the same 10 tensors are exported as
`model.safetensors` (23,747 parameters total):

```bash
python convert_to_safetensors.py            # model.png -> model.safetensors
```

Parameter count is verifiable without running code: `config.json`
(`total_parameters: 23747`, full per-layer breakdown) and the safetensors
header metadata (`total_parameters`, `param_breakdown`, `has_bias`,
`text_encoder_parameters: 0`, `vae_parameters: 0`).

## 📊 Benchmark results

Measured per the [Tiny-T2I-Leaderboard](https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard) protocol — see `eval.md` for full method and `EVAL_REPRODUCTION.md` to re-run:

| Metric | v0 | v1 | Tooling |
|---|---|---|---|
| FID ↓ | 566.84 (n=40) | **439.46** (n=5000) | `torchmetrics.image.fid.FrechetInceptionDistance` |
| CLIP Score ↑ | 18.60 (n=40) | **20.02** (n=5000) | `torchmetrics.multimodal.CLIPScore`, `openai/clip-vit-base-patch32` |
| Native resolution | 32×32 | 64×64 | |

Real set: MS-COCO val2014 pairs from `sayakpaul/coco-30-val-2014` (256×256
center-crop). Training images are hash-checked to be disjoint from the eval
images.

Expectation management: a 23K-parameter model does not draw recognizable
objects. It learns caption-conditioned color, layout, and texture statistics of
COCO photos — the outputs are blurry impressions, not pictures. The point is
the size:quality ratio and the PNG-as-model gimmick, executed honestly.

<img src="pixelmodel-v1-probe.png" alt="Target photos vs model outputs: the model produces caption-conditioned color washes" style="width: 100%; max-width: 640px;">

## ⚙️ Usage

```bash
# training data (COCO subset)
python fetch_coco_subset.py --out ../pm-work

# train (writes model.png every epoch)
python train.py --data ../pm-work/coco_train.npz --epochs 50

# inference from model.png (canonical)
python main.py "a red double decker bus" --out bus.png

# inference from model.safetensors (standalone, needs only this one file + torch)
python convert_to_safetensors.py
python INFERENCE.py "a red double decker bus" --out bus.png

# any resolution from the same 23,747 weights
python main.py "a beach with palm trees" --res 256 --scale 1

# benchmark eval
python eval/run_eval.py --work ../pm-work --model model.png --n 5000
```

`main.py` and `INFERENCE.py` produce identical output for the same prompt and
resolution.

## 📁 Files

```text
model.png                   ← THE MODEL (160×149 px)
model.safetensors           ← same weights, standard format + param metadata
config.json                 ← architecture + parameter-count metadata
main.py                     ← inference, loads model.png
INFERENCE.py                ← inference, loads model.safetensors (standalone)
convert_to_safetensors.py   ← model.png -> model.safetensors
train.py                    ← training
model.py                    ← architecture + PNG weight codec
fetch_coco_subset.py        ← builds train/eval data from COCO
eval/run_eval.py            ← FID + CLIP Score via torchmetrics
eval.md                     ← benchmark results + method
EVAL_REPRODUCTION.md        ← step-by-step reproduction guide
```

---

*Still a toy. Slightly less useless. The model is literally a thumbnail.*

Bench Labs · Simple, Reliable, Open sourced