--- license: mit pipeline_tag: text-to-image language: - en tags: - image - t2i - text-to-image - custom-code - tiny model-index: - name: PixelModel-v1 results: - task: type: text-to-image dataset: name: fid type: fid metrics: - name: fid type: fid value: 439.46 - task: type: text-to-image dataset: name: clip type: clip metrics: - name: clip type: clip value: 0.2002 new_version: bench-labs/PixelModel-v2 ---
๐Ÿ† BenchLabs Leaderboard โ€ข ๐ŸŒ Tiny-T2I Leaderboard โ€ข ๐Ÿ“ Blog Post
# PixelModel v1 ๐Ÿ–ผ๏ธ The entire PixelModel v1: a 160x149 PNG containing all 23,747 weights A neural network where the weights **are** the image. Now the model is a thumbnail. ## ๐Ÿ“Œ What is this? `model.png` is not a picture โ€” it *is* the model. Every pixel encodes neural network weights. At inference, the PNG is decoded into weight matrices, the prompt is hashed into an embedding, and a coordinate-conditioned decoder paints an image โ€” at **any resolution**. [v0](https://huggingface.co/bench-labs/pixelmodel) was a 202,752-parameter MLP welded to 32ร—32 output, trained on 6 color swatches. v1 is **8.5ร— smaller (23,747 parameters)**, renders at any resolution (native 64ร—64), and is trained on ~20K real MS-COCO caption/image pairs. # cherry on top ๐Ÿ’ The model generates 600 images (cpu) in 5 (five) seconds. Thats 5000 images in 24 seconds on cpu. The model trained on cpu for just 30 minutes. ## ๐Ÿ†• v0 โ†’ v1 | | v0 | v1 | |---|---|---| | Parameters | 202,752 | **23,747** | | `model.png` | 64ร—3200 px | **160ร—149 px** (a thumbnail) | | Output head | 1 weight row per pixel (196K params, 97% of model) | CPPN decoder on (x, y) โ€” resolution-free | | Resolution | fixed 32ร—32 | any; native 64ร—64 | | Prompt embedding | char-sum (order-blind, collision-heavy) | hashed trigrams + words (FNV-1a), still 0 params | | Biases | none | yes | | Weight precision | 8-bit (G channel wasted) | **16-bit** (R=high byte, G=low byte) | | Training data | 6 solid-color swatches | ~20K MS-COCO caption/image pairs | Training loss curve crossing below both naive-predictor baselines ## ๐ŸŽจ Weight Encoding Each pixel stores one weight at 16-bit precision, mapped from [-2, 2]: - **R channel** โ†’ high byte - **G channel** โ†’ low byte - **B channel** โ†’ reserved Round-trip quantization error โ‰ˆ 3ร—10โปโต per weight. ## ๐Ÿง  Architecture ```text prompt string โ†’ hashed char-trigram + word embedding (64-dim, deterministic, 0 params) โ†’ T1 (80ร—64)+b โ†’ tanh โ†’ T2 (64ร—80)+b โ†’ tanh = latent z (64) for every pixel (x, y): concat(z, fourier features of (x, y)) โ† 18 coord dims, freqs 1/2/4/8 โ†’ D1 (80ร—82)+b โ†’ tanh โ†’ D2 (80ร—80)+b โ†’ tanh โ†’ D3 (3ร—80)+b โ†’ sigmoid = RGB ``` Because pixels are decoded from coordinates, parameter count is independent of resolution โ€” `--res 256` works with the same 23,747 weights. All weights live inside `model.png` (160ร—149 px). ## ๐Ÿ“ฆ Standard weights (safetensors) `model.png` is the canonical model โ€” training writes to it directly. For tooling that expects standard weight files, the same 10 tensors are exported as `model.safetensors` (23,747 parameters total): ```bash python convert_to_safetensors.py # model.png -> model.safetensors ``` Parameter count is verifiable without running code: `config.json` (`total_parameters: 23747`, full per-layer breakdown) and the safetensors header metadata (`total_parameters`, `param_breakdown`, `has_bias`, `text_encoder_parameters: 0`, `vae_parameters: 0`). ## ๐Ÿ“Š Benchmark results Measured per the [Tiny-T2I-Leaderboard](https://huggingface.co/spaces/FlameF0X/Tiny-T2I-Leaderboard) protocol โ€” see `eval.md` for full method and `EVAL_REPRODUCTION.md` to re-run: | Metric | v0 | v1 | Tooling | |---|---|---|---| | FID โ†“ | 566.84 (n=40) | **439.46** (n=5000) | `torchmetrics.image.fid.FrechetInceptionDistance` | | CLIP Score โ†‘ | 18.60 (n=40) | **20.02** (n=5000) | `torchmetrics.multimodal.CLIPScore`, `openai/clip-vit-base-patch32` | | Native resolution | 32ร—32 | 64ร—64 | | Real set: MS-COCO val2014 pairs from `sayakpaul/coco-30-val-2014` (256ร—256 center-crop). Training images are hash-checked to be disjoint from the eval images. Expectation management: a 23K-parameter model does not draw recognizable objects. It learns caption-conditioned color, layout, and texture statistics of COCO photos โ€” the outputs are blurry impressions, not pictures. The point is the size:quality ratio and the PNG-as-model gimmick, executed honestly. Target photos vs model outputs: the model produces caption-conditioned color washes ## โš™๏ธ Usage ```bash # training data (COCO subset) python fetch_coco_subset.py --out ../pm-work # train (writes model.png every epoch) python train.py --data ../pm-work/coco_train.npz --epochs 50 # inference from model.png (canonical) python main.py "a red double decker bus" --out bus.png # inference from model.safetensors (standalone, needs only this one file + torch) python convert_to_safetensors.py python INFERENCE.py "a red double decker bus" --out bus.png # any resolution from the same 23,747 weights python main.py "a beach with palm trees" --res 256 --scale 1 # benchmark eval python eval/run_eval.py --work ../pm-work --model model.png --n 5000 ``` `main.py` and `INFERENCE.py` produce identical output for the same prompt and resolution. ## ๐Ÿ“ Files ```text model.png โ† THE MODEL (160ร—149 px) model.safetensors โ† same weights, standard format + param metadata config.json โ† architecture + parameter-count metadata main.py โ† inference, loads model.png INFERENCE.py โ† inference, loads model.safetensors (standalone) convert_to_safetensors.py โ† model.png -> model.safetensors train.py โ† training model.py โ† architecture + PNG weight codec fetch_coco_subset.py โ† builds train/eval data from COCO eval/run_eval.py โ† FID + CLIP Score via torchmetrics eval.md โ† benchmark results + method EVAL_REPRODUCTION.md โ† step-by-step reproduction guide ``` --- *Still a toy. Slightly less useless. The model is literally a thumbnail.* Bench Labs ยท Simple, Reliable, Open sourced