A 47.6M-parameter diffusion language model for 16×16 pixel art. Each sprite is a 256-token “sentence” over a learned 512-color palette; a bidirectional hourglass transformer unmasks every pixel in parallel over ~16 steps, conditioned on a frozen CLIP text tower. No VAE, no latent space, no autoregression.
Prompts: red health potion bottle · golden crown with blue gems · green slime monster · iron sword with a leather hilt · wooden treasure chest · pink cherry blossom tree · blue magic spellbook · orange fox head · glowing purple crystal · cheeseburger · skull with a candle · rocket ship · brown leather boots · yellow rubber duck · ice cream cone · black cat.
| Architecture | joint text+pixel bidirectional transformer, hourglass 2+8+2 blocks (2×2 patch-merged core), width 512, 8 heads, QK-RMSNorm, 2D axial RoPE, Conv-SwiGLU, adaLN-single, 4 registers, self-conditioning |
| Vocabulary | 512 palette tokens (frequency-weighted OKLab k-means, mean ΔE 0.0099) + <MASK> |
| Objective | absorbing-state masked diffusion, cosine schedule, stratified t, blob masks, perceptual OKLab label smoothing, CFG dropout |
| Sampler | parallel confidence-ordered unmasking + CFG, Gumbel-annealed choice, spatial NMS, self-conditioning |
| Text encoder | frozen CLIP ViT-B/32 text tower (24 tokens) |
| Optimizer | Muon (hidden matrices) + AdamW, Warmup-Stable-Decay, EMA |
| Training data | Pixel Art Atlas 1.4M (Angais, CC0), 60k-sprite subset |
| Steps × batch | 2,400 × 8 sprites (CPU fp32, 2 vCPU / 4 GB) |
| Validation NLL | 2.164 nats / pixel (perplexity 8.81 of 512) |
| Validation accuracy | 55.8% of masked pixels |
| CLIP score (ViT-B/32) | 0.2285 generated · 0.2556 real |
| Reference run | Tessera-small: 82,342 × 512 (0.859 NLL) on one RTX PRO 6000 |
This is a genuine budget-limited from-scratch run on a CPU sandbox — roughly
4 orders of magnitude less compute than the reference architecture’s original
training — not a distillation or fine-tune. All numbers come from
metrics.jsonl in the model repo.
Token 0 = transparent (checker); the 511 opaque colors were chosen by frequency-weighted k-means in OKLab and snapped to real dataset colors.
# pip install torch transformers huggingface_hub pillow
from huggingface_hub import hf_hub_download
from trout import load_model_from_checkpoint, FrozenTextEncoder, sample, tokens_to_images, make_grid
ckpt = hf_hub_download("FrameXlabs/Trout", "trout-fp16.pt")
model, palette, meta = load_model_from_checkpoint(ckpt, device="cpu")
enc = FrozenTextEncoder(meta["text_encoder"], model.cfg.text_len, "cpu")
states, pooled = enc.encode(["golden crown with blue gems"] * 4)
toks = sample(model, states, pooled, steps=16, cfg_scale=3.0,
generator=torch.Generator().manual_seed(0))
images = tokens_to_images(toks, palette)
make_grid(images, ncols=4).save("crowns.png")
app.py, trout.py,
requirements.txt) and in the model repo. Hugging Face currently
requires a PRO subscription to host Gradio Spaces on free cpu-basic hardware;
once available, pointing the Space SDK at Gradio makes the demo live unchanged.