🐟

Trout

A 47.6M-parameter diffusion language model for 16×16 pixel art. Each sprite is a 256-token “sentence” over a learned 512-color palette; a bidirectional hourglass transformer unmasks every pixel in parallel over ~16 steps, conditioned on a frozen CLIP text tower. No VAE, no latent space, no autoregression.

from scratch absorbing-state masked diffusion 47,597,568 params Pixel Art Atlas 1.4M · CC0 Apache-2.0

Samples — 16 prompts × 4 sprites (steps 16, CFG 3.0)

Trout sample grid

Prompts: red health potion bottle · golden crown with blue gems · green slime monster · iron sword with a leather hilt · wooden treasure chest · pink cherry blossom tree · blue magic spellbook · orange fox head · glowing purple crystal · cheeseburger · skull with a candle · rocket ship · brown leather boots · yellow rubber duck · ice cream cone · black cat.

How it works

Architecturejoint text+pixel bidirectional transformer, hourglass 2+8+2 blocks (2×2 patch-merged core), width 512, 8 heads, QK-RMSNorm, 2D axial RoPE, Conv-SwiGLU, adaLN-single, 4 registers, self-conditioning
Vocabulary512 palette tokens (frequency-weighted OKLab k-means, mean ΔE 0.0099) + <MASK>
Objectiveabsorbing-state masked diffusion, cosine schedule, stratified t, blob masks, perceptual OKLab label smoothing, CFG dropout
Samplerparallel confidence-ordered unmasking + CFG, Gumbel-annealed choice, spatial NMS, self-conditioning
Text encoderfrozen CLIP ViT-B/32 text tower (24 tokens)
OptimizerMuon (hidden matrices) + AdamW, Warmup-Stable-Decay, EMA
Training dataPixel Art Atlas 1.4M (Angais, CC0), 60k-sprite subset
Steps × batch2,400 × 8 sprites (CPU fp32, 2 vCPU / 4 GB)
Validation NLL2.164 nats / pixel (perplexity 8.81 of 512)
Validation accuracy55.8% of masked pixels
CLIP score (ViT-B/32)0.2285 generated · 0.2556 real
Reference runTessera-small: 82,342 × 512 (0.859 NLL) on one RTX PRO 6000

This is a genuine budget-limited from-scratch run on a CPU sandbox — roughly 4 orders of magnitude less compute than the reference architecture’s original training — not a distillation or fine-tune. All numbers come from metrics.jsonl in the model repo.

The learned 512-color palette

Token 0 = transparent (checker); the 511 opaque colors were chosen by frequency-weighted k-means in OKLab and snapped to real dataset colors.

Try it

# pip install torch transformers huggingface_hub pillow
from huggingface_hub import hf_hub_download
from trout import load_model_from_checkpoint, FrozenTextEncoder, sample, tokens_to_images, make_grid

ckpt = hf_hub_download("FrameXlabs/Trout", "trout-fp16.pt")
model, palette, meta = load_model_from_checkpoint(ckpt, device="cpu")
enc = FrozenTextEncoder(meta["text_encoder"], model.cfg.text_len, "cpu")

states, pooled = enc.encode(["golden crown with blue gems"] * 4)
toks = sample(model, states, pooled, steps=16, cfg_scale=3.0,
              generator=torch.Generator().manual_seed(0))
images = tokens_to_images(toks, palette)
make_grid(images, ncols=4).save("crowns.png")
This Space is a static page (free hosting). A complete interactive Gradio demo (prompt → sprites, with CFG/steps/temperature controls) is included in this Space’s repo (app.py, trout.py, requirements.txt) and in the model repo. Hugging Face currently requires a PRO subscription to host Gradio Spaces on free cpu-basic hardware; once available, pointing the Space SDK at Gradio makes the demo live unchanged.

Sources

  • Tessera-small by Jagrat Patel (TheJagStudio) — the published architecture this from-scratch model implements.
  • Pixel Art Atlas 1.4M by Angais — training data (CC0).
  • Sahoo et al., Simple and Effective Masked Diffusion Language Models, NeurIPS 2024 (arXiv:2406.07524).
  • Shi et al., Simplified and Generalized Masked Diffusion for Discrete Data, NeurIPS 2024 (arXiv:2406.04329).
  • Chang et al., MaskGIT, CVPR 2022 (arXiv:2202.04200) — parallel confidence-ordered decoding.
  • Blecher et al., Nougat (arXiv:2308.13418) — hourglass transformer.
  • Peebles & Xie, DiT (arXiv:2212.09748); PixArt-α — adaLN-single.
  • Darcet et al., Vision Transformers Need Registers, ICLR 2024.
  • Chen, Analog Bits (arXiv:2208.04202) — self-conditioning.
  • Jordan et al., Muon (arXiv:2502.16982); Moonshot Muon is Scalable (arXiv:2508.02535).
  • Ottosson, OKLab perceptual color space (2020).