project_README_raw.md

# PXG-Tiny — offline text → 16×16 pixel-sprite model

**PXG-Tiny** is the deliberately miniature sibling of the Pixel AI (PXG) program:
a **483,040-parameter** generator that turns plain English into production-usable
16×16 pixel-art sprites (weapons, potions, treasure, flora, tiles, buildings…)
fully offline. It ships as a ~1 MB weight bundle and runs on a pure **NumPy**
inference runtime with **no PyTorch and no GPU required**.

**Author:** Chowdhury Tarul Ahsan · <tarulahsan@gmail.com> · v0.4 (2026-08-27)

**Headline numbers** (final acceptance, v0.4):
full-grid verification **38/38 classes — 152/152 prompts (100%)** ·
independent fresh-phrasing sweep **99.1% (116/117)** · raw held-out
(46 prompts, retry disabled) **59.4%** · showcase gallery **31/31** ·
val top-1 **98.89%** · bundle ~1.8 MB fp32

```
"a golden sword, glowing"   ──►  PXG-Tiny  ──►  16×16 RGBA sprite (+ retries,
                                                 clarify/refuse questions)
```

**Tags:** `text-to-image` `pixel-art` `game-assets` `sprite-generator`
`offline` `tiny-model` `numPy-runtime` `int8-quantization`
`self-supervised` `procedural-data` `gpt` `english-prompts`

---

## What it can do

| Capability | Detail |
|---|---|
| **38 grounded object classes** | weapons: sword, dagger, shield, staff · potions: round bottle, slim vial · treasure: chest, coin, gem, key · food: apple, bread · nature: oak/pine tree, bush, rock · tiles: grass/stone/water · building: house · **characters**: wizard, knight, archer, skeleton, zombie · **animals**: cat, dog, bird, fish, deer, mouse, bat · **furniture**: chair, table, stool · effects: fireball |
| Attribute grounding | materials (gold/iron/crystal/ruby/emerald/sapphire/amethyst/wood/copper/slate/terracotta), orientation (upright / lying sideways), glow, moss, berries, autumn leaves, size, roof colors — enforced only where the recipe supports them |
| Ask-first behavior | refuses photos / 3D / animations / other resolutions; asks clarifying questions on unknown objects or conflicting attribute words |
| Verifier-guided sampling | every generation is auto-checked against prompt-grounding constraints; failed seeds escalate through **self-guided palette bias → spatial priors (face box, vial margins, deer antler anchor) → canonical prompt anchoring** (8 tries) |
| Paraphrase-robust English | trained on **97,005** paraphrase-augmented rows across 38 classes: "made of pure iron", "of the crystal kind", "covered in moss", "whose orb is ruby", mid-adjectives, prose or comma forms |
| Deterministic & offline | char-level tokenizer with byte-fallback, fixed seeds, master-palette indices only |

## Quick start (no build step)

```bash
# single sprite
python3 cli.py "a golden sword" -o sword.png

# 4 variations on one sheet
python3 cli.py "an iron chest" --variations 4 --sheet chest_variants.png

# ask-first gate
python3 cli.py --ask "can you render a photo of my cat"

# batch: prompts.txt (one per line)
python3 cli.py --sheet-from-prompts prompts.txt --outdir sprites/
```

Python API:

```python
from pxg_tiny.pipeline import PXGPipeline

pipe = PXGPipeline()                # loads ./weights bundle
grid, meta = pipe.generate_pixels("a ruby potion that glows", seed=7)
if grid is None:
    print(meta["message"])          # clarify/refuse reply instead of pixels
else:
    png_path = pipe.generate_png("a ruby potion that glows",
                                 "ruby_potion.png")[1]["path"]
msg = PXGPipeline.ask("draw a 3d blender model")   # -> refusal message
```

Requirements: Python ≥3.9, `numpy`, `pillow` (rendering only; generation math
is NumPy-only).

## Architecture (483k params)

```
caption ids (32 slots)  ─► intent encoder (bi attn + ReLU FFN, fp32)
                        ─► 8 prefix vectors
[8 prefixes | 256 visual tokens]
  └─ 4× pre-LN causal blocks (d=96, 4 heads, FFN 256, tanh-GELU)
       └─ 32-way softmax over master-palette indices (0 = transparent alpha)

per-token accuracy 98%+ teacher-forced; KV-cached incremental decode at
inference (~0.6 s per sprite on 2 CPU cores).
```

*Visual "tokenizer"* is built-in by design: sprites are stored as grids of
32 master-palette indices, so decoding is byte-exact and alpha transparency
is exact by construction.

## Self-teaching training recipe

No external dataset was needed for v0.2:

1. A procedural rasterizer (11 asset families) draws sprites whose every
   caption word maps to a real rendered attribute — captions are honest by
   construction.
2. Counterfactual twin pairs share geometry but differ in exactly one
   attribute word ("gold" vs "iron"), teaching word→pixel causality.
3. **Paraphrase augmentation**: each ground-truth sprite is re-captioned in
   multiple surface forms (relative clauses, "made of/from", "of the … kind",
   "covered in moss", "lying sideways", polite frames, mid adjectives…) so the
   encoder learns paraphrase *invariance*. The final corpus is **97,005 rows**
   across 38 classes.
4. Quality gates (connectivity, isolated pixels, attribute thresholds) are
   calibrated so the ground-truth corpus passes 100%.

Training itself is `train/train_model.py` (CPU PyTorch, ~50 min @4,200 steps).
`train/quantize_export.py` exports the runnable bundle and proves numerical
parity against the torch reference.

## Bundle contents (`weights/`)

| file | role |
|---|---|
| `gen_int8.npz` | all model weights (see metadata for precision layout) |
| `vq_int8.npz` | master palette (32×4 RGBA lookup = visual codebook) |
| `runtime.json` | geometry + default sampling config |
| `tokenizer.json` | text tokenizer table |
| `metadata.json` | params, metrics, parity report, license, provenance |

`pxg_tiny/runtime_pipeline.OfflinePipeline` implements everything needed to
run it; `PXGPipeline` wraps it with gates, retries, turnarounds and PNG export.

## Testing

```bash
pip install pytest
pytest tests/ -q          # tokenizer, gates, NumPy kernels, end-to-end smoke
```

## Honest limitations

* 16×16 resolution only — larger canvases / spritesheets / tilemaps are
  refused by design (the ask-first gate explains this).
* Class vocabulary is the 38 trained families; anything outside gets a
  clarifying question rather than a hallucinated blob.
* Raw free-run sampling (verifier disabled) passes ~59% of hard held-out
  prompts; the verifier-guided pipeline lifts the shipped user path to
  ~100% (38-class grid) / 99.1% (fresh phrasings). Rare hard prompts can
  still end on an `accept_degraded` last-resort sprite (see EVAL_REPORT §5).
* Some material × class combos the recipes never trained (e.g. "emerald
  dagger") are silently downgraded to the class default instead of failing.
* Animation/multi-view consistency are roadmap items inherited from the main
  PXG program, not in this tiny cut.

See [EVAL_REPORT.md](EVAL_REPORT.md) for the full v0.1 → v0.4 evaluation
journey and honest weakness list, and [TUTORIAL.md](TUTORIAL.md) for a
5-minute quickstart.

## OpenGameArt-CC0 hook

v0.2 did not require external data (procedural ground truth gives perfect
caption alignment). To enrich distribution realism later: download a CC0-only
subset, record SHA256 + page URL per file in `data/oga_manifest.json`, run the
same quality gates, then extend `CLASS_DEFS`. License hygiene stays CC0-only.

## License

Code: **MIT** (see LICENSE). Weights + procedural corpus: **CC0** —
self-generated, no third-party assets ingested; sprites you generate are
yours to use anywhere, no attribution required.