orbit-image / README.md
paymansuper's picture
Update README.md
2411e82 verified
|
Raw History Blame Contribute Delete
4.5 kB
metadata
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: text-to-image
tags:
  - text-to-image
  - pixel-art
  - diffusion-transformer
  - flow-matching
  - repa
  - muon
  - starlex
  - orbit
  - from-scratch
  - research-preview
extra_gated_heading: Orbit-Image research preview
extra_gated_prompt: >-
  Orbit-Image is a research preview for non-commercial research. The licenses of
  the training images are still being reviewed. By requesting access you agree
  to use it for non-commercial research only and not to present its outputs as
  the work of the artists whose styles may appear in the training data.
extra_gated_fields:
  Affiliation: text
  Intended use: text
  I agree to use this model for non-commercial research only: checkbox

Orbit-Image v1 (research preview)

A 182M parameter image model trained from scratch in one night on a single GPU. 15 of 16 test prompts show the right subject.

orbit-image_16x9_v2

Orbit-Image turns short English prompts into pixel art. It is a pixel space diffusion transformer trained from random weights by Starlex AI, and it reads prompts with our own language model Orbit-Nano as its text encoder.

samples

The grid shows 16 fixed test prompts after training (EMA weights, 50 steps, guidance 4), for example "a red dragon", "a cozy cottage in the forest", "a wizard casting a spell" and "a city skyline at night".

Parameters 182M, trained from scratch
Training about 11.5 hours on 1 GPU, 47,068 steps
Text encoder Orbit-Nano by Starlex AI, frozen
Output 128 x 128 px pixel art
License CC-BY-NC-4.0, research preview

Try it

pip install torch transformers safetensors pillow
python orbit_image.py "a red dragon" "a blue potion bottle" --out dragon

The text encoder StarlexAI/Orbit-Nano-Compliance-E2B is downloaded automatically (about 8 GB in bf16, runs with trust_remote_code=True). Images are saved upscaled x4 with nearest neighbour so the pixels stay sharp.

How it was built

  1. No VAE. The transformer works directly on pixels, patch 4 (1,024 tokens), d 768, 12 blocks, 12 heads, QK-norm, adaLN conditioning and cross attention to the text tokens.
  2. x-prediction flow matching (Li and He, "Back to Basics") with logit-normal time sampling and an Euler sampler with classifier-free guidance (10 % caption dropout in training).
  3. Faster learning with REPA. During training the model's features were aligned to a frozen DINOv2-base. The REPA head is not part of this release, and DINOv2 is not needed for inference.
  4. Text. Final hidden states of Orbit-Nano (64 tokens), projected into the transformer.
  5. Training. Muon on the transformer matrices and AdamW on the rest, WSD schedule, EMA 0.9995, batch 256, bf16.
  6. Data. Angais/pixel-art-atlas-1.4m, Chan-Y/pixelart-308k and azyg0s/pixelart-172k with equal weights, center cropped and resized with nearest neighbour to 128 px.

Limitations

  • Research preview after one night of training. Objects on plain backgrounds work best. Complex scenes such as city skylines or spaceships in space are often wrong.
  • English prompts only, short prompts work best. Fixed resolution 128 x 128.
  • The training images come from public pixel art collections whose original sources and licenses are not fully documented. Outputs can resemble existing works or styles. Not for commercial use.

The Orbit family

Model What it does
Orbit-Nano-Compliance-E2B German and English compliance chat, rank 1 of 7 on our compliance benchmark
Orbit-Nano-Compliance-E2B-GGUF The same model for laptops
Orbit Phone Speech recognition, 2.60 % WER on LibriSpeech test-clean
Orbit-Vision Questions about images, charts and documents (research preview)
Orbit-Image Pixel art from text

License

CC-BY-NC-4.0 for the weights and code in this repository. The text encoder has its own license (see its repo).

Built by Starlex AI, a Foundation AI Lab.