orbit-image / README.md
paymansuper's picture
Update README.md
2411e82 verified
|
Raw History Blame Contribute Delete
4.5 kB
---
license: cc-by-nc-4.0
library_name: pytorch
pipeline_tag: text-to-image
tags: [text-to-image, pixel-art, diffusion-transformer, flow-matching, repa, muon, starlex, orbit, from-scratch, research-preview]
extra_gated_heading: Orbit-Image research preview
extra_gated_prompt: >-
Orbit-Image is a research preview for non-commercial research. The licenses of the training images are still
being reviewed. By requesting access you agree to use it for non-commercial research only and not to present
its outputs as the work of the artists whose styles may appear in the training data.
extra_gated_fields:
Affiliation: text
Intended use: text
I agree to use this model for non-commercial research only: checkbox
---
# Orbit-Image v1 (research preview)
**A 182M parameter image model trained from scratch in one night on a single GPU. 15 of 16 test prompts show the
right subject.**
![orbit-image_16x9_v2](https://cdn-uploads.huggingface.co/production/uploads/6996621a93e0b3b5807ba110/7lo7DxAPe8NMe7C4TCbF8.png)
Orbit-Image turns short English prompts into pixel art. It is a pixel space diffusion transformer trained from
random weights by Starlex AI, and it reads prompts with our own language model
[Orbit-Nano](https://huggingface.co/StarlexAI/Orbit-Nano-Compliance-E2B) as its text encoder.
![samples](samples.png)
The grid shows 16 fixed test prompts after training (EMA weights, 50 steps, guidance 4), for example "a red
dragon", "a cozy cottage in the forest", "a wizard casting a spell" and "a city skyline at night".
| | |
|---|---|
| Parameters | 182M, trained from scratch |
| Training | about 11.5 hours on 1 GPU, 47,068 steps |
| Text encoder | Orbit-Nano by Starlex AI, frozen |
| Output | 128 x 128 px pixel art |
| License | CC-BY-NC-4.0, research preview |
## Try it
```bash
pip install torch transformers safetensors pillow
python orbit_image.py "a red dragon" "a blue potion bottle" --out dragon
```
The text encoder [StarlexAI/Orbit-Nano-Compliance-E2B](https://huggingface.co/StarlexAI/Orbit-Nano-Compliance-E2B)
is downloaded automatically (about 8 GB in bf16, runs with `trust_remote_code=True`). Images are saved upscaled x4
with nearest neighbour so the pixels stay sharp.
## How it was built
1. **No VAE.** The transformer works directly on pixels, patch 4 (1,024 tokens), d 768, 12 blocks, 12 heads,
QK-norm, adaLN conditioning and cross attention to the text tokens.
2. **x-prediction flow matching** (Li and He, "Back to Basics") with logit-normal time sampling and an Euler
sampler with classifier-free guidance (10 % caption dropout in training).
3. **Faster learning with REPA.** During training the model's features were aligned to a frozen DINOv2-base. The
REPA head is not part of this release, and DINOv2 is not needed for inference.
4. **Text.** Final hidden states of Orbit-Nano (64 tokens), projected into the transformer.
5. **Training.** Muon on the transformer matrices and AdamW on the rest, WSD schedule, EMA 0.9995, batch 256, bf16.
6. **Data.** Angais/pixel-art-atlas-1.4m, Chan-Y/pixelart-308k and azyg0s/pixelart-172k with equal weights,
center cropped and resized with nearest neighbour to 128 px.
## Limitations
- Research preview after one night of training. Objects on plain backgrounds work best. Complex scenes such as city
skylines or spaceships in space are often wrong.
- English prompts only, short prompts work best. Fixed resolution 128 x 128.
- The training images come from public pixel art collections whose original sources and licenses are not fully
documented. Outputs can resemble existing works or styles. Not for commercial use.
## The Orbit family
| Model | What it does |
|---|---|
| [Orbit-Nano-Compliance-E2B](https://huggingface.co/StarlexAI/Orbit-Nano-Compliance-E2B) | German and English compliance chat, rank 1 of 7 on our compliance benchmark |
| [Orbit-Nano-Compliance-E2B-GGUF](https://huggingface.co/StarlexAI/Orbit-Nano-Compliance-E2B-GGUF) | The same model for laptops |
| [Orbit Phone](https://huggingface.co/StarlexAI/orbit-phone) | Speech recognition, 2.60 % WER on LibriSpeech test-clean |
| [Orbit-Vision](https://huggingface.co/StarlexAI/orbit-vision) | Questions about images, charts and documents (research preview) |
| [Orbit-Image](https://huggingface.co/StarlexAI/orbit-image) | Pixel art from text |
## License
CC-BY-NC-4.0 for the weights and code in this repository. The text encoder has its own license (see its repo).
Built by Starlex AI, a Foundation AI Lab.