Nyx Image Official Icon

Nyx Image (NyxImage23M)

Sub-40M (23.19M) Discrete Masked Diffusion Transformer for Edge Pixel Art

Built by Sednium Technologies β€’ Licensed under Apache-2.0

Live Space Demo

Nyx Image 16-Sprite Showcase Grid

Nyx Image (NyxImage23M) is an ultra-compact 23.19 Million parameter discrete masked diffusion transformer engineered to synthesize authentic 16Γ—16 retro game sprites directly from text prompts. By eliminating latent autoencoders and continuous VAE bottlenecks, Nyx Image generates discrete pixel palette tokens with precise alpha transparency borders and high-contrast pixel shading in 16 parallel steps.


Key Features

  • Direct Discrete Palette Diffusion: Operates natively on 16Γ—16 discrete pixel tokens (vocab_size = 513) with built-in transparent background masking (Index 0).
  • Frozen CLIP Text Conditioning: Conditioned on frozen sequence embeddings (24 tokens Γ— 512-dim) and pooled projections from CLIP ViT-B/32.
  • Continuous Color Grounding: Infused with 6-frequency Fourier OKLab color representations, mapping discrete palette IDs into perceptual color space.
  • Asymmetric Hourglass Backbone: 2 High-Res blocks (16Γ—16) β†’ 2Γ—2 Patch Merge β†’ 6 Core Bottleneck blocks (8Γ—8 with Conv-SwiGLU) β†’ 2Γ—2 Patch Unmerge β†’ 2 High-Res Decoder blocks (16Γ—16).
  • Edge-Ready Deployments: Available in both native Safetensors (.safetensors) and universal ONNX (.onnx) formats for sub-millisecond edge and embedded deployment.
  • Commercial Apache-2.0 License: Complete freedom for commercial games, game jams, indie development, and personal projects.

Technical Comparison: How Nyx Image Compares

A transparent, grounded comparison of Nyx Image against the primary open and commercial approaches used for AI pixel art generation:

Model / Approach Architecture Model Size Target Output Native Alpha Transparency Sampling Steps Open Weights & License
Nyx Image 23M (Ours) Discrete Masked Diffusion (MDLM) 23.19M (~46 MB fp16) Native $16\times16$ Discrete Palette Yes (Index 0 true alpha) 16 parallel steps Open Weights (Apache-2.0)
Tessera-Small (thejagstudio) Discrete Masked Diffusion (MDLM) 47.6M (~95 MB fp16) Native $16\times16$ Discrete Palette Yes (Index 0 true alpha) 16–32 parallel steps Open Weights (CC BY-NC 4.0 Non-commercial)
SDXL Pixel Art (stablediffusionapi) Continuous Latent Diffusion (UNet+VAE) ~2,600M (2.6B) $1024\times1024$ Continuous RGB No (Solid background, color bleed) 30–50 steps Open Weights (OpenRAIL++ restrictions)
SD 1.5 PixelArt SpriteSheet Continuous Latent Diffusion (UNet+VAE) ~860M $512\times512$ Continuous RGB No (Requires manual chroma keying) 25–50 steps Open Weights (CreativeML OpenRAIL-M)
Retro Diffusion (Astropulse) Proprietary Diffusion + Downsampler Proprietary (~hundreds of MB) Multi-resolution Sprite Sheets Varies (plugin post-processing) Cloud / API Latency Closed Source / Paid Commercial Service
Discrete Autoregressive (Pixel Transformers) Autoregressive Sequence Model ~10M–30M Native $16\times16$ Discrete Tokens Yes 256 sequential steps ($O(N)$) Research Checkpoints / Varies

What Makes Nyx Image Different & Honest Trade-offs

  1. True Discrete Tokens vs. Continuous VAEs: Continuous models (SD 1.5, SDXL) generate continuous RGB pixels with anti-aliasing fuzziness, requiring post-process color reduction and downscaling. Nyx Image directly outputs palette indices from step 1, guaranteeing hard pixel boundaries and authentic retro shading.
  2. 2.05Γ— Smaller Footprint than Tessera: By utilizing a 2D patch-merge bottleneck ($16\times16 \to 8\times8 \to 16\times16$) with Conv-SwiGLU, Nyx Image achieves dense semantic representation with only 23.19M parameters compared to Tessera's flat 47.6M architecture.
  3. Inference-Time Bilateral Symmetrizer: Unlike previous discrete diffusion models that suffer from lopsided limbs or distorted wings, Nyx Image features an inference-time logit mirror-blending mechanism (symmetry_strength=0.5) that locks bilateral sprites into clean symmetry without retraining.
  4. Universal Edge Deployment: Exported directly to Safetensors and ONNX (Opset 17) with verified sub-millisecond execution for native Rust game engines (ort/tract) and mobile devices.
  5. Commercial License: Released under permissive Apache-2.0 for complete freedom in indie and commercial game development.

Truthful Limitations: Nyx Image is strictly specialized for $16\times16$ retro game sprites. It cannot generate high-resolution illustrations or larger canvas sizes without multi-tile compositing, and complex multi-subject prompts may lose fine detail within a 256-pixel canvas budget.


Model Weights & Formats

The model weights are provided in three standardized formats for different runtime environments:

File Name Format Size Target Environment
nyx-image-23m.safetensors Safetensors 88.5 MB PyTorch, Hugging Face safetensors, Python pipelines
nyx-image-23m.onnx ONNX (Opset 17) 90.4 MB ONNX Runtime, Rust (ort), C/C++, WebAssembly, Mobile
nyx-image-23m-fp16.pt PyTorch Checkpoint 46.5 MB Training resumption, PyTorch evaluation, EMA weights
palette_513.npy NumPy Array 1.6 KB 513-entry RGB color lookup table for pixel rendering

Quickstart & Inference Guide

1. Installation

pip install torch torchvision transformers pillow numpy safetensors onnxruntime

2. Python Inference (PyTorch & Safetensors)

import torch
import numpy as np
from PIL import Image
from transformers import CLIPTokenizer, CLIPTextModel
from safetensors.torch import load_file
from nyx_image import NyxImage23M, ModelConfig, sample_sprites, tokens_to_rgba_image

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# 1. Load Architecture & Safetensors Weights
config = ModelConfig()
model = NyxImage23M(config).to(device)
weights = load_file("nyx-image-23m.safetensors")
model.load_state_dict(weights, strict=False)
model.eval()

# 2. Load Color Palette
palette = np.load("palette_513.npy")
model.set_palette(palette)

# 3. Encode Prompt with CLIP
prompt = "healing potion bottle with pink sparkling liquid"
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-base-patch32")
text_model = CLIPTextModel.from_pretrained("openai/clip-vit-base-patch32").to(device)
text_model.eval()

inputs = tok([prompt], padding="max_length", max_length=24, truncation=True, return_tensors="pt").to(device)
with torch.no_grad():
    text_out = text_model(inputs["input_ids"])
    text_tokens = text_out.last_hidden_state  # (1, 24, 512)
    text_pooled = text_out.pooler_output      # (1, 512)

# 4. Generate 16x16 Sprite (16 Steps)
# Set symmetry_strength=0.5 for bilateral symmetry (dragons, monsters, shields, armor)
sampled_tokens = sample_sprites(
    model, text_tokens, text_pooled,
    steps=16, cfg_scale=3.0, temp_start=0.8, temp_end=0.2, choice_temp=1.0,
    symmetry_strength=0.0
)

# 5. Convert to RGBA and Save (Upscaled 8x to 128x128 for viewing)
img = tokens_to_rgba_image(sampled_tokens[0].cpu().numpy(), palette, upscale=8)
img.save("generated_potion.png")
print("Sprite generated successfully!")

3. Edge Inference with ONNX Runtime (Zero-PyTorch Pipeline)

Because Nyx Image is fully exported to ONNX, edge devices can execute the model with zero PyTorch training dependencies:

import onnxruntime as ort
import numpy as np
import math

# 1. Initialize ONNX Runtime Session
session = ort.InferenceSession("nyx-image-23m.onnx", providers=["CPUExecutionProvider"])
palette = np.load("palette_513.npy")

# Inputs expected by ONNX graph:
# - pixel_tokens: (B, 256) int64
# - timesteps: (B,) float32
# - text_tokens: (B, 24, 512) float32
# - text_pooled: (B, 512) float32
# - x0_self_cond: (B, 256, 512) float32
# - drop_text_mask: (B,) bool

def run_onnx_step(tokens, t, text_tokens, text_pooled, sc, drop):
    ort_inputs = {
        "pixel_tokens": tokens.astype(np.int64),
        "timesteps": np.array([t], dtype=np.float32),
        "text_tokens": text_tokens.astype(np.float32),
        "text_pooled": text_pooled.astype(np.float32),
        "x0_self_cond": sc.astype(np.float32),
        "drop_text_mask": np.array([drop], dtype=bool)
    }
    logits = session.run(["logits"], ort_inputs)[0]
    return logits

print("ONNX Runtime engine initialized.")

4. Running in Rust (Native Edge Game Engines)

The ONNX model can be embedded directly into native Rust game engines (Bevy, Macroquad, Fyrox) using ort or tract:

// Cargo.toml
// [dependencies]
// ort = "2.0"

use ort::session::Session;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let session = Session::builder()?
        .commit_from_file("nyx-image-23m.onnx")?;
    println!("Nyx Image ONNX model loaded successfully in native Rust runtime!");
    Ok(())
}

Technical Specifications

Parameter Specification Technical Rationale
Model Architecture NyxImage23M Discrete Masked Diffusion Transformer (MDLM)
Total Parameters 23,191,824 Optimal sub-40M capacity for detailed edge sprites
Canvas Resolution $16 \times 16$ (256 tokens) Standard retro game sprite dimensions
Vocabulary Size 513 Tokens Index 0 = Transparent, 1–511 = OKLab Palette, 512 = [MASK]
Hidden Dimension ($d_{\text{model}}$) 384 Balanced representational capacity and low FLOPs
Attention Heads 6 heads ($d_{\text{head}} = 64$) QK-RMSNorm stabilized multi-head attention
Bottleneck Structure $8 \times 8$ Grid (Patch Merge/Unmerge) Captures macro composition before fine detail decoding
Position Encoding 2D Axial Rotary (RoPE) Enforces strict Cartesian spatial relationships
Text Conditioning Frozen CLIP ViT-B/32 24 token sequence cross-attention + pooled global vector
Color Embedding Fourier OKLab (6 bands) Continuous perceptual color geometry per token
Inference Budget 16 Steps Fast parallel unmasking with Spatial NMS

Training Details

  • Dataset: Angais/pixel-art-atlas-1.4m (1.4M $16 \times 16$ pixel art game sprites)
  • Total Samples Processed: 6,400,000 sprites (4.57 full passes)
  • Hardware: Kaggle Dual Tesla T4 GPUs (Distributed Data Parallel)
  • Optimization: Fused AdamW ($\beta_1=0.9, \beta_2=0.95$) with cosine learning rate schedule
  • Loss: Focal weighted discrete cross-entropy ($w_0 = 0.15$ on background, $w_k = 1.0$ on foreground)
  • Final Convergence: 1.5569 nats (Perplexity: 4.74)

License

Nyx Image is licensed under the Apache-2.0 License.

Copyright 2026 Sednium Technologies

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

❀️ Made with Love by Bhoid

"A huge thank you to everyone in the open-source community for 1,000+ downloads on Charon v2! πŸŽ‰πŸš€ Honestly, I really didn't expect this much love and support on my very first fine-tune. You guys inspire me to keep building, experimenting, and open-sourcing cool AI models like Nyx. Thank you all so much! βœ¨πŸ™"

Sednium Technologies

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Dataset used to train Bhoidy/nyx-image-23m

Space using Bhoidy/nyx-image-23m 1

Evaluation results

  • Final Loss on Angais/pixel-art-atlas-1.4m
    self-reported
    1.5569 nats
  • Perplexity on Angais/pixel-art-atlas-1.4m
    self-reported
    4.740
  • Parameters on Angais/pixel-art-atlas-1.4m
    self-reported
    23191824.000