emotiongoes/nitro-e-wikiart

Nitro-E · WikiArt Themes & Feelings (EMA-aligned)

A LoRA post-training of amd/Nitro-E (304M E-MMDiT, flow-matching, MIT) from AMD Brain that renders subjects in fine-art styles conditioned on art movement, genre, emotion and theme. This repo holds the EMA-aligned, merged, serveable checkpoint (nitro_cc_ema_merged.safetensors) plus the LoRA adapter it was merged from.

• Base: amd/Nitro-E (Nitro-E-512px), text encoder meta-llama/Llama-3.2-1B (gated), VAE mit-han-lab/dc-ae-f32c32-sana-1.0.
• Resolution: 512x512. License: MIT (base) — see data notes below.
• Tracking: https://wandb.ai/imaging-ai/more-art-than-science

Files

• nitro_cc_ema_merged.safetensors — full E-MMDiT transformer state dict (LoRA merged into base). Load this to serve.
• adapter_config.json + adapter_model.safetensors — the PEFT LoRA adapter (apply on top of amd/Nitro-E if you prefer not to use the merged weights).

Training data

— A private dataset of 32,061 WikiArt works (metadata + image URLs). ~29,000 images were downloadable and used. The consolidated-caption field carries the emotional/style signal used as the prompt.

Recipe

• LoRA rank 16 (alpha=32) on attention projections (to_q,to_k,to_v,to_add_out,add_q_proj,add_k_proj,add_v_proj); base frozen (full FT collapses the prior).
• Flow matching (logit-normal timestep sampling, SD3 loss weighting), AdamW lr 1e-05, cosine decay, bf16, SDPA.
• 2000 steps, batch 8 x grad-accum 2, CFG dropout 0.1. EMA rate 0.9999; best-val checkpoint kept (deterministic validation — fixed noise + stratified timesteps).
• Single AMD R9700 (ROCm). Trainer: finetune_wikiart.py.

Usage

Nitro-E uses AMD custom E-MMDiT pipeline (not diffusers-native). See the AMD-AGI/Nitro-E repo and this projects [finetune_wikiart.py](https://github.com/mascharkh/more-art-than-science) / merge_and_sample.py. Example prompt: "a tranquil river landscape, in the style of Impressionism, evoking calm"`.

Limitations

• Inherits Nitro-E 304M quality ceiling; can overfit WikiArt style cues.
• Requires gated meta-llama/Llama-3.2-1B access to run.
• Data rights: trained on WikiArt images but does not redistribute them; the dataset is metadata-only. Many works are public domain but not all so use outputs accordingly.