---
license: mit
base_model: amd/Nitro-E
datasets:
- emotiongoes/wikiart-themes-feelings
tags:
- text-to-image
- diffusion
- flow-matching
- lora
- nitro-e
- wikiart
- art
- emotiongoes
- more-art-than-science
library_name: diffusers
pipeline_tag: text-to-image
---

# Nitro-E · WikiArt Themes & Feelings (EMA-aligned)

A LoRA post-training of **[amd/Nitro-E](https://huggingface.co/amd/Nitro-E)** (304M E-MMDiT,
flow-matching, MIT) from AMD Brain that renders subjects in fine-art styles conditioned on art movement, genre, emotion and theme.
This repo holds the EMA-aligned, merged, serveable checkpoint
(`nitro_cc_ema_merged.safetensors`) plus the LoRA adapter it was merged from.

- **Base:** `amd/Nitro-E` (Nitro-E-512px), text encoder `meta-llama/Llama-3.2-1B` (gated), VAE `mit-han-lab/dc-ae-f32c32-sana-1.0`.
- **Resolution:** 512x512. **License:** MIT (base) — see data notes below.
- **Tracking:** [https://wandb.ai/imaging-ai/more-art-than-science](https://wandb.ai/imaging-ai/more-art-than-science)

## Files
- `nitro_cc_ema_merged.safetensors` — full E-MMDiT transformer state dict (LoRA merged into base). Load this to serve.
- `adapter_config.json` + `adapter_model.safetensors` — the PEFT LoRA adapter (apply on top of `amd/Nitro-E` if you prefer not to use the merged weights).

## Training data
— A private dataset of 32,061 WikiArt works (metadata + image URLs). ~29,000 images were downloadable and
used. The `consolidated-caption` field carries the emotional/style signal used as the prompt.

## Recipe
- **LoRA** rank 16 (alpha=32) on attention projections
  (`to_q,to_k,to_v,to_add_out,add_q_proj,add_k_proj,add_v_proj`); base frozen (full FT collapses the prior).
- Flow matching (logit-normal timestep sampling, SD3 loss weighting), AdamW lr 1e-05, cosine decay, bf16, SDPA.
- 2000 steps, batch 8 x grad-accum 2, CFG dropout 0.1. **EMA rate 0.9999**; best-val checkpoint kept
  (deterministic validation — fixed noise + stratified timesteps).
- Single AMD R9700 (ROCm). Trainer: [`finetune_wikiart.py`](https://github.com/mascharkh/more-art-than-science).

## Usage
Nitro-E uses AMD custom E-MMDiT pipeline (not diffusers-native). See the
[AMD-AGI/Nitro-E](https://github.com/AMD-AGI/Nitro-E) repo and this project`s
[`finetune_wikiart.py`](https://github.com/mascharkh/more-art-than-science) / `merge_and_sample.py`.
Example prompt: `"a tranquil river landscape, in the style of Impressionism, evoking calm"`.

## Limitations
- Inherits Nitro-E 304M quality ceiling; can overfit WikiArt style cues.
- Requires gated `meta-llama/Llama-3.2-1B` access to run.
- **Data rights:** trained on WikiArt images but does not redistribute them; the dataset is
  metadata-only. Many works are public domain but not all so use outputs accordingly.