| --- |
| license: mit |
| base_model: amd/Nitro-E |
| datasets: |
| - emotiongoes/wikiart-themes-feelings |
| tags: |
| - text-to-image |
| - diffusion |
| - flow-matching |
| - lora |
| - nitro-e |
| - wikiart |
| - art |
| - emotiongoes |
| - more-art-than-science |
| library_name: diffusers |
| pipeline_tag: text-to-image |
| --- |
| |
| # Nitro-E · WikiArt Themes & Feelings (EMA-aligned) |
|
|
| A LoRA post-training of **[amd/Nitro-E](https://huggingface.co/amd/Nitro-E)** (304M E-MMDiT, |
| flow-matching, MIT) from AMD Brain that renders subjects in fine-art styles conditioned on art movement, genre, emotion and theme. |
| This repo holds the EMA-aligned, merged, serveable checkpoint |
| (`nitro_cc_ema_merged.safetensors`) plus the LoRA adapter it was merged from. |
|
|
| - **Base:** `amd/Nitro-E` (Nitro-E-512px), text encoder `meta-llama/Llama-3.2-1B` (gated), VAE `mit-han-lab/dc-ae-f32c32-sana-1.0`. |
| - **Resolution:** 512x512. **License:** MIT (base) — see data notes below. |
| - **Tracking:** [https://wandb.ai/imaging-ai/more-art-than-science](https://wandb.ai/imaging-ai/more-art-than-science) |
|
|
| ## Files |
| - `nitro_cc_ema_merged.safetensors` — full E-MMDiT transformer state dict (LoRA merged into base). Load this to serve. |
| - `adapter_config.json` + `adapter_model.safetensors` — the PEFT LoRA adapter (apply on top of `amd/Nitro-E` if you prefer not to use the merged weights). |
|
|
| ## Training data |
| — A private dataset of 32,061 WikiArt works (metadata + image URLs). ~29,000 images were downloadable and |
| used. The `consolidated-caption` field carries the emotional/style signal used as the prompt. |
|
|
| ## Recipe |
| - **LoRA** rank 16 (alpha=32) on attention projections |
| (`to_q,to_k,to_v,to_add_out,add_q_proj,add_k_proj,add_v_proj`); base frozen (full FT collapses the prior). |
| - Flow matching (logit-normal timestep sampling, SD3 loss weighting), AdamW lr 1e-05, cosine decay, bf16, SDPA. |
| - 2000 steps, batch 8 x grad-accum 2, CFG dropout 0.1. **EMA rate 0.9999**; best-val checkpoint kept |
| (deterministic validation — fixed noise + stratified timesteps). |
| - Single AMD R9700 (ROCm). Trainer: [`finetune_wikiart.py`](https://github.com/mascharkh/more-art-than-science). |
|
|
| ## Usage |
| Nitro-E uses AMD custom E-MMDiT pipeline (not diffusers-native). See the |
| [AMD-AGI/Nitro-E](https://github.com/AMD-AGI/Nitro-E) repo and this project`s |
| [`finetune_wikiart.py`](https://github.com/mascharkh/more-art-than-science) / `merge_and_sample.py`. |
| Example prompt: `"a tranquil river landscape, in the style of Impressionism, evoking calm"`. |
|
|
| ## Limitations |
| - Inherits Nitro-E 304M quality ceiling; can overfit WikiArt style cues. |
| - Requires gated `meta-llama/Llama-3.2-1B` access to run. |
| - **Data rights:** trained on WikiArt images but does not redistribute them; the dataset is |
| metadata-only. Many works are public domain but not all so use outputs accordingly. |
|
|
