metadata
library_name: diffusers
base_model: sd2-community/stable-diffusion-2-1
tags:
- stable-diffusion
- lora
- text-to-image
- ukiyo-e
- art
- aetherart
license: creativeml-openrail-m
pipeline_tag: text-to-image
AetherArt — SD 2.1 Ukiyo-e LoRA
A rank-8 LoRA adapter that steers Stable Diffusion 2.1 toward Japanese ukiyo-e woodblock print style. Trained on 80 WikiArt Ukiyo-e images against the sd2-community/stable-diffusion-2-1 base model. Activate the style with the trigger token ukyowood anywhere in the prompt.
Usage
from diffusers import StableDiffusionPipeline
import torch
pipe = StableDiffusionPipeline.from_pretrained(
"sd2-community/stable-diffusion-2-1",
torch_dtype=torch.float16,
).to("cuda")
pipe.load_lora_weights("gauravgandhi2411/aetherart-ukiyo-sd21")
img = pipe(
"a ukyowood mountain landscape at sunset, traditional woodblock print",
negative_prompt="text, watermark, calligraphy, signature, words, letters",
num_inference_steps=30,
guidance_scale=7.5,
).images[0]
img.save("output.png")
Training details
| Parameter | Value |
|---|---|
| Base model | sd2-community/stable-diffusion-2-1 |
| LoRA rank | 8 |
| Training images | 80 (WikiArt Ukiyo-e) |
| Resolution | 512 × 512 |
| Steps | 1500 |
| Precision | fp16 mixed |
| Batch size | 1 (gradient accumulation = 4, effective batch = 4) |
| Learning rate | 1e-4 |
| Seed | 42 |
| Hardware | NVIDIA RTX 3070 Laptop GPU (8 GB VRAM) |
| Training time | ~2 h 8 min |
Checkpoint selection
Selected checkpoint-1000 over the other checkpoints trained during the run:
- checkpoint-500: underfit — style signal present but not saturated, more like a mild filter than a transformation.
- checkpoint-1000: selected — consistent warm amber palette and characteristic flatness of traditional woodblock prints across test prompts.
- checkpoint-1500: overfit — validation loss rose from 0.268 to 0.495. Outputs showed over-saturated colors and partial prompt-alignment breakdown.
Checkpoint selection was made by visual evaluation only — no quantitative held-out set was used.
Default negative prompt
The following negative prompt is applied automatically by the AetherArt application whenever this adapter is active:
text, watermark, calligraphy, signature, words, letters
Known limitations
- Calligraphy artifact (partially mitigated, not fixed): WikiArt Ukiyo-e source images contain metadata captions with artist signatures and script text embedded in the image margins. The adapter learned these as part of "ukiyo-e style." The default negative prompt suppresses most instances but does not eliminate the artifact entirely — the style signal and text signal are entangled in the adapter weights. The correct fix is retraining on a curated dataset with no text annotations, which would require approximately 5 hours of curation work.
- The adapter was trained and evaluated on 512 × 512 resolution. Results at other resolutions are untested.
- CLIP scoring does not capture the quality improvements from this adapter. See the CLIP-blindness finding linked below.
Links
- AetherArt repository: https://github.com/gaurav-gandhi-2411/AetherArt
- CLIP-blindness finding: see
reports/clip_blindness.md— nine Phase 6b experiments showing CLIP delta <1 SE while LPIPS ranged 0.40–0.73; underfitting paradox; why CLIP cannot guide LoRA training decisions. - Companion SDXL adapter (1024×1024):
gauravgandhi2411/aetherart-ukiyo-sdxl— same rank-8, same dataset, trained on GCP L4. Both runs independently select checkpoint-1000.