README.md

---
title: LoRA Ease
emoji: 🧞
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
startup_duration_timeout: 1h
pinned: true
license: mit
fullWidth: true
hf_oauth: true
hf_oauth_scopes:
- read-repos
- write-repos
- manage-repos
models:
- microsoft/Mage-VL
short_description: Train LoRAs with Ease
---

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

# LoRA Ease: SDXL training on Hugging Face

This fork trains LoRA adapters for `stabilityai/stable-diffusion-xl-base-1.0`.
The maintained Hugging Face Diffusers v0.39.0 advanced SDXL trainer is downloaded
from a pinned commit the first time training starts. Captions are generated in this
Space by the embedded Mage-VL model; the face-prior dataset is loaded only when needed.

## Deploy as a Hugging Face Space

Create or duplicate a Gradio Space, enable **ZeroGPU** hardware, and push these
files to it. `microsoft/Mage-VL` is loaded directly into this Space and the
captioning function is decorated with `spaces.GPU`; images never need to be sent
to a second Space or account. The first startup downloads the model and can take
several minutes.

When a user starts training, the app directly creates a private dataset repository
and a temporary private L40S Docker Space under that user's account. This keeps
AutoTrain's conflicting dependency stack out of the captioning/UI container while
preserving the same trainer runtime and finished-model upload behavior.

The user must:

1. Have a Hugging Face account with a payment method configured.
2. Accept the SDXL base-model license on their Hugging Face account.
3. Provide a Hugging Face **write** token when prompted. It is passed to the
   temporary trainer and is not written into this repository.

Training currently targets the `l40sx1` hardware flavor (48 GB VRAM). The UI's
cost display is an estimate; Hugging Face billing and hardware pricing are the
source of truth.

## Optional local development

Local mode remains useful for testing the UI or training on your own NVIDIA GPU.
Use Python 3.10 or 3.11, create a virtual environment, install a CUDA-enabled
PyTorch build appropriate for your driver, and then install the project:

```powershell
py -3.11 -m venv .venv
.venv\Scripts\Activate.ps1
# Install CUDA PyTorch first: https://pytorch.org/get-started/locally/
pip install -r requirements_local.txt
python app.py
```

SDXL training is memory-intensive. The UI defaults to batch size 1, gradient
accumulation 4, gradient checkpointing, latent caching, fp16, and 1024px. For local
out-of-memory errors, keep batch size 1 and try 768px; 1024px is preferred when the
GPU permits.

The base model is gated. Before the first run, accept its license on Hugging Face
and authenticate locally with `hf auth login`.

## Training a Daggerfall-like face style

If the images show many different people sharing the same visual language, this is
a **style LoRA**, not a face/identity LoRA:

1. Prepare roughly 20-60 varied, clean portraits. Remove menus, text, cursors,
   duplicates, and unrelated UI. Do not mix modern fan art with original game art
   unless you deliberately want both aesthetics.
2. Crop loosely around each head and shoulders, keeping several face angles,
   ancestries, genders, expressions, backgrounds, and lighting conditions. Do not
   enlarge tiny source images with heavy smoothing; crisp nearest-neighbor or mild
   restoration usually preserves the pixel-art language better.
3. In the UI choose **style** and keep the concept sentence
   `in the style of TOK`. `TOK` is the trainer's required token abstraction; do
   not replace it with `daggerfall` or another common word.
4. Caption what should remain controllable, for example:
   `portrait of an elderly elf man, stern expression, dark background, in the style of TOK`.
   Keep the trigger in every caption, but vary subject traits accurately.
5. Start with rank 32 and 800-1500 steps. Save/test intermediate versions if you
   raise the step count; overtraining shows up as every output copying the training
   backgrounds, poses, or facial proportions.

After uploading the images, click **Caption all images with Mage-VL**. The portrait
instruction asks for controllable character traits while deliberately excluding
the shared art style; the concept sentence is appended automatically. Review the
generated text boxes and correct names, species, clothing, or expressions that the
captioner guessed incorrectly. **Export captions** downloads the reviewed entries
as ImageFolder-compatible `metadata.jsonl` for backup or offline editing.

Use **character** only if all images are the same fictional character. Use **face**
only for one consistent real or fictional identity; that preset enables prior
preservation and is not appropriate for learning a whole game's portrait style.

The output folder contains `pytorch_lora_weights.safetensors`, which can be loaded
as an SDXL LoRA by Diffusers, ComfyUI, or compatible Stable Diffusion frontends.
Only train and distribute assets you have the rights or permission to use.