README.md

metadata

title: LoRA Ease
emoji: 🧞
colorFrom: blue
colorTo: purple
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
startup_duration_timeout: 1h
pinned: true
license: mit
fullWidth: true
hf_oauth: true
hf_oauth_scopes:
  - read-repos
  - write-repos
  - manage-repos
models:
  - microsoft/Mage-VL
short_description: Train LoRAs with Ease

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

LoRA Ease: SDXL training on Hugging Face

This fork trains LoRA adapters for stabilityai/stable-diffusion-xl-base-1.0. The maintained Hugging Face Diffusers v0.39.0 advanced SDXL trainer is downloaded from a pinned commit the first time training starts. Captions are generated in this Space by the embedded Mage-VL model; the face-prior dataset is loaded only when needed.

Deploy as a Hugging Face Space

Create or duplicate a Gradio Space, enable ZeroGPU hardware, and push these files to it. microsoft/Mage-VL is loaded directly into this Space and the captioning function is decorated with spaces.GPU; images never need to be sent to a second Space or account. The first startup downloads the model and can take several minutes.

When a user starts training, the app directly creates a private dataset repository and a temporary private L40S Docker Space under that user's account. This keeps AutoTrain's conflicting dependency stack out of the captioning/UI container while preserving the same trainer runtime and finished-model upload behavior.

The user must:

1. Have a Hugging Face account with a payment method configured.
2. Accept the SDXL base-model license on their Hugging Face account.
3. Provide a Hugging Face write token when prompted. It is passed to the temporary trainer and is not written into this repository.

Training currently targets the l40sx1 hardware flavor (48 GB VRAM). The UI's cost display is an estimate; Hugging Face billing and hardware pricing are the source of truth.

Optional local development

Local mode remains useful for testing the UI or training on your own NVIDIA GPU. Use Python 3.10 or 3.11, create a virtual environment, install a CUDA-enabled PyTorch build appropriate for your driver, and then install the project:

py -3.11 -m venv .venv
.venv\Scripts\Activate.ps1
# Install CUDA PyTorch first: https://pytorch.org/get-started/locally/
pip install -r requirements_local.txt
python app.py

SDXL training is memory-intensive. The UI defaults to batch size 1, gradient accumulation 4, gradient checkpointing, latent caching, fp16, and 1024px. For local out-of-memory errors, keep batch size 1 and try 768px; 1024px is preferred when the GPU permits.

The base model is gated. Before the first run, accept its license on Hugging Face and authenticate locally with hf auth login.

Training a Daggerfall-like face style

If the images show many different people sharing the same visual language, this is a style LoRA, not a face/identity LoRA:

1. Prepare roughly 20-60 varied, clean portraits. Remove menus, text, cursors, duplicates, and unrelated UI. Do not mix modern fan art with original game art unless you deliberately want both aesthetics.
2. Crop loosely around each head and shoulders, keeping several face angles, ancestries, genders, expressions, backgrounds, and lighting conditions. Do not enlarge tiny source images with heavy smoothing; crisp nearest-neighbor or mild restoration usually preserves the pixel-art language better.
3. In the UI choose style and keep the concept sentence in the style of TOK. TOK is the trainer's required token abstraction; do not replace it with daggerfall or another common word.
4. Caption what should remain controllable, for example: portrait of an elderly elf man, stern expression, dark background, in the style of TOK. Keep the trigger in every caption, but vary subject traits accurately.
5. Start with rank 32 and 800-1500 steps. Save/test intermediate versions if you raise the step count; overtraining shows up as every output copying the training backgrounds, poses, or facial proportions.

After uploading the images, click Caption all images with Mage-VL. The portrait instruction asks for controllable character traits while deliberately excluding the shared art style; the concept sentence is appended automatically. Review the generated text boxes and correct names, species, clothing, or expressions that the captioner guessed incorrectly. Export captions downloads the reviewed entries as ImageFolder-compatible metadata.jsonl for backup or offline editing.

Use character only if all images are the same fictional character. Use face only for one consistent real or fictional identity; that preset enables prior preservation and is not appropriate for learning a whole game's portrait style.

The output folder contains pytorch_lora_weights.safetensors, which can be loaded as an SDXL LoRA by Diffusers, ComfyUI, or compatible Stable Diffusion frontends. Only train and distribute assets you have the rights or permission to use.