# LoRA Configuration - Safetensors Only ## Directory Structure ``` data/lora/ ├── memo-scene-lora.safetensors # Main LoRA weights ├── readme.md # This file └── versions/ # Versioned LoRA files ├── v1.0/ └── v1.1/ ``` ## LoRA File Requirements ### Security Requirements - **ONLY .safetensors files** - No .bin, .ckpt, or other formats allowed - **Model signatures required** - All LoRA files must have proper signatures - **Version tracking** - Each version must be clearly identified ### Technical Requirements - **Format**: PyTorch safetensors - **Precision**: FP16 recommended for memory efficiency - **Compression**: Quantized versions for faster loading - **Metadata**: Include training information and compatibility notes ## Loading LoRA Weights ### Basic Loading ```python from models.image.sd_generator import get_generator generator = get_generator(lora_path="data/lora") ``` ### Version-Specific Loading ```python generator = get_generator(lora_path="data/lora/versions/v1.1") ``` ### Multiple LoRA Support ```python # Load multiple LoRA files lora_paths = [ "data/lora/memo-scene-lora.safetensors", "data/lora/style-lora.safetensors" ] for lora_path in lora_paths: generator.pipe.load_lora_weights( os.path.dirname(lora_path), weight_name=os.path.basename(lora_path) ) ``` ## LoRA Training Configuration ### Recommended Settings - **Base Model**: stabilityai/stable-diffusion-xl-base-1.0 - **LoRA Rank**: 16-64 (higher rank = more capacity) - **Alpha**: 32-128 (typically 2x the rank) - **Dropout**: 0.1-0.2 for regularization - **Precision**: FP16 for training, FP16 inference ### Training Script Usage ```bash python scripts/train_scene_lora.py \ --base_model "stabilityai/stable-diffusion-xl-base-1.0" \ --output_dir "data/lora/versions/v1.2" \ --rank 32 \ --alpha 64 \ --epochs 5 ``` ## Model Tier Configuration ### Free Tier - Base model only (no LoRA) - Lower inference steps (15-20) - Standard resolution (512x512) ### Pro Tier - Base + scene LoRA - Higher inference steps (25-30) - Higher resolution (768x768 or 1024x1024) - LCM acceleration ### Enterprise Tier - Base + multiple LoRAs - Highest quality settings - Custom resolution - Priority processing ## Security Notes 1. **Never load .bin files** - Use only safetensors 2. **Verify signatures** - Check LoRA file integrity 3. **Isolate environments** - Separate model loading contexts 4. **Audit logs** - Track all LoRA loading operations 5. **Version pinning** - Lock specific LoRA versions for production ## Performance Notes 1. **Memory optimization** - Use quantized LoRA when possible 2. **Preloading** - Load frequently used LoRA files at startup 3. **Caching** - Cache LoRA states for faster switching 4. **Cold start** - Minimize initial LoRA loading time 5. **Dynamic loading** - Load LoRA on-demand for different scenes