File size: 2,960 Bytes
a8fc815 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 |
# LoRA Configuration - Safetensors Only
## Directory Structure
```
data/lora/
├── memo-scene-lora.safetensors # Main LoRA weights
├── readme.md # This file
└── versions/ # Versioned LoRA files
├── v1.0/
└── v1.1/
```
## LoRA File Requirements
### Security Requirements
- **ONLY .safetensors files** - No .bin, .ckpt, or other formats allowed
- **Model signatures required** - All LoRA files must have proper signatures
- **Version tracking** - Each version must be clearly identified
### Technical Requirements
- **Format**: PyTorch safetensors
- **Precision**: FP16 recommended for memory efficiency
- **Compression**: Quantized versions for faster loading
- **Metadata**: Include training information and compatibility notes
## Loading LoRA Weights
### Basic Loading
```python
from models.image.sd_generator import get_generator
generator = get_generator(lora_path="data/lora")
```
### Version-Specific Loading
```python
generator = get_generator(lora_path="data/lora/versions/v1.1")
```
### Multiple LoRA Support
```python
# Load multiple LoRA files
lora_paths = [
"data/lora/memo-scene-lora.safetensors",
"data/lora/style-lora.safetensors"
]
for lora_path in lora_paths:
generator.pipe.load_lora_weights(
os.path.dirname(lora_path),
weight_name=os.path.basename(lora_path)
)
```
## LoRA Training Configuration
### Recommended Settings
- **Base Model**: stabilityai/stable-diffusion-xl-base-1.0
- **LoRA Rank**: 16-64 (higher rank = more capacity)
- **Alpha**: 32-128 (typically 2x the rank)
- **Dropout**: 0.1-0.2 for regularization
- **Precision**: FP16 for training, FP16 inference
### Training Script Usage
```bash
python scripts/train_scene_lora.py \
--base_model "stabilityai/stable-diffusion-xl-base-1.0" \
--output_dir "data/lora/versions/v1.2" \
--rank 32 \
--alpha 64 \
--epochs 5
```
## Model Tier Configuration
### Free Tier
- Base model only (no LoRA)
- Lower inference steps (15-20)
- Standard resolution (512x512)
### Pro Tier
- Base + scene LoRA
- Higher inference steps (25-30)
- Higher resolution (768x768 or 1024x1024)
- LCM acceleration
### Enterprise Tier
- Base + multiple LoRAs
- Highest quality settings
- Custom resolution
- Priority processing
## Security Notes
1. **Never load .bin files** - Use only safetensors
2. **Verify signatures** - Check LoRA file integrity
3. **Isolate environments** - Separate model loading contexts
4. **Audit logs** - Track all LoRA loading operations
5. **Version pinning** - Lock specific LoRA versions for production
## Performance Notes
1. **Memory optimization** - Use quantized LoRA when possible
2. **Preloading** - Load frequently used LoRA files at startup
3. **Caching** - Cache LoRA states for faster switching
4. **Cold start** - Minimize initial LoRA loading time
5. **Dynamic loading** - Load LoRA on-demand for different scenes |