Aguilar Elizondo commited on
Commit
f6c0ca4
Β·
1 Parent(s): ed8c1f8

Add comprehensive LoRA training guide and documentation

Browse files
Files changed (2) hide show
  1. README.md +20 -4
  2. TRAINING_GUIDE.md +326 -0
README.md CHANGED
@@ -22,14 +22,30 @@ Transform your architectural renders with AI-powered enhancement using Stable Di
22
  - 🎭 Photographic post-processing
23
  - βš™οΈ Configurable parameters
24
  - πŸš€ GPU-accelerated inference
 
25
 
26
  ## Usage
27
 
28
  1. Upload an architectural render
29
- 2. Adjust enhancement settings (optional)
30
- 3. Click "Enhance Image"
31
- 4. Wait 30-60 seconds for processing
32
- 5. Download your enhanced result
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
  ## Technical Details
35
 
 
22
  - 🎭 Photographic post-processing
23
  - βš™οΈ Configurable parameters
24
  - πŸš€ GPU-accelerated inference
25
+ - πŸŽ“ **Custom LoRA training** (local only)
26
 
27
  ## Usage
28
 
29
  1. Upload an architectural render
30
+ 2. Add custom prompt details (optional)
31
+ 3. Adjust enhancement settings (optional)
32
+ 4. Click "Enhance Image"
33
+ 5. Wait for processing (2-5 minutes on CPU, ~30 seconds on GPU)
34
+ 6. Download your enhanced result
35
+
36
+ ## πŸŽ“ Custom LoRA Training
37
+
38
+ Want to train the AI on your specific architectural style? Check out the complete training guide:
39
+
40
+ **πŸ“– [LoRA Training Guide](./TRAINING_GUIDE.md)**
41
+
42
+ Learn how to:
43
+ - Prepare training image pairs
44
+ - Train custom LoRA models locally
45
+ - Fine-tune for your specific rendering style
46
+ - Deploy your custom model
47
+
48
+ **Note**: Training requires running the backend locally due to computational requirements.
49
 
50
  ## Technical Details
51
 
TRAINING_GUIDE.md ADDED
@@ -0,0 +1,326 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # πŸŽ“ LoRA Training Guide - Architecture AI Enhancer
2
+
3
+ This guide explains how to train custom LoRA (Low-Rank Adaptation) models to specialize the AI enhancer for your specific architectural style.
4
+
5
+ ## πŸ“– What is LoRA Training?
6
+
7
+ LoRA allows you to fine-tune the Stable Diffusion model with a small set of your own images (10-50 pairs) to learn your specific architectural style, lighting preferences, or rendering aesthetic without retraining the entire model.
8
+
9
+ ### Benefits:
10
+ - βœ… **Fast Training**: Only a few hundred training steps needed
11
+ - βœ… **Small Models**: LoRA weights are ~10-50 MB vs. full model (4+ GB)
12
+ - βœ… **Style Consistency**: Learn your specific rendering style
13
+ - βœ… **Efficient**: Works on consumer GPUs or even CPU (slower)
14
+
15
+ ---
16
+
17
+ ## πŸš€ Quick Start
18
+
19
+ ### Step 1: Prepare Your Training Data
20
+
21
+ You need **image pairs**:
22
+ - **Input Image**: Your base architectural render (before enhancement)
23
+ - **Target Image**: The desired enhanced result (your ideal output)
24
+
25
+ **Requirements:**
26
+ - Minimum: 10 pairs (recommended: 20-50 pairs)
27
+ - Format: PNG or JPG
28
+ - Resolution: 512x512 to 1024x1024 recommended
29
+ - Naming convention: `{pair_id}_input.png` and `{pair_id}_target.png`
30
+
31
+ **Example:**
32
+ ```
33
+ training_data/
34
+ inputs/
35
+ building_001_input.png
36
+ building_002_input.png
37
+ office_003_input.png
38
+ targets/
39
+ building_001_target.png
40
+ building_002_target.png
41
+ office_003_target.png
42
+ ```
43
+
44
+ ### Step 2: Start the Backend Server (Local Training)
45
+
46
+ The current HF Spaces deployment **does not include training capabilities** due to resource constraints. Training must be done locally.
47
+
48
+ ```bash
49
+ cd architecture-ai-enhancer/backend
50
+ uvicorn main:app --host 0.0.0.0 --port 8000
51
+ ```
52
+
53
+ Access the API documentation at: `http://localhost:8000/docs`
54
+
55
+ ### Step 3: Upload Training Pairs
56
+
57
+ Use the `/training/upload_pair` endpoint:
58
+
59
+ ```python
60
+ import requests
61
+
62
+ # Upload a training pair
63
+ files = {
64
+ 'input_image': open('building_001_input.png', 'rb'),
65
+ 'target_image': open('building_001_target.png', 'rb')
66
+ }
67
+ response = requests.post('http://localhost:8000/training/upload_pair', files=files)
68
+ print(response.json())
69
+ ```
70
+
71
+ Or use the interactive docs at `/docs` to upload via the UI.
72
+
73
+ ### Step 4: Start Training
74
+
75
+ Once you have uploaded all pairs, start training:
76
+
77
+ ```python
78
+ import requests
79
+
80
+ training_config = {
81
+ "train_steps": 1000, # Number of training iterations
82
+ "learning_rate": 1e-4, # Learning rate (lower = more stable)
83
+ "lora_rank": 8, # LoRA rank (higher = more capacity)
84
+ "batch_size": 1 # Batch size (increase if you have enough VRAM)
85
+ }
86
+
87
+ response = requests.post(
88
+ 'http://localhost:8000/training/start',
89
+ json=training_config
90
+ )
91
+ print(response.json())
92
+ ```
93
+
94
+ **Training Parameters Guide:**
95
+ - `train_steps`: 500-2000 (more steps = better learning, but risk overfitting)
96
+ - `learning_rate`: 1e-4 to 1e-5 (lower for subtle changes, higher for dramatic)
97
+ - `lora_rank`: 4-16 (higher = more model capacity, but slower)
98
+ - `batch_size`: 1-4 (depends on your GPU memory)
99
+
100
+ ### Step 5: Monitor Training Progress
101
+
102
+ Check training status:
103
+
104
+ ```python
105
+ response = requests.get('http://localhost:8000/training/status')
106
+ print(response.json())
107
+ ```
108
+
109
+ Training typically takes:
110
+ - **GPU (RTX 3080+)**: 10-30 minutes for 1000 steps
111
+ - **CPU**: 2-4 hours for 1000 steps
112
+
113
+ ### Step 6: Use Your Trained LoRA
114
+
115
+ Once training completes, the model is automatically saved to `models/lora/office_style.safetensors`.
116
+
117
+ The enhancement pipeline will automatically detect and use the trained LoRA model for all future enhancements.
118
+
119
+ ---
120
+
121
+ ## 🎨 Advanced Tips
122
+
123
+ ### Creating Quality Training Data
124
+
125
+ 1. **Consistency is Key**
126
+ - Use similar lighting conditions across pairs
127
+ - Maintain consistent camera angles
128
+ - Keep architectural styles related
129
+
130
+ 2. **Variety Matters**
131
+ - Include different views (exterior, interior, details)
132
+ - Mix daytime and nighttime scenes
133
+ - Vary weather/lighting conditions
134
+
135
+ 3. **Quality Over Quantity**
136
+ - 20 high-quality pairs > 100 mediocre pairs
137
+ - Ensure target images represent your desired style accurately
138
+ - Avoid blurry or poorly composed targets
139
+
140
+ ### Optimal Training Parameters by Use Case
141
+
142
+ **Learning Subtle Enhancements:**
143
+ ```json
144
+ {
145
+ "train_steps": 500,
146
+ "learning_rate": 5e-5,
147
+ "lora_rank": 4
148
+ }
149
+ ```
150
+
151
+ **Learning Dramatic Style Changes:**
152
+ ```json
153
+ {
154
+ "train_steps": 1500,
155
+ "learning_rate": 1e-4,
156
+ "lora_rank": 12
157
+ }
158
+ ```
159
+
160
+ **Balanced Training (Recommended Start):**
161
+ ```json
162
+ {
163
+ "train_steps": 1000,
164
+ "learning_rate": 1e-4,
165
+ "lora_rank": 8
166
+ }
167
+ ```
168
+
169
+ ### Troubleshooting
170
+
171
+ **Problem: Training loss not decreasing**
172
+ - Solution: Increase learning rate or train for more steps
173
+ - Check: Ensure training pairs are properly aligned
174
+
175
+ **Problem: Model overfitting (memorizing training images)**
176
+ - Solution: Reduce train_steps or increase dataset size
177
+ - Try: Lower lora_rank
178
+
179
+ **Problem: Out of memory errors**
180
+ - Solution: Reduce batch_size to 1
181
+ - Try: Use gradient checkpointing (enabled by default)
182
+
183
+ **Problem: Results don't match training targets**
184
+ - Solution: Increase train_steps
185
+ - Check: Ensure sufficient variety in training data
186
+
187
+ ---
188
+
189
+ ## πŸ”„ Deploying Your Custom LoRA
190
+
191
+ ### Option 1: Local Deployment
192
+
193
+ Your trained LoRA is automatically used by the local FastAPI backend.
194
+
195
+ ### Option 2: HF Spaces Deployment
196
+
197
+ To use your custom LoRA on HF Spaces:
198
+
199
+ 1. Upload your LoRA file to Hugging Face Hub
200
+ 2. Modify `hf_deployment/backend/config.py`:
201
+ ```python
202
+ LORA_MODEL_NAME = "your_lora_model.safetensors"
203
+ ```
204
+ 3. Add code to download the LoRA from HF Hub on startup
205
+ 4. Redeploy to HF Spaces
206
+
207
+ ---
208
+
209
+ ## πŸ“Š Example Training Script
210
+
211
+ Complete Python script for training workflow:
212
+
213
+ ```python
214
+ import requests
215
+ import time
216
+ from pathlib import Path
217
+
218
+ BASE_URL = "http://localhost:8000"
219
+
220
+ def upload_training_pairs(input_dir, target_dir):
221
+ """Upload all training pairs from directories"""
222
+ input_files = sorted(Path(input_dir).glob("*_input.*"))
223
+
224
+ for input_file in input_files:
225
+ pair_id = input_file.stem.replace("_input", "")
226
+ target_file = Path(target_dir) / f"{pair_id}_target{input_file.suffix}"
227
+
228
+ if not target_file.exists():
229
+ print(f"Warning: No target for {input_file.name}")
230
+ continue
231
+
232
+ files = {
233
+ 'input_image': open(input_file, 'rb'),
234
+ 'target_image': open(target_file, 'rb')
235
+ }
236
+
237
+ response = requests.post(f"{BASE_URL}/training/upload_pair", files=files)
238
+
239
+ if response.status_code == 200:
240
+ print(f"βœ“ Uploaded pair: {pair_id}")
241
+ else:
242
+ print(f"βœ— Failed: {pair_id} - {response.text}")
243
+
244
+ def start_training(steps=1000, lr=1e-4, rank=8):
245
+ """Start LoRA training"""
246
+ config = {
247
+ "train_steps": steps,
248
+ "learning_rate": lr,
249
+ "lora_rank": rank,
250
+ "batch_size": 1
251
+ }
252
+
253
+ response = requests.post(f"{BASE_URL}/training/start", json=config)
254
+ print(f"Training started: {response.json()}")
255
+
256
+ def monitor_training():
257
+ """Monitor training progress"""
258
+ while True:
259
+ response = requests.get(f"{BASE_URL}/training/status")
260
+ status = response.json()
261
+
262
+ print(f"Status: {status['status']} - {status['message']}")
263
+
264
+ if status['status'] in ['completed', 'failed']:
265
+ break
266
+
267
+ time.sleep(30) # Check every 30 seconds
268
+
269
+ # Main workflow
270
+ if __name__ == "__main__":
271
+ # 1. Upload training data
272
+ print("Uploading training pairs...")
273
+ upload_training_pairs("training_data/inputs", "training_data/targets")
274
+
275
+ # 2. Start training
276
+ print("\nStarting training...")
277
+ start_training(steps=1000, lr=1e-4, rank=8)
278
+
279
+ # 3. Monitor progress
280
+ print("\nMonitoring training...")
281
+ monitor_training()
282
+
283
+ print("\nβœ“ Training complete! Your LoRA is ready to use.")
284
+ ```
285
+
286
+ ---
287
+
288
+ ## πŸ€” FAQ
289
+
290
+ **Q: Can I train on HF Spaces?**
291
+ A: No, the free CPU tier doesn't have enough resources. Training must be done locally or on a paid GPU space.
292
+
293
+ **Q: How many images do I need?**
294
+ A: Minimum 10 pairs, recommended 20-50 pairs for best results.
295
+
296
+ **Q: Can I use photos instead of renders?**
297
+ A: Yes! You can train input=render, target=photo to learn photorealistic enhancement.
298
+
299
+ **Q: How long does training take?**
300
+ A: On GPU: 10-30 min. On CPU: 2-4 hours for 1000 steps.
301
+
302
+ **Q: Can I train multiple LoRAs?**
303
+ A: Yes, but only one can be active at a time. Rename your LoRA files accordingly.
304
+
305
+ **Q: Will this work with other Stable Diffusion models?**
306
+ A: The code uses SD 1.5. For SDXL or SD 2.x, you'll need to modify the training engine.
307
+
308
+ ---
309
+
310
+ ## πŸ“š Additional Resources
311
+
312
+ - [LoRA Paper](https://arxiv.org/abs/2106.09685) - Original research
313
+ - [Diffusers Documentation](https://huggingface.co/docs/diffusers/) - Hugging Face Diffusers library
314
+ - [PEFT Library](https://github.com/huggingface/peft) - Parameter-Efficient Fine-Tuning
315
+
316
+ ---
317
+
318
+ ## πŸ’‘ Next Steps
319
+
320
+ 1. βœ… Prepare your training dataset
321
+ 2. βœ… Run the backend locally
322
+ 3. βœ… Upload and train your LoRA
323
+ 4. βœ… Test enhancements with your custom style
324
+ 5. βœ… Iterate and refine
325
+
326
+ **Happy Training! πŸš€**