TRAINING_GUIDE.md

🎓 LoRA Training Guide - Architecture AI Enhancer

This guide explains how to train custom LoRA (Low-Rank Adaptation) models to specialize the AI enhancer for your specific architectural style.

📖 What is LoRA Training?

LoRA allows you to fine-tune the Stable Diffusion model with a small set of your own images (10-50 pairs) to learn your specific architectural style, lighting preferences, or rendering aesthetic without retraining the entire model.

Benefits:

• ✅ Fast Training: Only a few hundred training steps needed
• ✅ Small Models: LoRA weights are ~10-50 MB vs. full model (4+ GB)
• ✅ Style Consistency: Learn your specific rendering style
• ✅ Efficient: Works on consumer GPUs or even CPU (slower)

---

🚀 Quick Start

Step 1: Prepare Your Training Data

You need image pairs:

• Input Image: Your base architectural render (before enhancement)
• Target Image: The desired enhanced result (your ideal output)

Requirements:

• Minimum: 10 pairs (recommended: 20-50 pairs)
• Format: PNG or JPG
• Resolution: 512x512 to 1024x1024 recommended
• Naming convention: {pair_id}_input.png and {pair_id}_target.png

Example:

training_data/
  inputs/
    building_001_input.png
    building_002_input.png
    office_003_input.png
  targets/
    building_001_target.png
    building_002_target.png
    office_003_target.png

Step 2: Start the Backend Server (Local Training)

The current HF Spaces deployment does not include training capabilities due to resource constraints. Training must be done locally.

cd architecture-ai-enhancer/backend
uvicorn main:app --host 0.0.0.0 --port 8000

Access the API documentation at: http://localhost:8000/docs

Step 3: Upload Training Pairs

Use the /training/upload_pair endpoint:

import requests

# Upload a training pair
files = {
    'input_image': open('building_001_input.png', 'rb'),
    'target_image': open('building_001_target.png', 'rb')
}
response = requests.post('http://localhost:8000/training/upload_pair', files=files)
print(response.json())

Or use the interactive docs at /docs to upload via the UI.

Step 4: Start Training

Once you have uploaded all pairs, start training:

import requests

training_config = {
    "train_steps": 1000,      # Number of training iterations
    "learning_rate": 1e-4,    # Learning rate (lower = more stable)
    "lora_rank": 8,           # LoRA rank (higher = more capacity)
    "batch_size": 1           # Batch size (increase if you have enough VRAM)
}

response = requests.post(
    'http://localhost:8000/training/start',
    json=training_config
)
print(response.json())

Training Parameters Guide:

• train_steps: 500-2000 (more steps = better learning, but risk overfitting)
• learning_rate: 1e-4 to 1e-5 (lower for subtle changes, higher for dramatic)
• lora_rank: 4-16 (higher = more model capacity, but slower)
• batch_size: 1-4 (depends on your GPU memory)

Step 5: Monitor Training Progress

Check training status:

response = requests.get('http://localhost:8000/training/status')
print(response.json())

Training typically takes:

• GPU (RTX 3080+): 10-30 minutes for 1000 steps
• CPU: 2-4 hours for 1000 steps

Step 6: Use Your Trained LoRA

Once training completes, the model is automatically saved to models/lora/office_style.safetensors.

The enhancement pipeline will automatically detect and use the trained LoRA model for all future enhancements.

---

🎨 Advanced Tips

Creating Quality Training Data

1. Consistency is Key

   • Use similar lighting conditions across pairs
   • Maintain consistent camera angles
   • Keep architectural styles related

2. Variety Matters

   • Include different views (exterior, interior, details)
   • Mix daytime and nighttime scenes
   • Vary weather/lighting conditions

3. Quality Over Quantity

   • 20 high-quality pairs > 100 mediocre pairs
   • Ensure target images represent your desired style accurately
   • Avoid blurry or poorly composed targets

Optimal Training Parameters by Use Case

Learning Subtle Enhancements:

{
  "train_steps": 500,
  "learning_rate": 5e-5,
  "lora_rank": 4
}

Learning Dramatic Style Changes:

{
  "train_steps": 1500,
  "learning_rate": 1e-4,
  "lora_rank": 12
}

Balanced Training (Recommended Start):

{
  "train_steps": 1000,
  "learning_rate": 1e-4,
  "lora_rank": 8
}

Troubleshooting

Problem: Training loss not decreasing

• Solution: Increase learning rate or train for more steps
• Check: Ensure training pairs are properly aligned

Problem: Model overfitting (memorizing training images)

• Solution: Reduce train_steps or increase dataset size
• Try: Lower lora_rank

Problem: Out of memory errors

• Solution: Reduce batch_size to 1
• Try: Use gradient checkpointing (enabled by default)

Problem: Results don't match training targets

• Solution: Increase train_steps
• Check: Ensure sufficient variety in training data

---

🔄 Deploying Your Custom LoRA

Option 1: Local Deployment

Your trained LoRA is automatically used by the local FastAPI backend.

Option 2: HF Spaces Deployment

To use your custom LoRA on HF Spaces:

1. Upload your LoRA file to Hugging Face Hub
2. Modify hf_deployment/backend/config.py:

   LORA_MODEL_NAME = "your_lora_model.safetensors"
   

3. Add code to download the LoRA from HF Hub on startup
4. Redeploy to HF Spaces

---

📊 Example Training Script

Complete Python script for training workflow:

import requests
import time
from pathlib import Path

BASE_URL = "http://localhost:8000"

def upload_training_pairs(input_dir, target_dir):
    """Upload all training pairs from directories"""
    input_files = sorted(Path(input_dir).glob("*_input.*"))
    
    for input_file in input_files:
        pair_id = input_file.stem.replace("_input", "")
        target_file = Path(target_dir) / f"{pair_id}_target{input_file.suffix}"
        
        if not target_file.exists():
            print(f"Warning: No target for {input_file.name}")
            continue
        
        files = {
            'input_image': open(input_file, 'rb'),
            'target_image': open(target_file, 'rb')
        }
        
        response = requests.post(f"{BASE_URL}/training/upload_pair", files=files)
        
        if response.status_code == 200:
            print(f"✓ Uploaded pair: {pair_id}")
        else:
            print(f"✗ Failed: {pair_id} - {response.text}")

def start_training(steps=1000, lr=1e-4, rank=8):
    """Start LoRA training"""
    config = {
        "train_steps": steps,
        "learning_rate": lr,
        "lora_rank": rank,
        "batch_size": 1
    }
    
    response = requests.post(f"{BASE_URL}/training/start", json=config)
    print(f"Training started: {response.json()}")

def monitor_training():
    """Monitor training progress"""
    while True:
        response = requests.get(f"{BASE_URL}/training/status")
        status = response.json()
        
        print(f"Status: {status['status']} - {status['message']}")
        
        if status['status'] in ['completed', 'failed']:
            break
        
        time.sleep(30)  # Check every 30 seconds

# Main workflow
if __name__ == "__main__":
    # 1. Upload training data
    print("Uploading training pairs...")
    upload_training_pairs("training_data/inputs", "training_data/targets")
    
    # 2. Start training
    print("\nStarting training...")
    start_training(steps=1000, lr=1e-4, rank=8)
    
    # 3. Monitor progress
    print("\nMonitoring training...")
    monitor_training()
    
    print("\n✓ Training complete! Your LoRA is ready to use.")

---

🤔 FAQ

Q: Can I train on HF Spaces?
A: No, the free CPU tier doesn't have enough resources. Training must be done locally or on a paid GPU space.

Q: How many images do I need?
A: Minimum 10 pairs, recommended 20-50 pairs for best results.

Q: Can I use photos instead of renders?
A: Yes! You can train input=render, target=photo to learn photorealistic enhancement.

Q: How long does training take?
A: On GPU: 10-30 min. On CPU: 2-4 hours for 1000 steps.

Q: Can I train multiple LoRAs?
A: Yes, but only one can be active at a time. Rename your LoRA files accordingly.

Q: Will this work with other Stable Diffusion models?
A: The code uses SD 1.5. For SDXL or SD 2.x, you'll need to modify the training engine.

---

📚 Additional Resources

• LoRA Paper - Original research
• Diffusers Documentation - Hugging Face Diffusers library
• PEFT Library - Parameter-Efficient Fine-Tuning

---

💡 Next Steps

1. ✅ Prepare your training dataset
2. ✅ Run the backend locally
3. ✅ Upload and train your LoRA
4. ✅ Test enhancements with your custom style
5. ✅ Iterate and refine

Happy Training! 🚀