# 🚀 AudioForge Production Launch Guide

**Complete step-by-step guide for launching AudioForge to production**

---

## 📋 Pre-Launch Requirements

### System Requirements
- **Python**: 3.11+
- **Node.js**: 18+ 
- **pnpm**: 8+
- **Docker**: 24+ (optional, for containerized deployment)
- **PostgreSQL**: 16+
- **Redis**: 7+

### Hardware Recommendations
- **CPU**: 4+ cores (8+ recommended for music generation)
- **RAM**: 8GB minimum (16GB+ recommended)
- **Storage**: 50GB+ (models require ~10GB, audio storage scales with usage)
- **GPU**: Optional but highly recommended (CUDA-compatible for faster generation)

---

## 🔍 Step 1: Automated Verification

Run the comprehensive verification script to check all systems:

```bash
# From project root
python scripts/launch_verification.py --verbose

# Auto-fix common issues
python scripts/launch_verification.py --fix

# Export results to JSON
python scripts/launch_verification.py --json launch-report.json
```

**Expected Output**: 100% success rate on all checks

---

## 🛠️ Step 2: Environment Setup

### Backend Configuration

1. **Create `.env` file**:
```bash
cd backend
cp .env.example .env
```

2. **Configure environment variables**:
```bash
# Database
DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/audioforge

# Redis
REDIS_URL=redis://localhost:6379/0

# AI Models
MUSICGEN_DEVICE=cuda  # or 'cpu'
BARK_DEVICE=cuda      # or 'cpu'
DEMUCS_DEVICE=cuda    # or 'cpu'

# Application
DEBUG=false
ENVIRONMENT=production
SECRET_KEY=<generate-secure-key>
ALLOWED_ORIGINS=https://yourdomain.com

# Optional: Monitoring
SENTRY_DSN=<your-sentry-dsn>
```

3. **Generate secure secret key**:
```bash
python -c "import secrets; print(secrets.token_urlsafe(32))"
```

### Frontend Configuration

1. **Create `.env.local`**:
```bash
cd frontend
echo "NEXT_PUBLIC_API_URL=https://api.yourdomain.com" > .env.local
```

2. **For production build**:
```bash
# .env.production
NEXT_PUBLIC_API_URL=https://api.yourdomain.com
NEXT_PUBLIC_SENTRY_DSN=<your-sentry-dsn>
```

---

## 📦 Step 3: Install Dependencies

### Backend
```bash
cd backend

# Using uv (recommended)
uv pip install -e ".[dev]"

# Or using pip
pip install -e ".[dev]"

# Verify installation
python scripts/verify_setup.py
```

### Frontend
```bash
cd frontend

# Install dependencies
pnpm install

# Verify no errors
pnpm run type-check
pnpm run lint
```

---

## 🗄️ Step 4: Database Setup

### Initialize Database

```bash
cd backend

# Run migrations
python scripts/init_db.py

# Verify connection
python -c "from app.db.database import engine; print('✅ Database connected')"
```

### Create Required Tables

The `init_db.py` script automatically creates:
- `generations` table
- Indexes on frequently queried fields
- Initial schema

### Backup Strategy

```bash
# Create backup
pg_dump audioforge > backup_$(date +%Y%m%d).sql

# Restore backup
psql audioforge < backup_20260116.sql
```

---

## 🎵 Step 5: Download AI Models

### Automatic Download (Recommended)

Models will download automatically on first use. To pre-download:

```bash
cd backend
python -c "
from app.services.music_generation import MusicGenerationService
service = MusicGenerationService()
print('✅ Models downloaded')
"
```

### Manual Download

If automatic download fails:

1. **MusicGen** (~2GB):
```bash
python -c "
from transformers import AutoProcessor, MusicgenForConditionalGeneration
model = MusicgenForConditionalGeneration.from_pretrained('facebook/musicgen-small')
processor = AutoProcessor.from_pretrained('facebook/musicgen-small')
"
```

2. **Bark** (for vocals, ~3GB):
```bash
python -c "
from transformers import AutoProcessor, BarkModel
model = BarkModel.from_pretrained('suno/bark-small')
processor = AutoProcessor.from_pretrained('suno/bark-small')
"
```

3. **Demucs** (for separation, ~300MB):
```bash
python -c "
import torch
torch.hub.load('facebookresearch/demucs', 'demucs')
"
```

---

## 🧪 Step 6: Run Tests

### Backend Tests
```bash
cd backend
pytest tests/ -v --cov=app --cov-report=html

# Run specific test
pytest tests/test_prompt_understanding.py -v
```

### Frontend Tests
```bash
cd frontend

# Unit tests
pnpm test

# Integration tests
pnpm test src/test/integration.test.tsx

# Coverage report
pnpm test:coverage

# Watch mode during development
pnpm test:watch
```

### Integration Tests
```bash
# Ensure both services are running
# Terminal 1: Backend
cd backend && uvicorn app.main:app --reload

# Terminal 2: Frontend  
cd frontend && pnpm dev

# Terminal 3: Run E2E tests
python scripts/launch_verification.py --section integration
```

---

## 🚀 Step 7: Build for Production

### Backend

```bash
cd backend

# No build step needed for Python
# Ensure all dependencies are installed
pip freeze > requirements-lock.txt
```

### Frontend

```bash
cd frontend

# Production build
pnpm run build

# Test production build locally
pnpm run start

# Verify at http://localhost:3000
```

### Build Verification

```bash
# Check build output
ls -lh frontend/.next/

# Expected: optimized bundles, static assets
# Build should complete in < 2 minutes
```

---

## 🐳 Step 8: Docker Deployment (Recommended)

### Build Images

```bash
# Build all services
docker-compose build

# Build specific service
docker-compose build backend
docker-compose build frontend
```

### Start Services

```bash
# Start all services
docker-compose up -d

# Check status
docker-compose ps

# View logs
docker-compose logs -f

# Stop services
docker-compose down
```

### Health Checks

```bash
# Backend health
curl http://localhost:8000/health

# Frontend health
curl http://localhost:3000

# Database health
docker-compose exec postgres pg_isready

# Redis health
docker-compose exec redis redis-cli ping
```

---

## 🔧 Step 9: Manual Deployment

### Backend Deployment

1. **Using systemd** (Linux):

Create `/etc/systemd/system/audioforge-backend.service`:
```ini
[Unit]
Description=AudioForge Backend API
After=network.target postgresql.service redis.service

[Service]
Type=simple
User=audioforge
WorkingDirectory=/opt/audioforge/backend
Environment="PATH=/opt/audioforge/venv/bin"
ExecStart=/opt/audioforge/venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 8000 --workers 4
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

Enable and start:
```bash
sudo systemctl enable audioforge-backend
sudo systemctl start audioforge-backend
sudo systemctl status audioforge-backend
```

2. **Using Gunicorn** (alternative):
```bash
gunicorn app.main:app \
  --workers 4 \
  --worker-class uvicorn.workers.UvicornWorker \
  --bind 0.0.0.0:8000 \
  --access-logfile - \
  --error-logfile -
```

### Frontend Deployment

1. **Using PM2**:
```bash
cd frontend

# Install PM2
npm install -g pm2

# Start application
pm2 start pnpm --name "audioforge-frontend" -- start

# Save configuration
pm2 save

# Setup startup script
pm2 startup
```

2. **Using systemd**:

Create `/etc/systemd/system/audioforge-frontend.service`:
```ini
[Unit]
Description=AudioForge Frontend
After=network.target

[Service]
Type=simple
User=audioforge
WorkingDirectory=/opt/audioforge/frontend
Environment="NODE_ENV=production"
ExecStart=/usr/bin/pnpm start
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```

---

## 🌐 Step 10: Reverse Proxy Setup

### Nginx Configuration

Create `/etc/nginx/sites-available/audioforge`:

```nginx
# Backend API
upstream backend {
    server localhost:8000;
}

# Frontend
upstream frontend {
    server localhost:3000;
}

# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name yourdomain.com api.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

# Frontend HTTPS
server {
    listen 443 ssl http2;
    server_name yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header X-XSS-Protection "1; mode=block" always;

    location / {
        proxy_pass http://frontend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

# Backend API HTTPS
server {
    listen 443 ssl http2;
    server_name api.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/api.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options "nosniff" always;

    # CORS headers (if needed)
    add_header Access-Control-Allow-Origin "https://yourdomain.com" always;
    add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS" always;
    add_header Access-Control-Allow-Headers "Authorization, Content-Type" always;

    location / {
        proxy_pass http://backend;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        
        # Increase timeouts for long-running generation requests
        proxy_read_timeout 300s;
        proxy_connect_timeout 75s;
    }

    # API documentation
    location /docs {
        proxy_pass http://backend/docs;
        proxy_set_header Host $host;
    }
}
```

Enable and reload:
```bash
sudo ln -s /etc/nginx/sites-available/audioforge /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
```

### SSL Certificates (Let's Encrypt)

```bash
# Install certbot
sudo apt install certbot python3-certbot-nginx

# Obtain certificates
sudo certbot --nginx -d yourdomain.com -d api.yourdomain.com

# Auto-renewal is configured automatically
# Test renewal
sudo certbot renew --dry-run
```

---

## 📊 Step 11: Monitoring Setup

### Application Monitoring

1. **Sentry** (Error Tracking):
```bash
# Backend
pip install sentry-sdk[fastapi]

# Add to app/main.py
import sentry_sdk
sentry_sdk.init(dsn="YOUR_SENTRY_DSN")

# Frontend
pnpm add @sentry/nextjs
# Configure in next.config.js
```

2. **Prometheus** (Metrics):
```bash
# Install prometheus client
pip install prometheus-client

# Expose metrics endpoint
# Already configured in app/core/metrics.py
```

3. **Grafana** (Dashboards):
```bash
docker run -d \
  -p 3001:3000 \
  --name=grafana \
  -e "GF_SECURITY_ADMIN_PASSWORD=admin" \
  grafana/grafana
```

### System Monitoring

```bash
# Install monitoring tools
sudo apt install htop iotop nethogs

# Monitor processes
htop

# Monitor disk I/O
iotop

# Monitor network
nethogs
```

### Log Aggregation

```bash
# Using journalctl
sudo journalctl -u audioforge-backend -f
sudo journalctl -u audioforge-frontend -f

# Centralized logging (optional)
# Configure with ELK stack or similar
```

---

## ✅ Step 12: Final Verification

### Automated Checks

```bash
# Run full verification
python scripts/launch_verification.py --verbose

# Expected: 100% pass rate
```

### Manual Verification Checklist

- [ ] Backend health endpoint responds: `curl https://api.yourdomain.com/health`
- [ ] Frontend loads: Visit `https://yourdomain.com`
- [ ] API documentation accessible: `https://api.yourdomain.com/docs`
- [ ] Create test generation works
- [ ] Audio playback works
- [ ] Status updates in real-time
- [ ] Error messages are user-friendly
- [ ] Mobile responsive design works
- [ ] All animations smooth (60fps)
- [ ] SSL certificates valid
- [ ] Monitoring dashboards active

### Performance Benchmarks

```bash
# Backend API response time
ab -n 1000 -c 10 https://api.yourdomain.com/health

# Frontend load time
lighthouse https://yourdomain.com --view

# Expected metrics:
# - Backend: < 200ms average
# - Frontend FCP: < 1.5s
# - Frontend TTI: < 3s
# - Lighthouse score: > 90
```

---

## 🚨 Step 13: Launch Day Procedures

### T-1 Hour

```bash
# 1. Final backup
pg_dump audioforge > pre-launch-backup.sql

# 2. Final verification
python scripts/launch_verification.py

# 3. Monitor system resources
htop

# 4. Clear logs
sudo journalctl --vacuum-time=1d

# 5. Notify team
echo "Launch in 1 hour" | mail -s "AudioForge Launch" team@example.com
```

### Launch (T=0)

```bash
# 1. Start services (if not already running)
docker-compose up -d

# 2. Verify all services healthy
docker-compose ps

# 3. Test end-to-end flow
curl -X POST https://api.yourdomain.com/api/v1/generations \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Launch test", "duration": 10}'

# 4. Monitor logs
docker-compose logs -f
```

### T+1 Hour

```bash
# 1. Check error rates
curl https://api.yourdomain.com/metrics | grep error_total

# 2. Check generation success rate
# View in monitoring dashboard

# 3. Review user feedback
# Check support channels

# 4. Monitor system resources
htop
```

---

## 🔧 Troubleshooting

### Backend Won't Start

```bash
# Check logs
docker-compose logs backend

# Common issues:
# 1. Database connection
docker-compose exec postgres pg_isready

# 2. Missing dependencies
cd backend && pip install -e ".[dev]"

# 3. Port already in use
sudo lsof -i :8000
```

### Frontend Won't Build

```bash
# Check Node version
node --version  # Should be 18+

# Clear cache
rm -rf frontend/.next frontend/node_modules
cd frontend && pnpm install

# Check for TypeScript errors
pnpm run type-check
```

### Generation Fails

```bash
# Check model files
ls -lh ~/.cache/huggingface/

# Check GPU availability
python -c "import torch; print(torch.cuda.is_available())"

# Check disk space
df -h

# Check memory
free -h
```

### High CPU/Memory Usage

```bash
# Identify process
top -o %CPU

# Restart service
docker-compose restart backend

# Scale workers
# Edit docker-compose.yml: --workers 2
```

---

## 📈 Post-Launch Monitoring

### Daily Checks

- [ ] Error rate < 5%
- [ ] Average response time < 200ms
- [ ] Generation success rate > 95%
- [ ] Disk space > 20% free
- [ ] Database connections healthy
- [ ] SSL certificates valid (> 30 days)

### Weekly Tasks

- [ ] Review user feedback
- [ ] Analyze performance metrics
- [ ] Update dependencies
- [ ] Database backup verification
- [ ] Security audit

### Monthly Tasks

- [ ] Performance optimization review
- [ ] Cost analysis
- [ ] Feature roadmap update
- [ ] Team retrospective

---

## 🎉 Success Metrics

### Week 1 Goals
- 100+ generations created
- < 5% error rate
- Average processing time < 60s
- 90%+ user satisfaction

### Month 1 Goals
- 1,000+ total generations
- 100+ active users
- Feature requests collected
- Roadmap for v2 defined

---

## 📞 Support

### Emergency Contacts
- **Backend Issues**: Check `/var/log/audioforge/backend.log`
- **Frontend Issues**: Check browser console + Next.js logs
- **Database Issues**: Check PostgreSQL logs
- **Infrastructure**: Contact DevOps team

### Useful Commands

```bash
# Restart everything
docker-compose restart

# View all logs
docker-compose logs -f --tail=100

# Check service status
systemctl status audioforge-*

# Database backup
pg_dump audioforge > backup.sql

# Restore from backup
psql audioforge < backup.sql
```

---

**🐼⚡ You're ready to launch! The panda believes in you.** 🎵

*Remember: Launch is just the beginning. Listen to users, iterate fast, and keep the creative energy flowing.*