docs_archive/SETUP_GUIDE.md

# MorphGuard Development Environment Setup Guide

This guide will help you set up your MorphGuard development environment and optionally migrate to new storage.

## Table of Contents

1. [Quick Start (Current Location)](#quick-start-current-location)
2. [Setting Up New SSDs](#setting-up-new-ssds)
3. [Migrating to New Storage](#migrating-to-new-storage)
4. [Troubleshooting](#troubleshooting)

---

## Quick Start (Current Location)

If you want to start developing immediately at the current location (`/home/juanquy/dev/MorphGuard`):

### Step 1: Make Scripts Executable

```bash
cd /home/juanquy/dev/MorphGuard
chmod +x setup_complete_dev_environment.sh
chmod +x migrate_to_new_storage.sh
```

### Step 2: Run Complete Setup

```bash
sudo bash setup_complete_dev_environment.sh
```

This will:
- Install all system dependencies
- Install Python 3.12
- Install PostgreSQL and TimescaleDB
- Set up virtual environment with all packages
- Download AI models
- Install Grafana (optional)
- Run validation tests

**Estimated time:** 30-60 minutes (depending on internet speed)

### Step 3: Start Development

After setup completes:

```bash
./start_dev.sh
```

Then visit: `http://localhost:5000`

---

## Setting Up New SSDs

When you install your 2x 500GB SSDs, follow these steps:

### Step 1: Physical Installation

1. Power off your computer
2. Install the 2x 500GB SSDs
3. Power on and boot into your OS

### Step 2: Format and Mount SSDs

#### Option A: Single Large Volume (RAID 0 - Fastest, 1TB total)

**⚠️ Warning:** RAID 0 provides no redundancy. If one drive fails, all data is lost.

```bash
# Install mdadm
sudo apt install mdadm

# Identify your new drives (e.g., /dev/sdb and /dev/sdc)
lsblk

# Create RAID 0 array
sudo mdadm --create --verbose /dev/md0 --level=0 --raid-devices=2 /dev/sdb /dev/sdc

# Create filesystem
sudo mkfs.ext4 -F /dev/md0

# Create mount point
sudo mkdir -p /mnt/morphguard-storage

# Mount the array
sudo mount /dev/md0 /mnt/morphguard-storage

# Add to fstab for automatic mounting
echo "/dev/md0 /mnt/morphguard-storage ext4 defaults 0 2" | sudo tee -a /etc/fstab

# Save RAID configuration
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf
sudo update-initramfs -u
```

#### Option B: Two Separate Volumes (Safe, 500GB each)

```bash
# Identify your new drives
lsblk

# Format first drive (adjust /dev/sdb to your actual device)
sudo mkfs.ext4 /dev/sdb
sudo mkdir -p /mnt/ssd1
sudo mount /dev/sdb /mnt/ssd1

# Format second drive
sudo mkfs.ext4 /dev/sdc
sudo mkdir -p /mnt/ssd2
sudo mount /dev/sdc /mnt/ssd2

# Get UUIDs for fstab
sudo blkid /dev/sdb
sudo blkid /dev/sdc

# Add to fstab (replace UUIDs with actual values from blkid)
echo "UUID=your-uuid-here /mnt/ssd1 ext4 defaults 0 2" | sudo tee -a /etc/fstab
echo "UUID=your-uuid-here /mnt/ssd2 ext4 defaults 0 2" | sudo tee -a /etc/fstab
```

#### Option C: Mirror (RAID 1 - Most Safe, 500GB usable)

**Recommended for production data**

```bash
# Install mdadm
sudo apt install mdadm

# Create RAID 1 array
sudo mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb /dev/sdc

# Format and mount (same as RAID 0)
sudo mkfs.ext4 -F /dev/md0
sudo mkdir -p /mnt/morphguard-storage
sudo mount /dev/md0 /mnt/morphguard-storage
echo "/dev/md0 /mnt/morphguard-storage ext4 defaults 0 2" | sudo tee -a /etc/fstab

# Save configuration
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf
sudo update-initramfs -u
```

### Step 3: Set Permissions

```bash
# For RAID setup
sudo chown -R $USER:$USER /mnt/morphguard-storage

# For separate drives
sudo chown -R $USER:$USER /mnt/ssd1
sudo chown -R $USER:$USER /mnt/ssd2
```

---

## Migrating to New Storage

### Before Migration

1. Ensure new storage is mounted and accessible
2. Verify sufficient space: `df -h /mnt/morphguard-storage`
3. Backup important data (optional but recommended)

### Run Migration Script

#### For RAID or Single Volume:

```bash
cd /home/juanquy/dev/MorphGuard
sudo bash migrate_to_new_storage.sh /mnt/morphguard-storage/MorphGuard
```

#### For Separate Drives:

```bash
# Use first drive for project
sudo bash migrate_to_new_storage.sh /mnt/ssd1/MorphGuard
```

### Migration Process

The script will:
1. ✅ Check disk space
2. ✅ Copy all files (using rsync)
3. ✅ Verify copy integrity
4. ✅ Update configuration paths
5. ✅ Rebuild virtual environment
6. ✅ Run validation tests
7. ✅ Create symbolic link (optional)

**Estimated time:** 15-30 minutes (for ~23GB project)

### After Migration

1. **Test the new location:**
   ```bash
   cd /mnt/morphguard-storage/MorphGuard  # or /mnt/ssd1/MorphGuard
   ./start_dev.sh
   ```

2. **Verify everything works:**
   - Upload test images
   - Check GPU usage: `watch -n 1 nvidia-smi`
   - Test detection and demorphing

3. **Remove old directory** (after confirming everything works):
   ```bash
   sudo rm -rf /home/juanquy/dev/MorphGuard.backup
   ```

---

## Storage Layout Recommendations

### Recommended: RAID 0 for Development

**Best performance for AI workloads:**

```
/mnt/morphguard-storage/
├── MorphGuard/              # Main project
│   ├── models/             # AI models (6GB)
│   ├── data/               # Training data (6GB)
│   ├── pSp/                # GAN models (6GB)
│   └── venv/               # Virtual environment (5GB)
└── workspace/              # Additional space
    ├── experiments/        # Training experiments
    ├── datasets/           # Additional datasets
    └── backups/            # Project backups
```

**Total project size:** ~23GB
**Available after migration:** ~977GB

### Alternative: Separate Drives

**For better organization:**

```
/mnt/ssd1/                  # Drive 1 (500GB)
├── MorphGuard/            # Main project (23GB)
└── workspace/             # Development space

/mnt/ssd2/                  # Drive 2 (500GB)
├── datasets/              # Large datasets
├── experiments/           # Training runs
└── backups/               # Backups
```

---

## Troubleshooting

### Issue: "PostgreSQL not installed"

**Solution:**
```bash
sudo apt update
sudo apt install postgresql-15
sudo systemctl start postgresql
```

### Issue: "CUDA not available"

**Check GPU:**
```bash
nvidia-smi
```

**Reinstall drivers if needed:**
```bash
sudo apt install nvidia-driver-535
sudo reboot
```

### Issue: "Python packages not found"

**Rebuild virtual environment:**
```bash
cd /your/morphguard/location
rm -rf venv
python3.12 -m venv venv
source venv/bin/activate
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
```

### Issue: "Port 5000 already in use"

**Find and kill the process:**
```bash
sudo lsof -i :5000
sudo kill <PID>
```

### Issue: "Permission denied" errors

**Fix permissions:**
```bash
sudo chown -R $USER:$USER /path/to/MorphGuard
chmod +x *.sh
```

### Issue: "Database connection failed"

**Check PostgreSQL status:**
```bash
sudo systemctl status postgresql
```

**Test connection:**
```bash
PGPASSWORD=morphguard psql -h localhost -U morphguard -d morphguard -c "SELECT version();"
```

### Issue: "Out of disk space during setup"

**Check available space:**
```bash
df -h
```

**Clean up if needed:**
```bash
# Remove old kernels
sudo apt autoremove

# Clean package cache
sudo apt clean

# Remove old logs
sudo journalctl --vacuum-time=7d
```

---

## Quick Reference Commands

### Start Development Server
```bash
cd /path/to/MorphGuard
./start_dev.sh
```

### Check GPU Status
```bash
nvidia-smi
watch -n 1 nvidia-smi  # Live monitoring
```

### Database Management
```bash
./manage_db.sh status   # Check database
./manage_db.sh backup   # Backup database
./manage_db.sh reset    # Reset database (careful!)
```

### Activate Virtual Environment
```bash
cd /path/to/MorphGuard
source venv/bin/activate
```

### Check Services
```bash
sudo systemctl status postgresql
sudo systemctl status grafana-server
sudo systemctl status morphguard  # If deployed as service
```

### View Logs
```bash
# Application logs
tail -f logs/*.log

# System logs
sudo journalctl -u morphguard -f

# PostgreSQL logs
sudo tail -f /var/log/postgresql/postgresql-15-main.log
```

---

## Environment Variables

### Development (.env.development)
- Used for local testing
- Debug mode enabled
- Localhost connections

### Production (.env.production)
- Used for deployment
- Debug mode disabled
- Network-accessible
- Hardened security settings

### Switch Environments
```bash
# Development
./start_morphguard.sh development

# Production
./start_morphguard.sh production
```

---

## Best Practices

1. **Always use virtual environment**
   - Prevents package conflicts
   - Isolates dependencies

2. **Regular backups**
   - Database: `./manage_db.sh backup`
   - Project: `rsync -ah --progress MorphGuard/ backup/`

3. **Monitor GPU temperature**
   - Check: `nvidia-smi --query-gpu=temperature.gpu --format=csv`
   - Keep under 80°C during training

4. **Use version control**
   - Initialize git: `git init`
   - Commit regularly: `git commit -am "message"`

5. **Test before deploying**
   - Run tests: `pytest`
   - Check code: `flake8 .`
   - Verify imports work

---

## Getting Help

### Check Documentation
- `HOW_TO_START_MORPHGUARD.md` - Startup guide
- `COMPREHENSIVE_DOCUMENTATION.md` - Full documentation
- `DEPLOYMENT_STATUS.md` - Deployment info

### Check Logs
```bash
# Application logs
cat logs/error.log

# System messages
dmesg | grep -i error

# PostgreSQL
sudo grep ERROR /var/log/postgresql/postgresql-15-main.log
```

### Verify System
```bash
# Python environment
python -c "import torch; print(torch.__version__); print(torch.cuda.is_available())"

# Database
psql -h localhost -U morphguard -d morphguard -c "\dt"

# Services
systemctl list-units | grep -E 'postgres|grafana|morphguard'
```

---

## Next Steps After Setup

1. ✅ Run setup script
2. ✅ Test with `./start_dev.sh`
3. ✅ Upload test image
4. ✅ Verify detection works
5. ✅ Check GPU utilization
6. 📝 Start developing new features!

---

## Support

For issues or questions:
1. Check logs: `logs/` directory
2. Review documentation: `*.md` files
3. Test components individually
4. Check system resources: `htop`, `nvidia-smi`

---

**Happy developing! 🚀**