# Service Monitoring & Auto-Recovery Guide

This guide explains how to ensure the Visa Photo Maker services (Frontend & Backend) remain online and automatically recover from crashes or stops.

## 1. Watchdog Script

A simple bash script `scripts/watchdog.sh` has been created to check the health of your services every minute.

- **Location**: `scripts/watchdog.sh`
- **What it does**:
  1. Checks if **Port 13002** (Backend) is open.
  2. Checks if **Port 13001** (Frontend) is open.
  3. If a port is closed, it triggers `pm2 restart <app_name>`.
  4. Logs all incidents to `watchdog.log` in the project root.

## 2. Setup Instructions

### Step 1: Make Script Executable
Run this in your terminal:
```bash
chmod +x scripts/watchdog.sh
```

### Step 2: Test the Script
Run it manually once to make sure it works (it should be silent if everything is OK):
```bash
./scripts/watchdog.sh
```

### Step 3: Enable Auto-Run (Crontab)
To make it run every minute automatically:

1. Open your crontab editor:
   ```bash
   crontab -e
   ```

2. Add the following line at the bottom of the file:
   *(Update the path if you moved the project)*
   ```bash
   * * * * * /home/ubuntu/code/visa_photo_maker/scripts/watchdog.sh
   ```

3. Save and exit (usually `Ctrl+O`, `Enter`, `Ctrl+X` if using nano).

## 3. Viewing Logs

To see if the watchdog has restarted anything, check the log file:
```bash
tail -f watchdog.log
```

## 4. Other Protections Enabled

Besides this script, the following PM2 protections are already active:

1.  **Memory Limit**: Backend restarts if it exceeds **2GB** RAM.
2.  **Backoff Restart**: If the app crashes repeatedly, PM2 waits 100ms+ before restarting to prevent CPU spikes.
3.  **4 Workers**: The backend runs 4 parallel instances to utilize the quad-core CPU.