SETUP.md

# Setup Instructions

## 1. Install Dependencies

### Option A: Using pip (recommended)

```bash
pip install -r requirements.txt
```

### Option B: Using virtual environment (recommended for isolation)

```bash
# Create virtual environment
python -m venv venv

# Activate virtual environment
# On Windows (PowerShell):
.\venv\Scripts\Activate.ps1
# On Windows (CMD):
venv\Scripts\activate.bat
# On Linux/Mac:
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt
```

## 2. Prepare Data

Place your CSV files in the `data/` directory:

```bash
mkdir -p data
# Copy your CSV files to data/
```

## 3. Train Models

### Train Job Failure Prediction Model

```bash
# Option 1: Use all CSV files in data/ directory (recommended)
python train_job_failure.py

# Option 2: Specify files explicitly
python train_job_failure.py data/true_export_report_20260120.csv data/true_export_report_20260121.csv

# Option 3: Use glob pattern (works in PowerShell and bash)
python train_job_failure.py data/*.csv
```

### Train Anomaly Detection Model

```bash
# Option 1: Use all CSV files in data/ directory (recommended)
python train_anomaly.py

# Option 2: Specify files explicitly
python train_anomaly.py data/true_export_report_20260120.csv data/true_export_report_20260121.csv

# Option 3: Use glob pattern
python train_anomaly.py data/*.csv
```

## 4. Verify Models

After training, check that models were created:

```bash
ls models/
```

You should see:
- `job_fail_pipeline_cpu.joblib`
- `anomaly_autoencoder_cpu.keras`
- `anomaly_scaler.joblib`
- `feature_schema.json`
- `shap_background.npy`
- `anomaly_features.joblib`
- `anomaly_threshold.joblib`

## 5. Run the Service

### Local Development

```bash
uvicorn app:app --host 0.0.0.0 --port 8000
```

### Docker

```bash
docker-compose up --build
```

## Troubleshooting

### "ModuleNotFoundError: No module named 'tensorflow'"

Install dependencies:
```bash
pip install -r requirements.txt
```

### "No data files found!"

1. Check that CSV files exist in `data/` directory:
   ```bash
   ls data/
   ```

2. Use explicit file paths:
   ```bash
   python train_job_failure.py data/true_export_report_20260120.csv
   ```

### PowerShell glob pattern not working

The scripts now handle glob patterns internally. You can use:
```powershell
python train_job_failure.py data/*.csv
```

Or just run without arguments to use all CSV files in `data/`:
```powershell
python train_job_failure.py
```