models/mlp/README.md

# models/mlp

Two-hidden-layer perceptron for binary focus classification. Takes the same 10 features as XGBoost but trains via backpropagation in PyTorch.

## Architecture

```
Input (10) --> Linear(64) --> ReLU --> Linear(32) --> ReLU --> Linear(2) --> Softmax
```

~2,850 trainable parameters. Small enough for real-time CPU inference with no perceptible latency.

## Training

```bash
python -m models.mlp.train
```

Reads all hyperparameters from `config/default.yaml`:

| Parameter | Default | Source |
|-----------|---------|--------|
| Epochs | 30 | `mlp.epochs` |
| Batch size | 32 | `mlp.batch_size` |
| Learning rate | 0.001 | `mlp.lr` |
| Hidden sizes | [64, 32] | `mlp.hidden_sizes` |
| Weight decay | 0 | `mlp.weight_decay` |
| Seed | 42 | `data.seed` |

Training uses `CrossEntropyLoss` with `Adam` optimiser and early stopping on validation F1. Best checkpoint saved to `checkpoints/mlp_best.pt`.

## ClearML integration

Enable with `USE_CLEARML=1`:

```bash
USE_CLEARML=1 python -m models.mlp.train
USE_CLEARML=1 CLEARML_QUEUE=gpu python -m models.mlp.train
```

Logs to project "FocusGuards Large Group Project" on the KCL ClearML server. Tracked:

- Full hyperparameter config from `default.yaml`
- Per-epoch scalars: train/val loss, accuracy, F1, ROC-AUC
- Confusion matrix and ROC curve plots
- Model checkpoint registration via `OutputModel` with headline metrics
- Dataset statistics (sample counts, class balance)
- Reproducibility artifacts (config YAML, requirements.txt, git SHA)

## LOPO evaluation

The training script runs Leave-One-Person-Out cross-validation after the main training loop. Each of the 9 participants is held out in turn, and a fresh model is trained on the remaining 8. Results:

| Metric | Value |
|--------|-------|
| LOPO AUC | 0.862 |
| Optimal threshold (Youden's J) | 0.228 |
| F1 at optimal threshold | 0.858 |
| F1 at default 0.50 | 0.815 |
| Improvement from threshold tuning | +4.3 pp |

## Outputs

| File | Location |
|------|----------|
| Best model checkpoint | `checkpoints/mlp_best.pt` |
| Scaler | `checkpoints/scaler_mlp.joblib` |
| Test predictions | `evaluation/logs/mlp_test_predictions.csv` |
| Test metrics | `evaluation/logs/mlp_test_metrics_summary.json` |
| Training log | `evaluation/logs/mlp_training_log.csv` |

## Hyperparameter sweep

```bash
python -m models.mlp.sweep
```

Optuna-based search over learning rate, hidden sizes, batch size, and weight decay using LOPO F1 as the objective.