models/mlp/README.md

models/mlp

Two-hidden-layer perceptron for binary focus classification. Takes the same 10 features as XGBoost but trains via backpropagation in PyTorch.

Architecture

Input (10) --> Linear(64) --> ReLU --> Linear(32) --> ReLU --> Linear(2) --> Softmax

~2,850 trainable parameters. Small enough for real-time CPU inference with no perceptible latency.

Training

python -m models.mlp.train

Reads all hyperparameters from config/default.yaml:

Parameter	Default	Source
Epochs	30	mlp.epochs
Batch size	32	mlp.batch_size
Learning rate	0.001	mlp.lr
Hidden sizes	[64, 32]	mlp.hidden_sizes
Weight decay	0	mlp.weight_decay
Seed	42	data.seed

Training uses CrossEntropyLoss with Adam optimiser and early stopping on validation F1. Best checkpoint saved to checkpoints/mlp_best.pt.

ClearML integration

Enable with USE_CLEARML=1:

USE_CLEARML=1 python -m models.mlp.train
USE_CLEARML=1 CLEARML_QUEUE=gpu python -m models.mlp.train

Logs to project "FocusGuards Large Group Project" on the KCL ClearML server. Tracked:

• Full hyperparameter config from default.yaml
• Per-epoch scalars: train/val loss, accuracy, F1, ROC-AUC
• Confusion matrix and ROC curve plots
• Model checkpoint registration via OutputModel with headline metrics
• Dataset statistics (sample counts, class balance)
• Reproducibility artifacts (config YAML, requirements.txt, git SHA)

LOPO evaluation

The training script runs Leave-One-Person-Out cross-validation after the main training loop. Each of the 9 participants is held out in turn, and a fresh model is trained on the remaining 8. Results:

Metric	Value
LOPO AUC	0.862
Optimal threshold (Youden's J)	0.228
F1 at optimal threshold	0.858
F1 at default 0.50	0.815
Improvement from threshold tuning	+4.3 pp

Outputs

File	Location
Best model checkpoint	checkpoints/mlp_best.pt
Scaler	checkpoints/scaler_mlp.joblib
Test predictions	evaluation/logs/mlp_test_predictions.csv
Test metrics	evaluation/logs/mlp_test_metrics_summary.json
Training log	evaluation/logs/mlp_training_log.csv

Hyperparameter sweep

python -m models.mlp.sweep

Optuna-based search over learning rate, hidden sizes, batch size, and weight decay using LOPO F1 as the objective.