docs/20-router-training-report.md

Exercise Router Training Report

Generated after the full Modal training run on June 12, 2026.

Summary

The exercise router was trained and evaluated for squat, push_up, shoulder_press, and unknown. The final active artifact is the BiLSTM temporal model. The scikit-learn baseline is retained as a reference and fallback artifact.

Field	Value
Modal run	https://modal.com/apps/nlag/main/ap-9CDLM3tMgOlCYE2JCBe71H
Hugging Face repo	build-small-hackathon/pozify-exercise-router
Selected model	temporal.pt
Selected artifact	temporal.pt
Selection rule	Prefer BiLSTM temporal when available; baseline falls back when missing
Local active path	models/exercise_router/active/temporal.pt
Baseline artifact path	models/exercise_router/active/baseline.joblib
Router baseline alias	models/exercise_router/active/router.joblib

Data

Metric	Value
Feature examples	134
Window count	2,224
Failed feature extractions	0
Push-up windows	287
Shoulder press windows	646
Squat windows	659
Unknown windows	632

Unsupported Riccio dataset classes such as bicep curl are mapped to unknown.

Training Setup

Model	Setup
Baseline	scikit-learn HistGradientBoostingClassifier over engineered 30-frame window vectors
Temporal	PyTorch BiLSTM over 30-frame feature tensors on Modal A10

The Modal image and local verification environment use the Python 3.10 dependency set:

Dependency	Version
Python	3.10.20
scikit-learn	1.7.2
joblib	1.5.3
torch	2.11.0
numpy	1.26.4
scipy	1.15.3

The BiLSTM hyperparameters follow the Riccio exercise-classification paper:

Hyperparameter	Value
Epochs	73
Hidden units	73
Dropout	0.2174
Learning rate	0.0004
Batch size	54
Final training loss	0.0003

Model Complexity

Counts were checked from the local trained artifacts with uv run under Python 3.10.

Model	Count type	Value
Baseline	Neural-network-style trainable parameters	0
Baseline	Input features	1,422
Baseline	Trees	800
Baseline	Total tree nodes	21,254
Baseline	Split nodes	10,227
Baseline	Leaves	11,027
Baseline	Approximate learned scalar state	41,708
BiLSTM temporal	Trainable parameters	182,796
BiLSTM temporal	Input features per frame	237
BiLSTM temporal	Hidden units	73
BiLSTM temporal	Layers	1
BiLSTM temporal	Output classes	4

The baseline is a tree-based HistGradientBoostingClassifier, so it does not have trainable tensor parameters in the same sense as a neural network. The BiLSTM parameter count includes both LSTM directions and the linear classification head.

Training Metrics

Model	Validation accuracy	Unknown rejection rate
Baseline	0.9910	Not reported in baseline training stage
BiLSTM temporal	0.9843	0.9843

Selection Evaluation

The final evaluation scored every available trained artifact on the cached router windows. The baseline scored slightly higher on this cache, but the active router is BiLSTM so routing uses the temporal pose-window sequence directly.

Model	Artifact	Accuracy	Unknown rejection rate
Baseline	baseline.joblib	0.9982	0.9968
BiLSTM temporal	temporal.pt	0.9969	0.9968

Baseline Precision/Recall

Label	Precision	Recall
squat	0.9985	0.9970
push_up	0.9965	1.0000
shoulder_press	0.9985	1.0000
unknown	0.9984	0.9968

BiLSTM Precision/Recall

Label	Precision	Recall
squat	0.9939	0.9970
push_up	1.0000	1.0000
shoulder_press	0.9984	0.9954
unknown	0.9968	0.9968

Confusion Matrices

Rows are true labels. Columns are predicted labels.

Baseline

True \ Predicted	squat	push_up	shoulder_press	unknown
squat	657	1	0	1
push_up	0	287	0	0
shoulder_press	0	0	646	0
unknown	1	0	1	630

BiLSTM Temporal

True \ Predicted	squat	push_up	shoulder_press	unknown
squat	657	0	1	1
push_up	0	287	0	0
shoulder_press	2	0	643	1
unknown	2	0	0	630

Published Artifacts

The full run uploaded the following artifacts to Hugging Face and the fresh Hub download matched the local Modal volume artifacts.

Artifact	SHA-256
baseline.joblib	dbe53cab28ff664d1eb08546b24e0b5a9cd374d70b6a59bb5f17c2e9af58517f
router.joblib	dbe53cab28ff664d1eb08546b24e0b5a9cd374d70b6a59bb5f17c2e9af58517f
temporal.pt	db07644553a37ed7e939f22b8eb720b9cf392149bb219c7e5039cfcf2ac583a2
evaluation.json	b1e5530d62532d18c512fb38f2c1422f08caa5f067afd978589a1621bed11560
router_selection.json	d8263f0cf739c6aa75a27bbf23246277a15222ce24e449fe300ded72921c4177

baseline.joblib and router.joblib are byte-identical. Runtime selection still loads temporal.pt through router_selection.json.

Reproduction Commands

Run the full training, evaluation, and publish flow:

uv run modal run scripts/exercise_router_modal.py --stage all --repo-id build-small-hackathon/pozify-exercise-router

Download the active artifact, baseline artifacts, selection file, and metrics after evaluation:

uv run modal volume get --force pozify-router-models /temporal.pt models/exercise_router/active/temporal.pt
uv run modal volume get --force pozify-router-models /baseline.joblib models/exercise_router/active/baseline.joblib
uv run modal volume get --force pozify-router-models /router.joblib models/exercise_router/active/router.joblib
uv run modal volume get --force pozify-router-models /router_selection.json models/exercise_router/active/router_selection.json
uv run modal volume get --force pozify-router-models /evaluation.json models/exercise_router/active/evaluation.json
uv run modal volume get --force pozify-router-models /baseline_metrics.json models/exercise_router/active/baseline_metrics.json
uv run modal volume get --force pozify-router-models /temporal_metrics.json models/exercise_router/active/temporal_metrics.json

Verification

The refreshed artifacts were verified under Python 3.10:

uv run --extra dev ruff check
uv run python -m compileall src scripts tests app.py
uv run python -m unittest discover -s tests

The test suite passed with 75 tests, 1 skipped.

Notes

These metrics are router-window metrics from the current Modal feature cache, not a claim of generalization to every capture setup. Add more custom unknown clips and independent held-out video sets before treating the router as production-grade.