Spaces:
Running on Zero
Running on Zero
| # Exercise Router Training Report | |
| Generated after the full Modal training run on June 12, 2026. | |
| ## Summary | |
| The exercise router was trained and evaluated for `squat`, `push_up`, `shoulder_press`, and | |
| `unknown`. The final active artifact is the BiLSTM temporal model. The scikit-learn baseline is | |
| retained as a reference and fallback artifact. | |
| | Field | Value | | |
| | ---------------------- | ------------------------------------------------------------------------- | | |
| | Modal run | `https://modal.com/apps/nlag/main/ap-9CDLM3tMgOlCYE2JCBe71H` | | |
| | Hugging Face repo | `build-small-hackathon/pozify-exercise-router` | | |
| | Selected model | `temporal.pt` | | |
| | Selected artifact | `temporal.pt` | | |
| | Selection rule | Prefer BiLSTM temporal when available; baseline falls back when missing | | |
| | Local active path | `models/exercise_router/active/temporal.pt` | | |
| | Baseline artifact path | `models/exercise_router/active/baseline.joblib` | | |
| | Router baseline alias | `models/exercise_router/active/router.joblib` | | |
| ## Data | |
| | Metric | Value | | |
| | -------------------------- | ----: | | |
| | Feature examples | 134 | | |
| | Window count | 2,224 | | |
| | Failed feature extractions | 0 | | |
| | Push-up windows | 287 | | |
| | Shoulder press windows | 646 | | |
| | Squat windows | 659 | | |
| | Unknown windows | 632 | | |
| Unsupported Riccio dataset classes such as bicep curl are mapped to `unknown`. | |
| ## Training Setup | |
| | Model | Setup | | |
| | -------- | ------------------------------------------------------------------------------------- | | |
| | Baseline | scikit-learn `HistGradientBoostingClassifier` over engineered 30-frame window vectors | | |
| | Temporal | PyTorch BiLSTM over 30-frame feature tensors on Modal A10 | | |
| The Modal image and local verification environment use the Python 3.10 dependency set: | |
| | Dependency | Version | | |
| | -------------- | ------------ | | |
| | Python | `3.10.20` | | |
| | scikit-learn | `1.7.2` | | |
| | joblib | `1.5.3` | | |
| | torch | `2.11.0` | | |
| | numpy | `1.26.4` | | |
| | scipy | `1.15.3` | | |
| The BiLSTM hyperparameters follow the Riccio exercise-classification paper: | |
| | Hyperparameter | Value | | |
| | ------------------- | -----: | | |
| | Epochs | 73 | | |
| | Hidden units | 73 | | |
| | Dropout | 0.2174 | | |
| | Learning rate | 0.0004 | | |
| | Batch size | 54 | | |
| | Final training loss | 0.0003 | | |
| ## Model Complexity | |
| Counts were checked from the local trained artifacts with `uv run` under Python 3.10. | |
| | Model | Count type | Value | | |
| | --------------- | ----------------------------------------- | ------: | | |
| | Baseline | Neural-network-style trainable parameters | 0 | | |
| | Baseline | Input features | 1,422 | | |
| | Baseline | Trees | 800 | | |
| | Baseline | Total tree nodes | 21,254 | | |
| | Baseline | Split nodes | 10,227 | | |
| | Baseline | Leaves | 11,027 | | |
| | Baseline | Approximate learned scalar state | 41,708 | | |
| | BiLSTM temporal | Trainable parameters | 182,796 | | |
| | BiLSTM temporal | Input features per frame | 237 | | |
| | BiLSTM temporal | Hidden units | 73 | | |
| | BiLSTM temporal | Layers | 1 | | |
| | BiLSTM temporal | Output classes | 4 | | |
| The baseline is a tree-based `HistGradientBoostingClassifier`, so it does not have trainable tensor | |
| parameters in the same sense as a neural network. The BiLSTM parameter count includes both LSTM | |
| directions and the linear classification head. | |
| ## Training Metrics | |
| | Model | Validation accuracy | Unknown rejection rate | | |
| | --------------- | ------------------: | --------------------------------------: | | |
| | Baseline | 0.9910 | Not reported in baseline training stage | | |
| | BiLSTM temporal | 0.9843 | 0.9843 | | |
| ## Selection Evaluation | |
| The final evaluation scored every available trained artifact on the cached router windows. | |
| The baseline scored slightly higher on this cache, but the active router is BiLSTM so routing uses | |
| the temporal pose-window sequence directly. | |
| | Model | Artifact | Accuracy | Unknown rejection rate | | |
| | --------------- | ----------------- | -------: | ---------------------: | | |
| | Baseline | `baseline.joblib` | 0.9982 | 0.9968 | | |
| | BiLSTM temporal | `temporal.pt` | 0.9969 | 0.9968 | | |
| ### Baseline Precision/Recall | |
| | Label | Precision | Recall | | |
| | ---------------- | --------: | -----: | | |
| | `squat` | 0.9985 | 0.9970 | | |
| | `push_up` | 0.9965 | 1.0000 | | |
| | `shoulder_press` | 0.9985 | 1.0000 | | |
| | `unknown` | 0.9984 | 0.9968 | | |
| ### BiLSTM Precision/Recall | |
| | Label | Precision | Recall | | |
| | ---------------- | --------: | -----: | | |
| | `squat` | 0.9939 | 0.9970 | | |
| | `push_up` | 1.0000 | 1.0000 | | |
| | `shoulder_press` | 0.9984 | 0.9954 | | |
| | `unknown` | 0.9968 | 0.9968 | | |
| ## Confusion Matrices | |
| Rows are true labels. Columns are predicted labels. | |
| ### Baseline | |
| | True \\ Predicted | `squat` | `push_up` | `shoulder_press` | `unknown` | | |
| | ----------------- | ------: | --------: | ---------------: | --------: | | |
| | `squat` | 657 | 1 | 0 | 1 | | |
| | `push_up` | 0 | 287 | 0 | 0 | | |
| | `shoulder_press` | 0 | 0 | 646 | 0 | | |
| | `unknown` | 1 | 0 | 1 | 630 | | |
| ### BiLSTM Temporal | |
| | True \\ Predicted | `squat` | `push_up` | `shoulder_press` | `unknown` | | |
| | ----------------- | ------: | --------: | ---------------: | --------: | | |
| | `squat` | 657 | 0 | 1 | 1 | | |
| | `push_up` | 0 | 287 | 0 | 0 | | |
| | `shoulder_press` | 2 | 0 | 643 | 1 | | |
| | `unknown` | 2 | 0 | 0 | 630 | | |
| ## Published Artifacts | |
| The full run uploaded the following artifacts to Hugging Face and the fresh Hub download matched | |
| the local Modal volume artifacts. | |
| | Artifact | SHA-256 | | |
| | ----------------------- | ------------------------------------------------------------------ | | |
| | `baseline.joblib` | `dbe53cab28ff664d1eb08546b24e0b5a9cd374d70b6a59bb5f17c2e9af58517f` | | |
| | `router.joblib` | `dbe53cab28ff664d1eb08546b24e0b5a9cd374d70b6a59bb5f17c2e9af58517f` | | |
| | `temporal.pt` | `db07644553a37ed7e939f22b8eb720b9cf392149bb219c7e5039cfcf2ac583a2` | | |
| | `evaluation.json` | `b1e5530d62532d18c512fb38f2c1422f08caa5f067afd978589a1621bed11560` | | |
| | `router_selection.json` | `d8263f0cf739c6aa75a27bbf23246277a15222ce24e449fe300ded72921c4177` | | |
| `baseline.joblib` and `router.joblib` are byte-identical. Runtime selection still loads | |
| `temporal.pt` through `router_selection.json`. | |
| ## Reproduction Commands | |
| Run the full training, evaluation, and publish flow: | |
| ```bash | |
| uv run modal run scripts/exercise_router_modal.py --stage all --repo-id build-small-hackathon/pozify-exercise-router | |
| ``` | |
| Download the active artifact, baseline artifacts, selection file, and metrics after evaluation: | |
| ```bash | |
| uv run modal volume get --force pozify-router-models /temporal.pt models/exercise_router/active/temporal.pt | |
| uv run modal volume get --force pozify-router-models /baseline.joblib models/exercise_router/active/baseline.joblib | |
| uv run modal volume get --force pozify-router-models /router.joblib models/exercise_router/active/router.joblib | |
| uv run modal volume get --force pozify-router-models /router_selection.json models/exercise_router/active/router_selection.json | |
| uv run modal volume get --force pozify-router-models /evaluation.json models/exercise_router/active/evaluation.json | |
| uv run modal volume get --force pozify-router-models /baseline_metrics.json models/exercise_router/active/baseline_metrics.json | |
| uv run modal volume get --force pozify-router-models /temporal_metrics.json models/exercise_router/active/temporal_metrics.json | |
| ``` | |
| ## Verification | |
| The refreshed artifacts were verified under Python 3.10: | |
| ```bash | |
| uv run --extra dev ruff check | |
| uv run python -m compileall src scripts tests app.py | |
| uv run python -m unittest discover -s tests | |
| ``` | |
| The test suite passed with 75 tests, 1 skipped. | |
| ## Notes | |
| These metrics are router-window metrics from the current Modal feature cache, not a claim of | |
| generalization to every capture setup. Add more custom `unknown` clips and independent held-out video | |
| sets before treating the router as production-grade. | |