Deploy MotionBench Space

.dockerignore

@@ -0,0 +1,23 @@
+.git
+.github
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+
+.venv/
+venv/
+env/
+.env
+
+data/
+archive/
+models/
+results/
+checkpoints/
+wandb/
+runs/
+
+.ipynb_checkpoints/
+.DS_Store
+.vscode/

.gitignore

@@ -0,0 +1,26 @@
+__pycache__/
+.ipynb_checkpoints/
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+
+.venv/
+venv/
+env/
+
+.vscode/
+.idea/
+.DS_Store
+Thumbs.db
+
+
+data/**
+!data/.gitkeep
+
+models/**
+!models/.gitkeep
+!models/*/
+!models/*/train.py
+
+archive/**

Dockerfile

@@ -0,0 +1,34 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    libglib2.0-0 \
+    libgl1 \
+    libsm6 \
+    libxext6 \
+    libxrender1 \
+    ffmpeg \
+    git \
+    git-lfs \
+    && rm -rf /var/lib/apt/lists/*
+
+ENV PYTHONUNBUFFERED=1 \
+    STREAMLIT_BROWSER_GATHER_USAGE_STATS=false
+
+COPY requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+# scripts to run the app and utilities
+COPY scripts ./scripts
+
+# Clone the models from my Hugging Face repository
+RUN git clone --depth 1 https://huggingface.co/johnamit/motionbench-models /tmp/motionbench-models \
+    && cd /tmp/motionbench-models \
+    && git lfs pull \
+    && mv /tmp/motionbench-models/models /app/models \
+    && rm -rf /tmp/motionbench-models
+
+EXPOSE 7860
+
+ENTRYPOINT ["streamlit", "run", "scripts/app/motionbench.py", "--server.port=7860", "--server.address=0.0.0.0"]

README.md

@@ -0,0 +1,192 @@
+---
+title: MotionBench
+emoji: 🏋️
+colorFrom: blue
+colorTo: indigo
+sdk: docker
+app_port: 7860
+pinned: false
+---
+
+<img src="assets/motionbench.png" alt="MotionBench" width="900"><br>
+
+[<img src="https://img.shields.io/badge/HuggingFace-Dataset-black?style=for-the-badge&logo=huggingface&logoColor=FFD21E&labelColor=ff7f1e" alt="View Dataset on Hugging Face"/>](https://huggingface.co/datasets/johnamit/motionbench-data)
+&nbsp;&nbsp;
+[<img src="https://img.shields.io/badge/HuggingFace-Models-black?style=for-the-badge&logo=huggingface&logoColor=FFD21E&labelColor=ff7f1e" alt="View Models on Hugging Face"/>](https://huggingface.co/johnamit/motionbench-models)
+
+MotionBench is a real-time pose-based exercise recognition project designed for practical local usage. It classifies exercise motion from short temporal windows, estimates repetition counts with a deterministic finite-state method, and reports similarity against class-level motion centroids.
+
+<p>
+  <a href="#overview"><img src="https://img.shields.io/badge/Overview-111111?style=for-the-badge" alt="Overview"></a>
+  <a href="#dataset"><img src="https://img.shields.io/badge/Dataset-111111?style=for-the-badge" alt="Dataset"></a>
+  <a href="#models"><img src="https://img.shields.io/badge/Models-111111?style=for-the-badge" alt="Models"></a>
+  <a href="#training"><img src="https://img.shields.io/badge/Training-111111?style=for-the-badge" alt="Training"></a>
+  <a href="#inference-local"><img src="https://img.shields.io/badge/Inference-111111?style=for-the-badge" alt="Inference"></a>
+  <a href="#streamlit-app"><img src="https://img.shields.io/badge/Streamlit-111111?style=for-the-badge" alt="Streamlit"></a>
+  <a href="#citations"><img src="https://img.shields.io/badge/Citations-111111?style=for-the-badge" alt="Citations"></a>
+  <a href="#license"><img src="https://img.shields.io/badge/License-111111?style=for-the-badge" alt="License"></a>
+</p>
+
+## Overview
+MotionBench is built to run the full workflow from start to finish. You can prepare sequence data, train models, benchmark inference, and run real-time prediction from a webcam.
+
+The runtime pipeline is simple. It captures frames, extracts pose-based features, builds rolling windows, and predicts one of six exercise classes. It also estimates repetitions with a deterministic finite-state method and reports a centroid similarity score for live feedback.
+
+Core work happens in `data/`, `models/`, `scripts/`, and `results/`. Older or non-essential files are moved to `archive/` to keep the main repository clear and easy to review.
+
+## Dataset
+This repo stays lightweight on GitHub. Download the dataset files from Hugging Face and place them in `data/`.
+
+```bash
+git clone https://huggingface.co/datasets/johnamit/motionbench-data data
+```
+
+For local usage, keep split files under `data/`.
+
+The workflow expects fixed sequence splits (`train`, `val`, `test_internal`) and optionally a separate home/generalization test split.
+
+Expected split files:
+- `data/train_sequences.csv`
+- `data/val_sequences.csv`
+- `data/test_internal_sequences.csv`
+- `data/test_home_sequences.csv` (optional for home/generalization evaluation)
+
+To regenerate centralized fixed splits:
+
+```bash
+python scripts/preprocess/create_fixed_splits.py --input-file data/train_sequences_full.csv --output-dir data
+```
+
+## Models
+Download trained model files from Hugging Face and place them in `models/`.
+
+```bash
+git clone https://huggingface.co/johnamit/motionbench-models models
+```
+
+This project includes six sequence models with different strengths. Some are strong on temporal memory, some are better for latency, and some are better at capturing structured feature relationships.
+
+**BiLSTM:**
+The bidirectional LSTM processes each sequence in forward and backward directions within the input window, so the classifier can use context from both ends of the motion segment. This helps when important movement details are spread across the whole sequence, not just a single frame.
+
+**LSTM:**
+Unidirectional LSTM reads movement step by step in time. It is a simple and reliable sequence model, so it works well as a strong baseline for exercise classification while keeping runtime reasonable.
+
+**GRU:**
+The GRU uses gating similar to LSTM but with fewer internal components, which can reduce parameter count and improve efficiency. In practice, it is a strong candidate when you want robust sequence modeling with lighter recurrent overhead.
+
+**TCN:**
+The temporal convolutional network uses dilated 1D convolutions and residual blocks to to learn patterns over short and long time ranges. Because convolutional operations are parallelizable, it is often fast at inference, which makes it a good option when responsiveness matters.
+
+**CNN-BiLSTM:**
+This hybrid architecture first applies temporal convolutions to capture short local motion patterns, then a BiLSTM models how those patterns evolve over time. This gives both local detail and sequence context.
+
+**ST-GCN-inspired (feature-graph variant):**
+This ST-GCN-style model treats features as connected nodes and learns both their relationships and how they change over time. It can help when interactions between pose features are important for classification.
+
+## Training
+Train each model from the shared sequence splits in `data/`.
+
+```bash
+python models/bilstm/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/bilstm/results
+python models/lstm/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/lstm/results
+python models/gru/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/gru/results
+python models/tcn/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/tcn/results
+python models/cnn_bilstm/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/cnn_bilstm/results
+python models/st_gcn/train.py --train-file data/train_sequences.csv --val-file data/val_sequences.csv --test-file data/test_internal_sequences.csv --output-dir models/st_gcn/results
+```
+
+If centroid assets are missing or if models were retrained, rebuild similarity assets:
+
+```bash
+python scripts/preprocess/build_similarity_assets.py --train-file data/train_sequences.csv --models-root models
+```
+
+## Inference (Local)
+Run offline evaluation on home/generalization test data:
+
+```bash
+python scripts/evaluate/evaluate_home_set.py --test-file data/test_home_sequences.csv --models-root models --output-dir results/eval_offline_home
+```
+
+Run inference benchmarking:
+
+```bash
+python scripts/benchmark/benchmark_inference.py --input-file data/test_home_sequences.csv --models-root models --output-dir results/benchmark_inference
+```
+
+Run realtime webcam evaluation (CLI):
+
+```bash
+python scripts/realtime_eval/evaluate_realtime_webcam.py --model-name bilstm --models-root models --output-dir results/eval_realtime
+```
+
+## Streamlit App
+Launch the local Streamlit interface:
+
+```bash
+streamlit run scripts/app/motionbench.py
+```
+
+In the app you can select a model, test camera capture, start a live session, and monitor predicted class, repetition count, and similarity score.
+
+## Citations
+
+**Bidirectional Long Short-Term Memory (BiLSTM)**
+```bibtex
+@article{riccio2024real,
+  title={Real-time fitness exercise classification and counting from video frames},
+  author={Riccio, Riccardo},
+  journal={arXiv preprint arXiv:2411.11548},
+  year={2024}
+}
+```
+
+**Gated Recurrent Unit (GRU)**
+```bibtex
+@article{chung2014empirical,
+  title={Empirical evaluation of gated recurrent neural networks on sequence modeling},
+  author={Chung, Junyoung and Gulcehre, Caglar and Cho, KyungHyun and Bengio, Yoshua},
+  journal={arXiv preprint arXiv:1412.3555},
+  year={2014}
+}
+```
+
+**Temporal Convolutional Network (TCN)**
+```bibtex
+@inproceedings{lea2017temporal,
+  title={Temporal convolutional networks for action segmentation and detection},
+  author={Lea, Colin and Flynn, Michael D and Vidal, Rene and Reiter, Austin and Hager, Gregory D},
+  booktitle={proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
+  pages={156--165},
+  year={2017}
+}
+```
+
+**Spatial Temporal Graph Convolutional Network**
+```bibtex
+@inproceedings{yan2018spatial,
+  title={Spatial temporal graph convolutional networks for skeleton-based action recognition},
+  author={Yan, Sijie and Xiong, Yuanjun and Lin, Dahua},
+  booktitle={Proceedings of the AAAI conference on artificial intelligence},
+  volume={32},
+  number={1},
+  year={2018}
+}
+```
+
+**CNN BiLSTM Hybrid**
+```bibtex
+@online{dhomane2024cnnbilstm,
+  author  = {Shreyas Dhomane},
+  title   = {CNN + BiLSTM Architecture: A Practical Guide},
+  year    = {2024},
+  month   = oct,
+  day     = {23},
+  url     = {https://medium.com/@shreyas.dhomane22/cnn-bilstm-architecture-a-practical-guide-c81829022820},
+  note    = {Medium article. Accessed: 2026-04-22}
+}
+```
+
+## License
+This project is released under the MIT License.

requirements.txt

@@ -0,0 +1,15 @@
+numpy==2.2.6
+pandas==2.3.3
+scikit-learn==1.7.2
+scipy==1.15.3
+joblib==1.5.3
+matplotlib==3.10.9
+tqdm==4.67.3
+
+opencv-python==4.13.0.92
+mediapipe==0.10.14
+
+streamlit==1.57.0
+
+torch==2.11.0
+torchvision==0.26.0

scripts/app/motionbench.py

@@ -0,0 +1,244 @@
+from collections import Counter, deque
+import sys
+import time
+from pathlib import Path
+from types import SimpleNamespace
+
+import cv2
+import joblib
+import mediapipe as mp
+import numpy as np
+import streamlit as st
+import torch
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+from scripts.evaluate.rep_counting_methods import EXERCISE_CONFIGS, FixedThresholdFSMCounter, SmoothingBuffer, extract_primary_angle, normalize_exercise_name
+from scripts.realtime_eval.evaluate_realtime_webcam import (
+    MODEL_SPECS,
+    build_landmark_indices,
+    build_model_and_tools,
+    extract_frame_features,
+    get_angle_triplets,
+    load_pose_module,
+)
+
+
+SEQUENCE_LENGTH = 30
+FEATURE_COUNT = 78
+LABEL_SMOOTHING_WINDOW = 5
+DEFAULT_MODELS_ROOT = "models"
+DEFAULT_PREDICTION_INTERVAL = 1.0
+CAMERA_INDEX_CANDIDATES = [0, 1, 2]
+
+
+def load_runtime(model_name: str, models_root: str, feature_count: int):
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    args = SimpleNamespace(model_name=model_name, models_root=models_root, feature_count=feature_count)
+    model, scaler, label_encoder = build_model_and_tools(args, device)
+    return device, model, scaler, label_encoder
+
+
+def load_similarity_asset(model_name: str, models_root: str):
+    asset_path = Path(models_root) / model_name / "weights" / "similarity_centroids.pkl"
+    if not asset_path.exists():
+        return None
+    return joblib.load(asset_path)
+
+
+def cosine_similarity_percent(vector_a: np.ndarray, vector_b: np.ndarray) -> float:
+    denom = float(np.linalg.norm(vector_a) * np.linalg.norm(vector_b))
+    if denom <= 1e-8:
+        return 0.0
+    score = float(np.dot(vector_a, vector_b) / denom)
+    score = max(-1.0, min(1.0, score))
+    return ((score + 1.0) / 2.0) * 100.0
+
+
+def smooth_label(label_history: deque[str]) -> str:
+    if not label_history:
+        return "none"
+    counts = Counter(label_history)
+    return counts.most_common(1)[0][0]
+
+
+def read_valid_frame(capture: cv2.VideoCapture, max_reads: int = 20) -> np.ndarray | None:
+    frame_bgr = None
+    for _ in range(max_reads):
+        ok, candidate = capture.read()
+        if not ok:
+            continue
+        if float(np.mean(candidate)) > 5.0:
+            return candidate
+        frame_bgr = candidate
+    if frame_bgr is not None and float(np.mean(frame_bgr)) > 5.0:
+        return frame_bgr
+    return None
+
+
+def open_camera_with_fallback() -> cv2.VideoCapture | None:
+    for camera_index in CAMERA_INDEX_CANDIDATES:
+        capture = cv2.VideoCapture(camera_index)
+        if not capture.isOpened():
+            capture.release()
+            continue
+        frame_bgr = read_valid_frame(capture, max_reads=10)
+        if frame_bgr is not None:
+            return capture
+        capture.release()
+    return None
+
+
+def main():
+    st.set_page_config(page_title="MotionBench", layout="wide")
+    st.title("MotionBench Live")
+
+    if "session_active" not in st.session_state:
+        st.session_state.session_active = False
+
+    model_name = st.selectbox("Select Model", options=list(MODEL_SPECS.keys()), index=0)
+    start_clicked = st.button("Start Session", width="stretch")
+
+    if start_clicked:
+        st.session_state.session_active = True
+
+    if not st.session_state.session_active:
+        st.info("Select a model, then start session.")
+        return
+
+    device, model, scaler, label_encoder = load_runtime(model_name, DEFAULT_MODELS_ROOT, feature_count=FEATURE_COUNT)
+    similarity_asset = load_similarity_asset(model_name, DEFAULT_MODELS_ROOT)
+    pose_module = load_pose_module()
+    landmark_indices = build_landmark_indices(pose_module)
+    angle_triplets = get_angle_triplets()
+
+    capture = open_camera_with_fallback()
+    if capture is None:
+        st.error("Could not capture a valid camera frame.")
+        st.session_state.session_active = False
+        return
+
+    left_col, right_col = st.columns([2, 1])
+    with right_col:
+        stop_clicked = st.button("Stop Session", width="stretch")
+    if stop_clicked:
+        st.session_state.session_active = False
+        capture.release()
+        st.info("Session stopped.")
+        return
+
+    with left_col:
+        frame_slot = st.empty()
+    with right_col:
+        metrics_slot = st.empty()
+
+    counter = None
+    smoother = None
+    active_exercise = None
+    current_label = "none"
+    current_similarity = None
+    last_prediction_time = 0.0
+    label_history = deque(maxlen=LABEL_SMOOTHING_WINDOW)
+    window = []
+    prediction_interval = DEFAULT_PREDICTION_INTERVAL
+
+    with pose_module.Pose(
+        static_image_mode=False,
+        model_complexity=1,
+        min_detection_confidence=0.5,
+        min_tracking_confidence=0.5,
+    ) as pose_estimator:
+        drawing_utils = mp.solutions.drawing_utils
+
+        while True:
+            ok, frame_bgr = capture.read()
+            if not ok:
+                break
+            frame_rgb = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
+            pose = pose_estimator.process(frame_rgb)
+
+            if drawing_utils is not None and pose.pose_landmarks:
+                drawing_utils.draw_landmarks(
+                    frame_bgr,
+                    pose.pose_landmarks,
+                    pose_module.POSE_CONNECTIONS,
+                )
+
+            frame_features = extract_frame_features(pose, landmark_indices, angle_triplets)
+
+            if frame_features is not None:
+                window.append(frame_features)
+                if len(window) > SEQUENCE_LENGTH:
+                    window.pop(0)
+
+            if len(window) == SEQUENCE_LENGTH and (time.time() - last_prediction_time) >= prediction_interval:
+                sequence_flat = np.array(window, dtype=np.float32).reshape(1, -1)
+                scaled_flat = scaler.transform(sequence_flat)
+                scaled = scaled_flat.reshape(1, SEQUENCE_LENGTH, FEATURE_COUNT)
+                input_tensor = torch.tensor(scaled, dtype=torch.float32, device=device)
+                with torch.inference_mode():
+                    logits = model(input_tensor)
+                    prediction_index = int(torch.argmax(logits, dim=1).item())
+                    predicted_label = label_encoder.classes_[prediction_index]
+                label_history.append(predicted_label)
+                current_label = smooth_label(label_history)
+                if similarity_asset is not None:
+                    scaled_vector = scaled_flat[0]
+                    centroids = similarity_asset.get("centroids", {})
+                    centroid_vector = centroids.get(current_label)
+                    if centroid_vector is not None:
+                        current_similarity = cosine_similarity_percent(
+                            scaled_vector.astype(np.float32),
+                            np.asarray(centroid_vector, dtype=np.float32),
+                        )
+                    else:
+                        current_similarity = None
+                last_prediction_time = time.time()
+
+            normalized_label = normalize_exercise_name(current_label)
+            current_reps = 0
+            if pose.pose_landmarks and normalized_label in EXERCISE_CONFIGS:
+                if normalized_label != active_exercise:
+                    config = EXERCISE_CONFIGS[normalized_label]
+                    counter = FixedThresholdFSMCounter(config.fixed_low, config.fixed_high, config.min_state_frames)
+                    smoother = SmoothingBuffer(config.smoothing_window)
+                    active_exercise = normalized_label
+
+                landmarks = {}
+                for name, index in landmark_indices.items():
+                    lm = pose.pose_landmarks.landmark[index]
+                    landmarks[name] = np.array([lm.x, lm.y, lm.z], dtype=np.float32) if lm.visibility >= 0.5 else np.array([0.0, 0.0, 0.0], dtype=np.float32)
+
+                config = EXERCISE_CONFIGS[normalized_label]
+                raw_angle = extract_primary_angle(landmarks, config)
+                smoothed_angle = smoother.update(raw_angle)
+                counter.update(smoothed_angle)
+                current_reps = counter.reps
+            else:
+                active_exercise = None
+                counter = None
+                smoother = None
+
+            show_frame = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
+            frame_slot.image(show_frame, channels="RGB", width="stretch")
+            similarity_text = f"{current_similarity:0.1f}%" if current_similarity is not None else "N/A"
+            metrics_slot.markdown(
+                f"### Live Metrics\n"
+                f"Model: `{model_name}`\n\n"
+                f"Exercise: `{current_label}`\n\n"
+                f"Reps: `{current_reps}`\n\n"
+                f"Similarity: `{similarity_text}`"
+            )
+
+            if not st.session_state.session_active:
+                break
+
+    capture.release()
+    st.session_state.session_active = False
+    st.success("Session finished.")
+
+
+if __name__ == "__main__":
+    main()

scripts/benchmark/benchmark_inference.py

@@ -0,0 +1,377 @@
+import argparse
+import time
+from pathlib import Path
+
+import joblib
+import numpy as np
+import pandas as pd
+import torch
+from torch import nn
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input-file", default="data/test_home_sequences.csv")
+    parser.add_argument("--models-root", default="models")
+    parser.add_argument("--output-dir", default="results/benchmark_inference")
+    parser.add_argument("--sequence-length", type=int, default=30)
+    parser.add_argument("--feature-count", type=int, default=78)
+    parser.add_argument("--warmup-runs", type=int, default=50)
+    parser.add_argument("--timed-runs", type=int, default=500)
+    parser.add_argument("--seed", type=int, default=42)
+    return parser.parse_args()
+
+
+class BidirectionalLstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.bilstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size * 2, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.bilstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class LstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.lstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.lstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class GruClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.gru = nn.GRU(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.gru(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class Chomp1d(nn.Module):
+    def __init__(self, chomp_size):
+        super().__init__()
+        self.chomp_size = chomp_size
+
+    def forward(self, input_tensor):
+        if self.chomp_size == 0:
+            return input_tensor
+        return input_tensor[:, :, :-self.chomp_size].contiguous()
+
+
+class TemporalBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, kernel_size, dilation, dropout):
+        super().__init__()
+        padding = (kernel_size - 1) * dilation
+        self.conv1 = nn.Conv1d(input_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp1 = Chomp1d(padding)
+        self.relu1 = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout)
+        self.conv2 = nn.Conv1d(output_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp2 = Chomp1d(padding)
+        self.relu2 = nn.ReLU()
+        self.dropout2 = nn.Dropout(dropout)
+        self.downsample = nn.Conv1d(input_channels, output_channels, kernel_size=1) if input_channels != output_channels else None
+        self.final_relu = nn.ReLU()
+
+    def forward(self, input_tensor):
+        output_tensor = self.conv1(input_tensor)
+        output_tensor = self.chomp1(output_tensor)
+        output_tensor = self.relu1(output_tensor)
+        output_tensor = self.dropout1(output_tensor)
+        output_tensor = self.conv2(output_tensor)
+        output_tensor = self.chomp2(output_tensor)
+        output_tensor = self.relu2(output_tensor)
+        output_tensor = self.dropout2(output_tensor)
+        residual_tensor = input_tensor if self.downsample is None else self.downsample(input_tensor)
+        return self.final_relu(output_tensor + residual_tensor)
+
+
+class TcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, channel_width, kernel_size, dropout):
+        super().__init__()
+        self.input_projection = nn.Conv1d(feature_count, channel_width, kernel_size=1)
+        self.block1 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=1, dropout=dropout)
+        self.block2 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=2, dropout=dropout)
+        self.block3 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=4, dropout=dropout)
+        self.classifier = nn.Linear(channel_width, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = input_sequence.transpose(1, 2)
+        temporal_tensor = self.input_projection(temporal_tensor)
+        temporal_tensor = self.block1(temporal_tensor)
+        temporal_tensor = self.block2(temporal_tensor)
+        temporal_tensor = self.block3(temporal_tensor)
+        final_timestep_tensor = temporal_tensor[:, :, -1]
+        return self.classifier(final_timestep_tensor)
+
+
+class CnnBiLstmClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, cnn_filters, cnn_kernel_size, lstm_units, dropout_probability):
+        super().__init__()
+        cnn_padding = cnn_kernel_size // 2
+        self.conv1d = nn.Conv1d(in_channels=feature_count, out_channels=cnn_filters, kernel_size=cnn_kernel_size, padding=cnn_padding)
+        self.relu = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout_probability)
+        self.bilstm = nn.LSTM(input_size=cnn_filters, hidden_size=lstm_units, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout2 = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(lstm_units * 2, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = input_sequence.transpose(1, 2)
+        temporal_tensor = self.conv1d(temporal_tensor)
+        temporal_tensor = self.relu(temporal_tensor)
+        temporal_tensor = self.dropout1(temporal_tensor)
+        temporal_tensor = temporal_tensor.transpose(1, 2)
+        recurrent_output, _ = self.bilstm(temporal_tensor)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout2(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class GraphConvolution(nn.Module):
+    def __init__(self, input_channels, output_channels):
+        super().__init__()
+        self.projection = nn.Conv2d(input_channels, output_channels, kernel_size=1)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        projected_tensor = self.projection(input_tensor)
+        return torch.einsum("nctv,vw->nctw", projected_tensor, adjacency_matrix)
+
+
+class StGcnBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, dropout, stride=1):
+        super().__init__()
+        self.graph_convolution = GraphConvolution(input_channels, output_channels)
+        self.temporal_convolution = nn.Sequential(
+            nn.BatchNorm2d(output_channels),
+            nn.ReLU(inplace=True),
+            nn.Conv2d(output_channels, output_channels, kernel_size=(9, 1), stride=(stride, 1), padding=(4, 0)),
+            nn.BatchNorm2d(output_channels),
+            nn.Dropout(dropout),
+        )
+        if stride != 1 or input_channels != output_channels:
+            self.residual = nn.Sequential(nn.Conv2d(input_channels, output_channels, kernel_size=1, stride=(stride, 1)), nn.BatchNorm2d(output_channels))
+        else:
+            self.residual = nn.Identity()
+        self.activation = nn.ReLU(inplace=True)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        residual_tensor = self.residual(input_tensor)
+        output_tensor = self.graph_convolution(input_tensor, adjacency_matrix)
+        output_tensor = self.temporal_convolution(output_tensor)
+        return self.activation(output_tensor + residual_tensor)
+
+
+class StGcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, dropout):
+        super().__init__()
+        self.input_batch_norm = nn.BatchNorm1d(feature_count)
+        self.register_parameter("adjacency_logits", nn.Parameter(torch.eye(feature_count)))
+        self.block1 = StGcnBlock(1, 64, dropout=dropout, stride=1)
+        self.block2 = StGcnBlock(64, 64, dropout=dropout, stride=1)
+        self.block3 = StGcnBlock(64, 128, dropout=dropout, stride=1)
+        self.classifier = nn.Linear(128, class_count)
+
+    def get_normalized_adjacency(self):
+        return torch.softmax(self.adjacency_logits, dim=1)
+
+    def forward(self, input_sequence):
+        batch_size, sequence_length, feature_count = input_sequence.shape
+        normalized_input = input_sequence.reshape(batch_size * sequence_length, feature_count)
+        normalized_input = self.input_batch_norm(normalized_input)
+        normalized_input = normalized_input.reshape(batch_size, sequence_length, feature_count)
+        graph_tensor = normalized_input.unsqueeze(1)
+        adjacency_matrix = self.get_normalized_adjacency()
+        graph_tensor = self.block1(graph_tensor, adjacency_matrix)
+        graph_tensor = self.block2(graph_tensor, adjacency_matrix)
+        graph_tensor = self.block3(graph_tensor, adjacency_matrix)
+        pooled_tensor = graph_tensor.mean(dim=2).mean(dim=2)
+        return self.classifier(pooled_tensor)
+
+
+MODEL_SPECS = {
+    "bilstm": {
+        "weight": "bidirectionallstm_model.pt",
+        "scaler": "bidirectionallstm_scaler.pkl",
+        "encoder": "bidirectionallstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: BidirectionalLstmClassifier(feature_count, 73, class_count, 0.2174),
+    },
+    "lstm": {
+        "weight": "lstm_model.pt",
+        "scaler": "lstm_scaler.pkl",
+        "encoder": "lstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: LstmClassifier(feature_count, 117, class_count, 0.3829),
+    },
+    "gru": {
+        "weight": "gru_model.pt",
+        "scaler": "gru_scaler.pkl",
+        "encoder": "gru_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: GruClassifier(feature_count, 96, class_count, 0.2),
+    },
+    "tcn": {
+        "weight": "tcn_model.pt",
+        "scaler": "tcn_scaler.pkl",
+        "encoder": "tcn_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: TcnClassifier(feature_count, class_count, 128, 3, 0.2),
+    },
+    "cnn_bilstm": {
+        "weight": "cnn_bilstm_model.pt",
+        "scaler": "cnn_bilstm_scaler.pkl",
+        "encoder": "cnn_bilstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: CnnBiLstmClassifier(feature_count, class_count, 128, 3, 73, 0.2),
+    },
+    "st_gcn": {
+        "weight": "st_gcn_model.pt",
+        "scaler": "st_gcn_scaler.pkl",
+        "encoder": "st_gcn_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: StGcnClassifier(feature_count, class_count, 0.2),
+    },
+}
+
+
+def set_seed(seed):
+    np.random.seed(seed)
+    torch.manual_seed(seed)
+    torch.cuda.manual_seed_all(seed)
+
+
+def load_features(input_file_path):
+    table = pd.read_csv(input_file_path)
+    metadata_columns = {"video_id", "exercise_label", "start_frame_index", "end_frame_index"}
+    feature_columns = [column_name for column_name in table.columns if column_name not in metadata_columns]
+    return table[feature_columns].to_numpy(dtype=np.float32)
+
+
+def run_device_benchmark(model, input_tensor, device, warmup_runs, timed_runs):
+    model = model.to(device)
+    input_tensor = input_tensor.to(device)
+    model.eval()
+
+    if device.type == "cuda":
+        torch.cuda.empty_cache()
+        torch.cuda.reset_peak_memory_stats(device)
+
+    with torch.inference_mode():
+        for _ in range(warmup_runs):
+            _ = model(input_tensor)
+        if device.type == "cuda":
+            torch.cuda.synchronize(device)
+
+        latencies_ms = []
+        for _ in range(timed_runs):
+            step_start = time.perf_counter()
+            _ = model(input_tensor)
+            if device.type == "cuda":
+                torch.cuda.synchronize(device)
+            latencies_ms.append((time.perf_counter() - step_start) * 1000.0)
+
+    mean_latency_ms = float(np.mean(latencies_ms))
+    p95_latency_ms = float(np.percentile(latencies_ms, 95))
+
+    peak_memory_mb = None
+    if device.type == "cuda":
+        peak_memory_mb = float(torch.cuda.max_memory_allocated(device) / (1024.0 * 1024.0))
+
+    return {
+        "mean_latency_ms": mean_latency_ms,
+        "p95_latency_ms": p95_latency_ms,
+        "peak_memory_mb": peak_memory_mb,
+    }
+
+
+def main():
+    args = parse_args()
+    set_seed(args.seed)
+
+    input_file_path = Path(args.input_file)
+    models_root_path = Path(args.models_root)
+    output_directory_path = Path(args.output_dir)
+    output_directory_path.mkdir(parents=True, exist_ok=True)
+
+    full_features = load_features(input_file_path)
+    single_window_features = full_features[0:1]
+
+    benchmark_rows = []
+
+    cpu_device = torch.device("cpu")
+    has_cuda = torch.cuda.is_available()
+    cuda_device = torch.device("cuda") if has_cuda else None
+
+    for model_name, spec in MODEL_SPECS.items():
+        weights_root = models_root_path / model_name / "weights"
+
+        scaler = joblib.load(weights_root / spec["scaler"])
+        label_encoder = joblib.load(weights_root / spec["encoder"])
+        class_count = len(label_encoder.classes_)
+
+        scaled_window = scaler.transform(single_window_features).reshape(1, args.sequence_length, args.feature_count)
+        input_tensor = torch.tensor(scaled_window, dtype=torch.float32)
+
+        model = spec["builder"](args.feature_count, class_count)
+        state_dict = torch.load(weights_root / spec["weight"], map_location="cpu")
+        model.load_state_dict(state_dict)
+
+        model_size_mb = float((weights_root / spec["weight"]).stat().st_size / (1024.0 * 1024.0))
+
+        cpu_stats = run_device_benchmark(model, input_tensor, cpu_device, args.warmup_runs, args.timed_runs)
+        benchmark_rows.append(
+            {
+                "model": model_name,
+                "device": "cpu",
+                "model_size_mb": model_size_mb,
+                **cpu_stats,
+            }
+        )
+
+        if has_cuda:
+            cuda_stats = run_device_benchmark(model, input_tensor, cuda_device, args.warmup_runs, args.timed_runs)
+            benchmark_rows.append(
+                {
+                    "model": model_name,
+                    "device": "cuda",
+                    "model_size_mb": model_size_mb,
+                    **cuda_stats,
+                }
+            )
+
+        print(f"Benchmarked: {model_name}")
+
+    benchmark_table = pd.DataFrame(benchmark_rows)
+
+    cpu_table = benchmark_table[benchmark_table["device"] == "cpu"].sort_values("mean_latency_ms", ascending=True)
+    cpu_csv_output_path = output_directory_path / "inference_benchmark_cpu.csv"
+    cpu_table.to_csv(cpu_csv_output_path, index=False)
+
+    print("\nInference Benchmark (CPU)")
+    print(cpu_table.to_string(index=False))
+    print(f"\nSaved: {cpu_csv_output_path}")
+
+    if has_cuda:
+        cuda_table = benchmark_table[benchmark_table["device"] == "cuda"].sort_values("mean_latency_ms", ascending=True)
+        cuda_csv_output_path = output_directory_path / "inference_benchmark_cuda.csv"
+        cuda_table.to_csv(cuda_csv_output_path, index=False)
+
+        print("\nInference Benchmark (CUDA)")
+        print(cuda_table.to_string(index=False))
+        print(f"\nSaved: {cuda_csv_output_path}")
+
+
+if __name__ == "__main__":
+    main()

scripts/evaluate/benchmark_rep_counting.py

@@ -0,0 +1,202 @@
+import argparse
+import sys
+from pathlib import Path
+
+import cv2
+import mediapipe as mp
+import numpy as np
+import pandas as pd
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+from scripts.evaluate.rep_counting_methods import EXERCISE_CONFIGS, FixedThresholdFSMCounter, SmoothingBuffer, extract_primary_angle, normalize_exercise_name
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--manifest-file", required=True)
+    parser.add_argument("--output-dir", default="results/eval_rep_counting")
+    parser.add_argument("--min-visibility", type=float, default=0.5)
+    return parser.parse_args()
+
+
+def load_pose_module():
+    return mp.solutions.pose
+
+
+def build_landmark_indices(mp_pose):
+    names = [
+        "LEFT_SHOULDER",
+        "RIGHT_SHOULDER",
+        "LEFT_HIP",
+        "RIGHT_HIP",
+        "LEFT_KNEE",
+        "RIGHT_KNEE",
+        "LEFT_ELBOW",
+        "RIGHT_ELBOW",
+        "LEFT_WRIST",
+        "RIGHT_WRIST",
+        "LEFT_ANKLE",
+        "RIGHT_ANKLE",
+    ]
+    return {name: mp_pose.PoseLandmark[name].value for name in names}
+
+
+def extract_landmark_points(results, landmark_indices, min_visibility):
+    if not results.pose_landmarks:
+        return None
+    points = {}
+    for name, index in landmark_indices.items():
+        landmark = results.pose_landmarks.landmark[index]
+        if landmark.visibility >= min_visibility:
+            points[name] = np.array([landmark.x, landmark.y, landmark.z], dtype=np.float32)
+        else:
+            points[name] = np.array([0.0, 0.0, 0.0], dtype=np.float32)
+    return points
+
+
+def evaluate_video(video_path, exercise_label, pose_estimator, landmark_indices, min_visibility):
+    config = EXERCISE_CONFIGS[exercise_label]
+    fixed_counter = FixedThresholdFSMCounter(config.fixed_low, config.fixed_high, config.min_state_frames)
+    smoothing = SmoothingBuffer(window_size=config.smoothing_window)
+
+    capture = cv2.VideoCapture(str(video_path))
+    processed_frames = 0
+    valid_angle_frames = 0
+
+    while capture.isOpened():
+        read_ok, frame_bgr = capture.read()
+        if not read_ok:
+            break
+
+        frame_rgb = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
+        pose_result = pose_estimator.process(frame_rgb)
+        points = extract_landmark_points(pose_result, landmark_indices, min_visibility)
+        if points is None:
+            processed_frames += 1
+            continue
+
+        raw_angle = extract_primary_angle(points, config)
+        smoothed_angle = smoothing.update(raw_angle)
+        if not np.isnan(smoothed_angle):
+            valid_angle_frames += 1
+        fixed_counter.update(smoothed_angle)
+        processed_frames += 1
+
+    capture.release()
+    return {
+        "paper_fsm": fixed_counter.reps,
+        "processed_frames": processed_frames,
+        "valid_angle_frames": valid_angle_frames,
+    }
+
+
+def compute_error_metrics(predicted_reps, true_reps):
+    absolute_error = abs(predicted_reps - true_reps)
+    relative_error_percent = (absolute_error / true_reps) * 100.0 if true_reps > 0 else 0.0
+    missed_reps = max(0, true_reps - predicted_reps)
+    false_reps = max(0, predicted_reps - true_reps)
+    return absolute_error, relative_error_percent, missed_reps, false_reps
+
+
+def validate_manifest_columns(dataframe):
+    required = {"video_path", "exercise_label"}
+    missing = required - set(dataframe.columns)
+    if missing:
+        missing_text = ", ".join(sorted(missing))
+        raise ValueError(f"Manifest is missing required columns: {missing_text}")
+
+
+def main():
+    args = parse_args()
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    manifest = pd.read_csv(args.manifest_file)
+    validate_manifest_columns(manifest)
+
+    mp_pose = load_pose_module()
+    landmark_indices = build_landmark_indices(mp_pose)
+    per_video_rows = []
+
+    has_true_reps = "true_reps" in manifest.columns
+
+    with mp_pose.Pose(
+        static_image_mode=False,
+        model_complexity=1,
+        enable_segmentation=False,
+        min_detection_confidence=0.5,
+        min_tracking_confidence=0.5,
+    ) as pose_estimator:
+        for row in manifest.itertuples(index=False):
+            video_path = Path(row.video_path)
+            raw_exercise = str(row.exercise_label)
+            exercise_label = normalize_exercise_name(raw_exercise)
+            true_reps = int(row.true_reps) if has_true_reps and not pd.isna(row.true_reps) else None
+
+            if exercise_label not in EXERCISE_CONFIGS:
+                raise ValueError(f"Unsupported exercise '{raw_exercise}' from manifest row: {video_path}")
+            if not video_path.exists():
+                raise FileNotFoundError(f"Video file not found: {video_path}")
+
+            counts = evaluate_video(
+                video_path=video_path,
+                exercise_label=exercise_label,
+                pose_estimator=pose_estimator,
+                landmark_indices=landmark_indices,
+                min_visibility=args.min_visibility,
+            )
+            pred_reps = counts["paper_fsm"]
+            row_data = {
+                "video_path": str(video_path),
+                "exercise_label": exercise_label,
+                "method": "paper_fsm",
+                "predicted_reps": pred_reps,
+                "processed_frames": counts["processed_frames"],
+                "valid_angle_frames": counts["valid_angle_frames"],
+            }
+            if true_reps is not None:
+                abs_err, rel_err, missed, false = compute_error_metrics(pred_reps, true_reps)
+                row_data["true_reps"] = true_reps
+                row_data["absolute_count_error"] = abs_err
+                row_data["relative_error_percent"] = rel_err
+                row_data["missed_reps"] = missed
+                row_data["false_reps"] = false
+            per_video_rows.append(row_data)
+
+            print(
+                f"{video_path.name} | {exercise_label} | "
+                f"predicted_reps={counts['paper_fsm']}"
+            )
+
+    per_video_df = pd.DataFrame(per_video_rows)
+    if "absolute_count_error" in per_video_df.columns:
+        summary_df = (
+            per_video_df.groupby("method", as_index=False)
+            .agg(
+                videos=("video_path", "count"),
+                mean_absolute_count_error=("absolute_count_error", "mean"),
+                mean_relative_error_percent=("relative_error_percent", "mean"),
+                total_missed_reps=("missed_reps", "sum"),
+                total_false_reps=("false_reps", "sum"),
+            )
+            .sort_values(by=["mean_absolute_count_error", "total_false_reps"], ascending=[True, True])
+        )
+    else:
+        summary_df = per_video_df.groupby("method", as_index=False).agg(videos=("video_path", "count"))
+
+    per_video_file = output_dir / "rep_counting_per_video.csv"
+    summary_file = output_dir / "rep_counting_summary.csv"
+    per_video_df.to_csv(per_video_file, index=False)
+    summary_df.to_csv(summary_file, index=False)
+
+    print("\nRep counting summary")
+    print(summary_df.to_string(index=False))
+    print(f"\nSaved: {per_video_file}")
+    print(f"Saved: {summary_file}")
+
+
+if __name__ == "__main__":
+    main()

scripts/evaluate/evaluate_home_set.py

@@ -0,0 +1,399 @@
+import argparse
+from pathlib import Path
+
+import joblib
+import numpy as np
+import pandas as pd
+import torch
+from sklearn.metrics import accuracy_score, classification_report, confusion_matrix, f1_score, precision_score, recall_score
+from torch import nn
+from torch.utils.data import DataLoader, Dataset
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--test-file", default="data/test_home_sequences.csv")
+    parser.add_argument("--models-root", default="models")
+    parser.add_argument("--output-dir", default="results/eval_offline_home")
+    parser.add_argument("--sequence-length", type=int, default=30)
+    parser.add_argument("--feature-count", type=int, default=78)
+    parser.add_argument("--batch-size", type=int, default=256)
+    return parser.parse_args()
+
+
+class SequenceDataset(Dataset):
+    def __init__(self, feature_tensor, label_tensor):
+        self.feature_tensor = feature_tensor
+        self.label_tensor = label_tensor
+
+    def __len__(self):
+        return len(self.label_tensor)
+
+    def __getitem__(self, index):
+        return self.feature_tensor[index], self.label_tensor[index]
+
+
+class BidirectionalLstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.bilstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size * 2, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.bilstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class LstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.lstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.lstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class GruClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.gru = nn.GRU(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.gru(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class Chomp1d(nn.Module):
+    def __init__(self, chomp_size):
+        super().__init__()
+        self.chomp_size = chomp_size
+
+    def forward(self, input_tensor):
+        if self.chomp_size == 0:
+            return input_tensor
+        return input_tensor[:, :, :-self.chomp_size].contiguous()
+
+
+class TemporalBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, kernel_size, dilation, dropout):
+        super().__init__()
+        padding = (kernel_size - 1) * dilation
+        self.conv1 = nn.Conv1d(input_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp1 = Chomp1d(padding)
+        self.relu1 = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout)
+        self.conv2 = nn.Conv1d(output_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp2 = Chomp1d(padding)
+        self.relu2 = nn.ReLU()
+        self.dropout2 = nn.Dropout(dropout)
+        self.downsample = nn.Conv1d(input_channels, output_channels, kernel_size=1) if input_channels != output_channels else None
+        self.final_relu = nn.ReLU()
+
+    def forward(self, input_tensor):
+        output_tensor = self.conv1(input_tensor)
+        output_tensor = self.chomp1(output_tensor)
+        output_tensor = self.relu1(output_tensor)
+        output_tensor = self.dropout1(output_tensor)
+        output_tensor = self.conv2(output_tensor)
+        output_tensor = self.chomp2(output_tensor)
+        output_tensor = self.relu2(output_tensor)
+        output_tensor = self.dropout2(output_tensor)
+        residual_tensor = input_tensor if self.downsample is None else self.downsample(input_tensor)
+        return self.final_relu(output_tensor + residual_tensor)
+
+
+class TcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, channel_width, kernel_size, dropout):
+        super().__init__()
+        self.input_projection = nn.Conv1d(feature_count, channel_width, kernel_size=1)
+        self.block1 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=1, dropout=dropout)
+        self.block2 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=2, dropout=dropout)
+        self.block3 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=4, dropout=dropout)
+        self.classifier = nn.Linear(channel_width, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = input_sequence.transpose(1, 2)
+        temporal_tensor = self.input_projection(temporal_tensor)
+        temporal_tensor = self.block1(temporal_tensor)
+        temporal_tensor = self.block2(temporal_tensor)
+        temporal_tensor = self.block3(temporal_tensor)
+        final_timestep_tensor = temporal_tensor[:, :, -1]
+        return self.classifier(final_timestep_tensor)
+
+
+class CnnBiLstmClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, cnn_filters, cnn_kernel_size, lstm_units, dropout_probability):
+        super().__init__()
+        cnn_padding = cnn_kernel_size // 2
+        self.conv1d = nn.Conv1d(in_channels=feature_count, out_channels=cnn_filters, kernel_size=cnn_kernel_size, padding=cnn_padding)
+        self.relu = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout_probability)
+        self.bilstm = nn.LSTM(input_size=cnn_filters, hidden_size=lstm_units, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout2 = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(lstm_units * 2, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = input_sequence.transpose(1, 2)
+        temporal_tensor = self.conv1d(temporal_tensor)
+        temporal_tensor = self.relu(temporal_tensor)
+        temporal_tensor = self.dropout1(temporal_tensor)
+        temporal_tensor = temporal_tensor.transpose(1, 2)
+        recurrent_output, _ = self.bilstm(temporal_tensor)
+        final_timestep_output = recurrent_output[:, -1, :]
+        dropout_output = self.dropout2(final_timestep_output)
+        return self.classifier(dropout_output)
+
+
+class GraphConvolution(nn.Module):
+    def __init__(self, input_channels, output_channels):
+        super().__init__()
+        self.projection = nn.Conv2d(input_channels, output_channels, kernel_size=1)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        projected_tensor = self.projection(input_tensor)
+        return torch.einsum("nctv,vw->nctw", projected_tensor, adjacency_matrix)
+
+
+class StGcnBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, dropout, stride=1):
+        super().__init__()
+        self.graph_convolution = GraphConvolution(input_channels, output_channels)
+        self.temporal_convolution = nn.Sequential(
+            nn.BatchNorm2d(output_channels),
+            nn.ReLU(inplace=True),
+            nn.Conv2d(output_channels, output_channels, kernel_size=(9, 1), stride=(stride, 1), padding=(4, 0)),
+            nn.BatchNorm2d(output_channels),
+            nn.Dropout(dropout),
+        )
+        if stride != 1 or input_channels != output_channels:
+            self.residual = nn.Sequential(nn.Conv2d(input_channels, output_channels, kernel_size=1, stride=(stride, 1)), nn.BatchNorm2d(output_channels))
+        else:
+            self.residual = nn.Identity()
+        self.activation = nn.ReLU(inplace=True)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        residual_tensor = self.residual(input_tensor)
+        output_tensor = self.graph_convolution(input_tensor, adjacency_matrix)
+        output_tensor = self.temporal_convolution(output_tensor)
+        return self.activation(output_tensor + residual_tensor)
+
+
+class StGcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, dropout):
+        super().__init__()
+        self.input_batch_norm = nn.BatchNorm1d(feature_count)
+        self.register_parameter("adjacency_logits", nn.Parameter(torch.eye(feature_count)))
+        self.block1 = StGcnBlock(1, 64, dropout=dropout, stride=1)
+        self.block2 = StGcnBlock(64, 64, dropout=dropout, stride=1)
+        self.block3 = StGcnBlock(64, 128, dropout=dropout, stride=1)
+        self.classifier = nn.Linear(128, class_count)
+
+    def get_normalized_adjacency(self):
+        return torch.softmax(self.adjacency_logits, dim=1)
+
+    def forward(self, input_sequence):
+        batch_size, sequence_length, feature_count = input_sequence.shape
+        normalized_input = input_sequence.reshape(batch_size * sequence_length, feature_count)
+        normalized_input = self.input_batch_norm(normalized_input)
+        normalized_input = normalized_input.reshape(batch_size, sequence_length, feature_count)
+        graph_tensor = normalized_input.unsqueeze(1)
+        adjacency_matrix = self.get_normalized_adjacency()
+        graph_tensor = self.block1(graph_tensor, adjacency_matrix)
+        graph_tensor = self.block2(graph_tensor, adjacency_matrix)
+        graph_tensor = self.block3(graph_tensor, adjacency_matrix)
+        pooled_tensor = graph_tensor.mean(dim=2).mean(dim=2)
+        return self.classifier(pooled_tensor)
+
+
+MODEL_SPECS = {
+    "bilstm": {
+        "weight": "bidirectionallstm_model.pt",
+        "scaler": "bidirectionallstm_scaler.pkl",
+        "encoder": "bidirectionallstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: BidirectionalLstmClassifier(feature_count, 73, class_count, 0.2174),
+    },
+    "lstm": {
+        "weight": "lstm_model.pt",
+        "scaler": "lstm_scaler.pkl",
+        "encoder": "lstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: LstmClassifier(feature_count, 117, class_count, 0.3829),
+    },
+    "gru": {
+        "weight": "gru_model.pt",
+        "scaler": "gru_scaler.pkl",
+        "encoder": "gru_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: GruClassifier(feature_count, 96, class_count, 0.2),
+    },
+    "tcn": {
+        "weight": "tcn_model.pt",
+        "scaler": "tcn_scaler.pkl",
+        "encoder": "tcn_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: TcnClassifier(feature_count, class_count, 128, 3, 0.2),
+    },
+    "cnn_bilstm": {
+        "weight": "cnn_bilstm_model.pt",
+        "scaler": "cnn_bilstm_scaler.pkl",
+        "encoder": "cnn_bilstm_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: CnnBiLstmClassifier(feature_count, class_count, 128, 3, 73, 0.2),
+    },
+    "st_gcn": {
+        "weight": "st_gcn_model.pt",
+        "scaler": "st_gcn_scaler.pkl",
+        "encoder": "st_gcn_label_encoder.pkl",
+        "builder": lambda feature_count, class_count: StGcnClassifier(feature_count, class_count, 0.2),
+    },
+}
+
+
+def load_test_table(test_file_path):
+    table = pd.read_csv(test_file_path)
+    metadata_columns = {"video_id", "exercise_label", "start_frame_index", "end_frame_index"}
+    feature_columns = [column_name for column_name in table.columns if column_name not in metadata_columns]
+    features = table[feature_columns].to_numpy(dtype=np.float32)
+    labels = table["exercise_label"].to_numpy()
+    return features, labels
+
+
+def build_loader(features, labels, batch_size):
+    feature_tensor = torch.tensor(features, dtype=torch.float32)
+    label_tensor = torch.tensor(labels, dtype=torch.long)
+    dataset = SequenceDataset(feature_tensor, label_tensor)
+    return DataLoader(dataset, batch_size=batch_size, shuffle=False, num_workers=0, pin_memory=True)
+
+
+def predict_labels(model, data_loader, device):
+    model.eval()
+    true_labels = []
+    predicted_labels = []
+    with torch.inference_mode():
+        for feature_batch, label_batch in data_loader:
+            feature_batch = feature_batch.to(device, non_blocking=True)
+            logits = model(feature_batch)
+            prediction = torch.argmax(logits, dim=1)
+            true_labels.append(label_batch.numpy())
+            predicted_labels.append(prediction.cpu().numpy())
+    return np.concatenate(true_labels), np.concatenate(predicted_labels)
+
+
+def evaluate_one_model(model_name, spec, models_root_path, test_features, test_raw_labels, sequence_length, feature_count, batch_size, device, output_directory_path):
+    model_root = models_root_path / model_name
+    weights_root = model_root / "weights"
+    results_root = model_root / "results"
+
+    scaler = joblib.load(weights_root / spec["scaler"])
+    label_encoder = joblib.load(weights_root / spec["encoder"])
+
+    scaled_test = scaler.transform(test_features).reshape(-1, sequence_length, feature_count)
+    encoded_labels = label_encoder.transform(test_raw_labels)
+    data_loader = build_loader(scaled_test, encoded_labels, batch_size)
+
+    class_count = len(label_encoder.classes_)
+    model = spec["builder"](feature_count, class_count).to(device)
+    state_dict = torch.load(weights_root / spec["weight"], map_location=device)
+    model.load_state_dict(state_dict)
+
+    true_labels, predicted_labels = predict_labels(model, data_loader, device)
+
+    accuracy = accuracy_score(true_labels, predicted_labels)
+    precision = precision_score(true_labels, predicted_labels, average="weighted", zero_division=0)
+    recall = recall_score(true_labels, predicted_labels, average="weighted", zero_division=0)
+    f1_weighted = f1_score(true_labels, predicted_labels, average="weighted", zero_division=0)
+    f1_macro = f1_score(true_labels, predicted_labels, average="macro", zero_division=0)
+    matrix = confusion_matrix(true_labels, predicted_labels)
+    report_text = classification_report(true_labels, predicted_labels, target_names=label_encoder.classes_, zero_division=0)
+
+    metrics = {
+        "model": model_name,
+        "accuracy": float(accuracy),
+        "precision_weighted": float(precision),
+        "recall_weighted": float(recall),
+        "f1_weighted": float(f1_weighted),
+        "f1_macro": float(f1_macro),
+        "classes": list(label_encoder.classes_),
+        "confusion_matrix": matrix.tolist(),
+        "classification_report_text": report_text,
+    }
+
+    metrics_row = {
+        "model": model_name,
+        "accuracy": metrics["accuracy"],
+        "precision_weighted": metrics["precision_weighted"],
+        "recall_weighted": metrics["recall_weighted"],
+        "f1_weighted": metrics["f1_weighted"],
+        "f1_macro": metrics["f1_macro"],
+    }
+    output_metrics_path = output_directory_path / f"{model_name}_home_metrics.csv"
+    pd.DataFrame([metrics_row]).to_csv(output_metrics_path, index=False)
+    pd.DataFrame(matrix).to_csv(results_root / "home_confusion_matrix.csv", index=False)
+
+    return metrics
+
+
+def main():
+    args = parse_args()
+
+    test_file_path = Path(args.test_file)
+    models_root_path = Path(args.models_root)
+    output_directory_path = Path(args.output_dir)
+    output_directory_path.mkdir(parents=True, exist_ok=True)
+
+    sequence_length = args.sequence_length
+    feature_count = args.feature_count
+    batch_size = args.batch_size
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+    test_features, test_raw_labels = load_test_table(test_file_path)
+
+    all_metrics = []
+    for model_name, spec in MODEL_SPECS.items():
+        print(f"Evaluating: {model_name}")
+        metrics = evaluate_one_model(
+            model_name=model_name,
+            spec=spec,
+            models_root_path=models_root_path,
+            test_features=test_features,
+            test_raw_labels=test_raw_labels,
+            sequence_length=sequence_length,
+            feature_count=feature_count,
+            batch_size=batch_size,
+            device=device,
+            output_directory_path=output_directory_path,
+        )
+        all_metrics.append(metrics)
+
+    leaderboard_table = pd.DataFrame(
+        [
+            {
+                "model": metric["model"],
+                "accuracy": metric["accuracy"],
+                "f1_macro": metric["f1_macro"],
+                "f1_weighted": metric["f1_weighted"],
+                "precision_weighted": metric["precision_weighted"],
+                "recall_weighted": metric["recall_weighted"],
+            }
+            for metric in all_metrics
+        ]
+    ).sort_values("accuracy", ascending=False)
+
+    leaderboard_csv_path = output_directory_path / "home_leaderboard.csv"
+    leaderboard_table.to_csv(leaderboard_csv_path, index=False)
+
+    print("\nHome leaderboard")
+    print(leaderboard_table.to_string(index=False))
+    print(f"\nSaved: {leaderboard_csv_path}")
+
+
+if __name__ == "__main__":
+    main()

scripts/evaluate/rep_counting_methods.py

@@ -0,0 +1,148 @@
+from dataclasses import dataclass
+from typing import Dict, List
+
+import numpy as np
+
+
+@dataclass
+class ExerciseAngleConfig:
+    primary_triplet_left: tuple[str, str, str]
+    primary_triplet_right: tuple[str, str, str]
+    fixed_low: float
+    fixed_high: float
+    min_state_frames: int = 2
+    smoothing_window: int = 5
+
+
+EXERCISE_CONFIGS: Dict[str, ExerciseAngleConfig] = {
+    "squat": ExerciseAngleConfig(
+        primary_triplet_left=("LEFT_HIP", "LEFT_KNEE", "LEFT_ANKLE"),
+        primary_triplet_right=("RIGHT_HIP", "RIGHT_KNEE", "RIGHT_ANKLE"),
+        fixed_low=95.0,
+        fixed_high=160.0,
+    ),
+    "push up": ExerciseAngleConfig(
+        primary_triplet_left=("LEFT_SHOULDER", "LEFT_ELBOW", "LEFT_WRIST"),
+        primary_triplet_right=("RIGHT_SHOULDER", "RIGHT_ELBOW", "RIGHT_WRIST"),
+        fixed_low=95.0,
+        fixed_high=155.0,
+    ),
+    "barbell biceps curl": ExerciseAngleConfig(
+        primary_triplet_left=("LEFT_SHOULDER", "LEFT_ELBOW", "LEFT_WRIST"),
+        primary_triplet_right=("RIGHT_SHOULDER", "RIGHT_ELBOW", "RIGHT_WRIST"),
+        fixed_low=55.0,
+        fixed_high=145.0,
+    ),
+    "shoulder press": ExerciseAngleConfig(
+        primary_triplet_left=("LEFT_SHOULDER", "LEFT_ELBOW", "LEFT_WRIST"),
+        primary_triplet_right=("RIGHT_SHOULDER", "RIGHT_ELBOW", "RIGHT_WRIST"),
+        fixed_low=70.0,
+        fixed_high=155.0,
+    ),
+}
+
+
+EXERCISE_ALIASES = {
+    "push-up": "push up",
+    "pushups": "push up",
+    "pushup": "push up",
+    "curls": "barbell biceps curl",
+    "bicep curl": "barbell biceps curl",
+    "biceps curl": "barbell biceps curl",
+    "shoulder_press": "shoulder press",
+}
+
+
+def normalize_exercise_name(exercise_name: str) -> str:
+    key = exercise_name.strip().lower()
+    return EXERCISE_ALIASES.get(key, key)
+
+
+def calculate_angle_degrees(point_a: np.ndarray, point_b: np.ndarray, point_c: np.ndarray) -> float:
+    if np.allclose(point_a, 0.0) or np.allclose(point_b, 0.0) or np.allclose(point_c, 0.0):
+        return np.nan
+    vector_ab = point_a[:2] - point_b[:2]
+    vector_cb = point_c[:2] - point_b[:2]
+    denominator = np.linalg.norm(vector_ab) * np.linalg.norm(vector_cb)
+    if denominator == 0.0:
+        return np.nan
+    cosine_value = np.clip(np.dot(vector_ab, vector_cb) / denominator, -1.0, 1.0)
+    return float(np.degrees(np.arccos(cosine_value)))
+
+
+def extract_primary_angle(landmarks: Dict[str, np.ndarray], config: ExerciseAngleConfig) -> float:
+    left_angle = calculate_angle_degrees(
+        landmarks[config.primary_triplet_left[0]],
+        landmarks[config.primary_triplet_left[1]],
+        landmarks[config.primary_triplet_left[2]],
+    )
+    right_angle = calculate_angle_degrees(
+        landmarks[config.primary_triplet_right[0]],
+        landmarks[config.primary_triplet_right[1]],
+        landmarks[config.primary_triplet_right[2]],
+    )
+    if np.isnan(left_angle) and np.isnan(right_angle):
+        return np.nan
+    if np.isnan(left_angle):
+        return right_angle
+    if np.isnan(right_angle):
+        return left_angle
+    return float((left_angle + right_angle) / 2.0)
+
+
+class SmoothingBuffer:
+    def __init__(self, window_size: int):
+        self.window_size = window_size
+        self.values: List[float] = []
+
+    def update(self, value: float) -> float:
+        if np.isnan(value):
+            return np.nan
+        self.values.append(value)
+        if len(self.values) > self.window_size:
+            self.values.pop(0)
+        return float(np.mean(self.values))
+
+
+class FixedThresholdFSMCounter:
+    def __init__(self, low_threshold: float, high_threshold: float, min_state_frames: int = 2):
+        self.low_threshold = low_threshold
+        self.high_threshold = high_threshold
+        self.min_state_frames = min_state_frames
+        self.reps = 0
+        self.current_state = "unknown"
+        self.pending_state = "unknown"
+        self.pending_state_frames = 0
+
+    def _angle_state(self, angle: float) -> str:
+        if angle <= self.low_threshold:
+            return "flexed"
+        if angle >= self.high_threshold:
+            return "extended"
+        return "mid"
+
+    def update(self, angle: float) -> int:
+        if np.isnan(angle):
+            return self.reps
+
+        next_state = self._angle_state(angle)
+        if next_state == "mid":
+            self.pending_state = "unknown"
+            self.pending_state_frames = 0
+            return self.reps
+
+        if next_state == self.pending_state:
+            self.pending_state_frames += 1
+        else:
+            self.pending_state = next_state
+            self.pending_state_frames = 1
+
+        if self.pending_state_frames < self.min_state_frames:
+            return self.reps
+
+        if self.current_state != self.pending_state:
+            previous_state = self.current_state
+            self.current_state = self.pending_state
+            if previous_state == "flexed" and self.current_state == "extended":
+                self.reps += 1
+        return self.reps

scripts/preprocess/build_similarity_assets.py

@@ -0,0 +1,116 @@
+import argparse
+from pathlib import Path
+
+import joblib
+import numpy as np
+import pandas as pd
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--train-file", default="data/train_sequences.csv")
+    parser.add_argument("--models-root", default="models")
+    parser.add_argument("--supported-models", nargs="+", default=["bilstm", "lstm", "gru", "tcn", "cnn_bilstm", "st_gcn"])
+    parser.add_argument("--output-filename", default="similarity_centroids.pkl")
+    return parser.parse_args()
+
+
+MODEL_FILES = {
+    "bilstm": {"scaler": "bidirectionallstm_scaler.pkl", "encoder": "bidirectionallstm_label_encoder.pkl"},
+    "lstm": {"scaler": "lstm_scaler.pkl", "encoder": "lstm_label_encoder.pkl"},
+    "tcn": {"scaler": "tcn_scaler.pkl", "encoder": "tcn_label_encoder.pkl"},
+    "gru": {"scaler": "gru_scaler.pkl", "encoder": "gru_label_encoder.pkl"},
+    "cnn_bilstm": {"scaler": "cnn_bilstm_scaler.pkl", "encoder": "cnn_bilstm_label_encoder.pkl"},
+    "st_gcn": {"scaler": "st_gcn_scaler.pkl", "encoder": "st_gcn_label_encoder.pkl"},
+}
+
+
+def load_train_table(train_file_path):
+    train_table = pd.read_csv(train_file_path)
+    metadata_columns = {"video_id", "exercise_label", "start_frame_index", "end_frame_index"}
+    feature_columns = [column_name for column_name in train_table.columns if column_name not in metadata_columns]
+    feature_matrix = train_table[feature_columns].to_numpy(dtype=np.float32)
+    label_array = train_table["exercise_label"].to_numpy()
+    return feature_matrix, label_array
+
+
+def compute_normalized_centroid(vectors):
+    centroid = np.mean(vectors, axis=0)
+    centroid_norm = np.linalg.norm(centroid)
+    if centroid_norm == 0.0:
+        return centroid
+    return centroid / centroid_norm
+
+
+def build_model_centroids(model_name, model_files, models_root_path, train_features, train_labels):
+    weights_dir = models_root_path / model_name / "weights"
+    scaler = joblib.load(weights_dir / model_files["scaler"])
+    label_encoder = joblib.load(weights_dir / model_files["encoder"])
+
+    scaled_features = scaler.transform(train_features)
+    class_names = list(label_encoder.classes_)
+
+    centroids = {}
+    sample_counts = {}
+    for class_name in class_names:
+        class_mask = train_labels == class_name
+        class_vectors = scaled_features[class_mask]
+        class_vectors = class_vectors / np.clip(np.linalg.norm(class_vectors, axis=1, keepdims=True), 1e-8, None)
+        centroids[class_name] = compute_normalized_centroid(class_vectors)
+        sample_counts[class_name] = int(class_vectors.shape[0])
+
+    return {
+        "model": model_name,
+        "similarity_method": "cosine",
+        "class_order": class_names,
+        "sample_counts": sample_counts,
+        "centroids": centroids,
+    }
+
+
+def save_model_asset(weights_dir, output_filename, similarity_asset):
+    output_path = weights_dir / output_filename
+    joblib.dump(similarity_asset, output_path)
+    return output_path
+
+
+def save_supported_models_manifest(models_root_path, supported_models):
+    manifest_path = models_root_path / "similarity_supported_models.csv"
+    pd.DataFrame([{"model": model_name, "method": "cosine", "version": "v1_centroid"} for model_name in supported_models]).to_csv(manifest_path, index=False)
+    return manifest_path
+
+
+def main():
+    args = parse_args()
+
+    train_file_path = Path(args.train_file)
+    models_root_path = Path(args.models_root)
+    supported_models = args.supported_models
+    output_filename = args.output_filename
+
+    train_features, train_labels = load_train_table(train_file_path)
+
+    for model_name in supported_models:
+        if model_name not in MODEL_FILES:
+            print(f"Skipping unsupported model key: {model_name}")
+            continue
+
+        model_files = MODEL_FILES[model_name]
+        similarity_asset = build_model_centroids(
+            model_name=model_name,
+            model_files=model_files,
+            models_root_path=models_root_path,
+            train_features=train_features,
+            train_labels=train_labels,
+        )
+
+        weights_dir = models_root_path / model_name / "weights"
+        output_path = save_model_asset(weights_dir, output_filename, similarity_asset)
+        print(f"Saved: {output_path}")
+
+    manifest_path = save_supported_models_manifest(models_root_path, supported_models)
+    print(f"Saved: {manifest_path}")
+
+
+if __name__ == "__main__":
+    main()

scripts/preprocess/create_fixed_splits.py

@@ -0,0 +1,127 @@
+import argparse
+from pathlib import Path
+
+import pandas as pd
+from sklearn.model_selection import train_test_split
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input-file", default="data/train_sequences_full.csv")
+    parser.add_argument("--output-dir", default="data")
+    parser.add_argument("--train-ratio", type=float, default=0.70)
+    parser.add_argument("--val-ratio", type=float, default=0.15)
+    parser.add_argument("--test-ratio", type=float, default=0.15)
+    parser.add_argument("--seed", type=int, default=42)
+    return parser.parse_args()
+
+
+def validate_split_ratios(train_ratio, val_ratio, test_ratio):
+    ratio_sum = train_ratio + val_ratio + test_ratio
+    if abs(ratio_sum - 1.0) > 1e-8:
+        raise ValueError(f"Split ratios must sum to 1.0. Got {ratio_sum:.6f}")
+
+
+def build_split_tables(sequence_table, train_ratio, val_ratio, test_ratio, seed):
+    labels = sequence_table["exercise_label"]
+
+    train_table, holdout_table = train_test_split(
+        sequence_table,
+        test_size=(1.0 - train_ratio),
+        random_state=seed,
+        stratify=labels,
+    )
+
+    holdout_labels = holdout_table["exercise_label"]
+    val_fraction_of_holdout = val_ratio / (val_ratio + test_ratio)
+    val_table, test_table = train_test_split(
+        holdout_table,
+        test_size=(1.0 - val_fraction_of_holdout),
+        random_state=seed,
+        stratify=holdout_labels,
+    )
+
+    return train_table, val_table, test_table
+
+
+def save_split_table(split_table, output_file_path):
+    split_table.to_csv(output_file_path, index=False)
+
+
+def build_split_info(train_table, val_table, test_table, args):
+    split_info = {
+        "input_file": str(Path(args.input_file)),
+        "seed": args.seed,
+        "train_ratio": args.train_ratio,
+        "val_ratio": args.val_ratio,
+        "test_ratio": args.test_ratio,
+        "train_samples": int(len(train_table)),
+        "val_samples": int(len(val_table)),
+        "test_internal_samples": int(len(test_table)),
+        "class_counts": {
+            "train": train_table["exercise_label"].value_counts().to_dict(),
+            "val": val_table["exercise_label"].value_counts().to_dict(),
+            "test_internal": test_table["exercise_label"].value_counts().to_dict(),
+        },
+    }
+    return split_info
+
+
+def save_split_info(split_info, output_file_path):
+    info_rows = [
+        {"key": "input_file", "value": split_info["input_file"]},
+        {"key": "seed", "value": split_info["seed"]},
+        {"key": "train_ratio", "value": split_info["train_ratio"]},
+        {"key": "val_ratio", "value": split_info["val_ratio"]},
+        {"key": "test_ratio", "value": split_info["test_ratio"]},
+        {"key": "train_samples", "value": split_info["train_samples"]},
+        {"key": "val_samples", "value": split_info["val_samples"]},
+        {"key": "test_internal_samples", "value": split_info["test_internal_samples"]},
+    ]
+    for split_name, class_counts in split_info["class_counts"].items():
+        for class_name, class_count in class_counts.items():
+            info_rows.append({"key": f"class_counts.{split_name}.{class_name}", "value": class_count})
+    pd.DataFrame(info_rows).to_csv(output_file_path, index=False)
+
+
+def main():
+    args = parse_args()
+
+    validate_split_ratios(args.train_ratio, args.val_ratio, args.test_ratio)
+
+    input_file_path = Path(args.input_file)
+    output_directory_path = Path(args.output_dir)
+    output_directory_path.mkdir(parents=True, exist_ok=True)
+
+    sequence_table = pd.read_csv(input_file_path)
+    if "exercise_label" not in sequence_table.columns:
+        raise ValueError("Input file must include an 'exercise_label' column.")
+
+    train_table, val_table, test_table = build_split_tables(
+        sequence_table=sequence_table,
+        train_ratio=args.train_ratio,
+        val_ratio=args.val_ratio,
+        test_ratio=args.test_ratio,
+        seed=args.seed,
+    )
+
+    train_output_path = output_directory_path / "train_sequences.csv"
+    val_output_path = output_directory_path / "val_sequences.csv"
+    test_output_path = output_directory_path / "test_internal_sequences.csv"
+    info_output_path = output_directory_path / "split_info.csv"
+
+    save_split_table(train_table, train_output_path)
+    save_split_table(val_table, val_output_path)
+    save_split_table(test_table, test_output_path)
+
+    split_info = build_split_info(train_table, val_table, test_table, args)
+    save_split_info(split_info, info_output_path)
+
+    print(f"Saved: {train_output_path} ({len(train_table)} rows)")
+    print(f"Saved: {val_output_path} ({len(val_table)} rows)")
+    print(f"Saved: {test_output_path} ({len(test_table)} rows)")
+    print(f"Saved: {info_output_path}")
+
+
+if __name__ == "__main__":
+    main()

scripts/preprocess/create_sequence_of_features.py

@@ -0,0 +1,132 @@
+import argparse
+from pathlib import Path
+
+import numpy as np
+import pandas as pd
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input-dir", default="data/interim")
+    parser.add_argument("--input-pattern", default="*_frame_features.csv")
+    parser.add_argument("--output-file", default="data/train_sequences_full.csv")
+    parser.add_argument("--sequence-length", type=int, default=30)
+    parser.add_argument("--stride", type=int, default=30)
+    return parser.parse_args()
+
+# Load all per-frame feature tables from the specified directory and pattern
+def load_frame_feature_tables(input_directory_path, input_pattern):
+    frame_feature_paths = sorted(input_directory_path.glob(input_pattern))
+    if not frame_feature_paths:
+        raise FileNotFoundError(f"No files found in {input_directory_path} matching {input_pattern}")
+
+    dataframes = []
+    for frame_feature_path in frame_feature_paths:
+        dataframe = pd.read_csv(frame_feature_path)
+        dataframes.append(dataframe)
+    return dataframes
+
+# Extract the names of the feature columns, excluding metadata columns
+def get_frame_feature_column_names(frame_feature_table):
+    excluded_columns = {"video_id", "exercise_label", "frame_index"}
+    frame_feature_column_names = [
+        column_name for column_name in frame_feature_table.columns if column_name not in excluded_columns
+    ]
+    return frame_feature_column_names
+
+# Build a list of flattened feature names for the sequence table based on the frame feature column names and sequence length
+def build_flattened_sequence_feature_names(sequence_length, frame_feature_column_names):
+    flattened_feature_names = []
+    for timestep_index in range(sequence_length):
+        for feature_name in frame_feature_column_names:
+            flattened_feature_names.append(f"t{timestep_index:02d}_{feature_name}")
+    return flattened_feature_names
+
+# Create fixed-length sequences of frame features for a single video, returning a list of sequence rows with metadata and flattened features
+def create_sequences_from_video_table(video_table, frame_feature_column_names, sequence_length, stride):
+    sequence_rows = []
+    sorted_video_table = video_table.sort_values("frame_index")
+    total_frames = len(sorted_video_table)
+    max_start_index = total_frames - sequence_length
+
+    if max_start_index < 0:
+        return sequence_rows
+
+    for start_index in range(0, max_start_index + 1, stride):
+        end_index = start_index + sequence_length
+        sequence_slice = sorted_video_table.iloc[start_index:end_index]
+        sequence_label = sequence_slice.iloc[0]["exercise_label"]
+        sequence_video_id = sequence_slice.iloc[0]["video_id"]
+        sequence_feature_matrix = sequence_slice[frame_feature_column_names].to_numpy(dtype=np.float32)
+
+        sequence_rows.append(
+            {
+                "video_id": sequence_video_id,
+                "exercise_label": sequence_label,
+                "start_frame_index": int(sequence_slice.iloc[0]["frame_index"]),
+                "end_frame_index": int(sequence_slice.iloc[-1]["frame_index"]),
+                "flattened_features": sequence_feature_matrix.reshape(-1),
+            }
+        )
+
+    return sequence_rows
+
+# Convert the list of per-frame feature tables into a single sequence table with flattened features for each sequence, including metadata columns for video ID, exercise label, and frame indices
+def convert_frame_tables_to_sequence_table(frame_feature_tables, sequence_length, stride):
+    merged_frame_feature_table = pd.concat(frame_feature_tables, ignore_index=True)
+    frame_feature_column_names = get_frame_feature_column_names(merged_frame_feature_table)
+    flattened_feature_names = build_flattened_sequence_feature_names(sequence_length, frame_feature_column_names)
+
+    all_sequence_rows = []
+    grouped_video_tables = merged_frame_feature_table.groupby("video_id", sort=False)
+
+    for _, video_table in grouped_video_tables:
+        video_sequences = create_sequences_from_video_table(
+            video_table=video_table,
+            frame_feature_column_names=frame_feature_column_names,
+            sequence_length=sequence_length,
+            stride=stride,
+        )
+        all_sequence_rows.extend(video_sequences)
+
+    sequence_table_rows = []
+    for sequence_row in all_sequence_rows:
+        flat_feature_values = sequence_row["flattened_features"]
+        flattened_feature_dict = dict(zip(flattened_feature_names, flat_feature_values))
+
+        output_row = {
+            "video_id": sequence_row["video_id"],
+            "exercise_label": sequence_row["exercise_label"],
+            "start_frame_index": sequence_row["start_frame_index"],
+            "end_frame_index": sequence_row["end_frame_index"],
+        }
+        output_row.update(flattened_feature_dict)
+        sequence_table_rows.append(output_row)
+
+    return pd.DataFrame(sequence_table_rows)
+
+# Save the final sequence table to a CSV file
+def save_sequence_table(sequence_table, output_file_path):
+    output_file_path.parent.mkdir(parents=True, exist_ok=True)
+    sequence_table.to_csv(output_file_path, index=False)
+
+# Loads per-frame feature tables, converts them into fixed-length sequences with flattened features, and saves the resulting sequence table to a CSV file
+def main():
+    args = parse_args()
+
+    input_directory_path = Path(args.input_dir)
+    input_pattern = args.input_pattern
+    output_file_path = Path(args.output_file)
+    sequence_length = args.sequence_length
+    stride = args.stride
+
+    frame_feature_tables = load_frame_feature_tables(input_directory_path, input_pattern)
+    sequence_table = convert_frame_tables_to_sequence_table(frame_feature_tables, sequence_length, stride)
+    save_sequence_table(sequence_table, output_file_path)
+
+    print(f"Saved: {output_file_path}")
+    print(f"Sequences: {len(sequence_table)}")
+
+
+if __name__ == "__main__":
+    main()

scripts/preprocess/extract_features.py

@@ -0,0 +1,301 @@
+import argparse
+from pathlib import Path
+
+import cv2
+import mediapipe as mp
+import numpy as np
+import pandas as pd
+from tqdm import tqdm
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--dataset-root", default="data/raw/real-time-exercise-recognition-dataset")
+    parser.add_argument("--input-datasets", nargs="+", default=["final_kaggle_with_additional_video", "synthetic_dataset/synthetic_dataset", "similar_dataset"])
+    parser.add_argument("--output-dir", default="data/interim")
+    parser.add_argument("--min-visibility", type=float, default=0.5)
+    return parser.parse_args()
+
+# Build a mapping of landmark names to their corresponding MediaPipe indices
+def build_landmark_indices(mp_pose):
+    return {
+        "LEFT_SHOULDER": mp_pose.PoseLandmark.LEFT_SHOULDER.value,
+        "RIGHT_SHOULDER": mp_pose.PoseLandmark.RIGHT_SHOULDER.value,
+        "LEFT_HIP": mp_pose.PoseLandmark.LEFT_HIP.value,
+        "RIGHT_HIP": mp_pose.PoseLandmark.RIGHT_HIP.value,
+        "LEFT_KNEE": mp_pose.PoseLandmark.LEFT_KNEE.value,
+        "RIGHT_KNEE": mp_pose.PoseLandmark.RIGHT_KNEE.value,
+        "LEFT_ELBOW": mp_pose.PoseLandmark.LEFT_ELBOW.value,
+        "RIGHT_ELBOW": mp_pose.PoseLandmark.RIGHT_ELBOW.value,
+        "LEFT_WRIST": mp_pose.PoseLandmark.LEFT_WRIST.value,
+        "RIGHT_WRIST": mp_pose.PoseLandmark.RIGHT_WRIST.value,
+        "LEFT_ANKLE": mp_pose.PoseLandmark.LEFT_ANKLE.value,
+        "RIGHT_ANKLE": mp_pose.PoseLandmark.RIGHT_ANKLE.value,
+        "LEFT_HEEL": mp_pose.PoseLandmark.LEFT_HEEL.value,
+        "RIGHT_HEEL": mp_pose.PoseLandmark.RIGHT_HEEL.value,
+        "LEFT_FOOT_INDEX": mp_pose.PoseLandmark.LEFT_FOOT_INDEX.value,
+        "RIGHT_FOOT_INDEX": mp_pose.PoseLandmark.RIGHT_FOOT_INDEX.value,
+        "LEFT_PINKY": mp_pose.PoseLandmark.LEFT_PINKY.value,
+        "RIGHT_PINKY": mp_pose.PoseLandmark.RIGHT_PINKY.value,
+        "LEFT_INDEX": mp_pose.PoseLandmark.LEFT_INDEX.value,
+        "RIGHT_INDEX": mp_pose.PoseLandmark.RIGHT_INDEX.value,
+        "LEFT_THUMB": mp_pose.PoseLandmark.LEFT_THUMB.value,
+        "RIGHT_THUMB": mp_pose.PoseLandmark.RIGHT_THUMB.value,
+    }
+
+# Define the list of landmark names we want to extract coordinates for
+def get_coordinate_landmark_names():
+    return [
+        "LEFT_SHOULDER",
+        "RIGHT_SHOULDER",
+        "LEFT_HIP",
+        "RIGHT_HIP",
+        "LEFT_KNEE",
+        "RIGHT_KNEE",
+        "LEFT_ELBOW",
+        "RIGHT_ELBOW",
+        "LEFT_WRIST",
+        "RIGHT_WRIST",
+        "LEFT_ANKLE",
+        "RIGHT_ANKLE",
+        "LEFT_HEEL",
+        "RIGHT_HEEL",
+        "LEFT_FOOT_INDEX",
+        "RIGHT_FOOT_INDEX",
+        "LEFT_PINKY",
+        "RIGHT_PINKY",
+        "LEFT_INDEX",
+        "RIGHT_INDEX",
+        "LEFT_THUMB",
+        "RIGHT_THUMB",
+    ]
+
+# Define the triplets of landmarks for which we want to calculate joint angles
+def get_angle_triplets():
+    return [
+        ("LEFT_HIP", "LEFT_SHOULDER", "LEFT_ELBOW"),
+        ("RIGHT_HIP", "RIGHT_SHOULDER", "RIGHT_ELBOW"),
+        ("LEFT_SHOULDER", "LEFT_ELBOW", "LEFT_WRIST"),
+        ("RIGHT_SHOULDER", "RIGHT_ELBOW", "RIGHT_WRIST"),
+        ("LEFT_HIP", "LEFT_KNEE", "LEFT_ANKLE"),
+        ("RIGHT_HIP", "RIGHT_KNEE", "RIGHT_ANKLE"),
+        ("LEFT_SHOULDER", "LEFT_HIP", "LEFT_KNEE"),
+        ("RIGHT_SHOULDER", "RIGHT_HIP", "RIGHT_KNEE"),
+        ("LEFT_KNEE", "LEFT_ANKLE", "LEFT_HEEL"),
+        ("RIGHT_KNEE", "RIGHT_ANKLE", "RIGHT_HEEL"),
+        ("LEFT_ANKLE", "LEFT_HEEL", "LEFT_FOOT_INDEX"),
+        ("RIGHT_ANKLE", "RIGHT_HEEL", "RIGHT_FOOT_INDEX"),
+    ]
+
+# function to recursively list video files in a dataset folder, filtering by video file extensions
+def list_video_files(dataset_path):
+    allowed_suffixes = {".mp4", ".avi", ".mov", ".m4v", ".asf", ".MOV"}
+    video_paths = []
+    for file_path in dataset_path.rglob("*"):
+        if file_path.is_file() and file_path.suffix in allowed_suffixes:
+            video_paths.append(file_path)
+    return sorted(video_paths)
+
+# normalise exercise labels by mapping known variations to a standard label, and lowercasing/stripping whitespace for consistency
+def normalize_exercise_label(raw_label):
+    lower_label = raw_label.strip().lower()
+    label_mapping = {
+        "hammer curl": "barbell biceps curl",
+        "bicept curl": "barbell biceps curl",
+    }
+    return label_mapping.get(lower_label, lower_label)
+
+# Check if a MediaPipe landmark is valid based on its visibility score compared to a minimum threshold
+def is_landmark_valid(landmark, min_visibility):
+    return landmark.visibility >= min_visibility
+
+# Return a placeholder point (0, 0, 0) for missing or invalid landmarks to maintain consistent feature dimensions
+def get_placeholder_point():
+    return np.array([0.0, 0.0, 0.0], dtype=np.float32)
+
+# Calculate the angle in degrees between three points (A, B, C) where B is the vertex point. If any point is invalid (all zeros), return 0 degrees.
+def calculate_angle_degrees(point_a, point_b, point_c):
+    if np.allclose(point_a, 0.0) or np.allclose(point_b, 0.0) or np.allclose(point_c, 0.0):
+        return 0.0
+
+    vector_ab = point_a[:2] - point_b[:2]
+    vector_cb = point_c[:2] - point_b[:2]
+    denominator = np.linalg.norm(vector_ab) * np.linalg.norm(vector_cb)
+
+    if denominator == 0.0:
+        return 0.0
+
+    cosine_value = np.dot(vector_ab, vector_cb) / denominator 
+    cosine_value = np.clip(cosine_value, -1.0, 1.0) # Clip cosine value to the valid range to avoid numerical issues with arccos
+    angle_radians = np.arccos(cosine_value) # Calculate angle in radians and convert to degrees
+    return float(np.degrees(angle_radians))
+
+# Extract the specified landmarks from a MediaPipe pose estimation result for a single frame, checking visibility and using placeholders for missing landmarks. Returns a dictionary of landmark names to their (x, y, z) coordinates.
+def extract_frame_landmarks(media_pipe_results, landmark_indices, coordinate_landmark_names, min_visibility):
+    extracted_landmarks = {}
+    if not media_pipe_results.pose_landmarks:
+        return extracted_landmarks
+
+    for landmark_name in coordinate_landmark_names:
+        landmark_index = landmark_indices[landmark_name]
+        detected_landmark = media_pipe_results.pose_landmarks.landmark[landmark_index]
+
+        if is_landmark_valid(detected_landmark, min_visibility):
+            extracted_landmarks[landmark_name] = np.array(
+                [detected_landmark.x, detected_landmark.y, detected_landmark.z],
+                dtype=np.float32,
+            )
+        else:
+            extracted_landmarks[landmark_name] = get_placeholder_point()
+
+    return extracted_landmarks
+
+# Check if all essential landmarks for a given body side (LEFT or RIGHT) are valid (not all zeros) to determine if we can trust the pose estimation for that side. This helps filter out frames where the pose estimation failed for one side of the body.
+def has_valid_body_side(extracted_landmarks, side_prefix):
+    essential_points = ["SHOULDER", "ELBOW", "WRIST", "HIP", "KNEE", "ANKLE"]
+    for point_name in essential_points:
+        full_landmark_name = f"{side_prefix}_{point_name}"
+        if np.allclose(extracted_landmarks[full_landmark_name], 0.0):
+            return False
+    return True
+
+# Build a feature row dictionary for a single frame, including the video identifier, exercise label, frame index, landmark coordinates, and calculated angles based on the specified triplets. This function combines all the extracted information into a structured format for later saving to CSV.
+def build_feature_row(
+    extracted_landmarks,
+    coordinate_landmark_names,
+    angle_triplets,
+    frame_index,
+    video_identifier,
+    exercise_label,
+):
+    feature_row = {
+        "video_id": video_identifier,
+        "exercise_label": exercise_label,
+        "frame_index": frame_index,
+    }
+
+    for landmark_name in coordinate_landmark_names:
+        landmark_value = extracted_landmarks[landmark_name]
+        feature_row[f"{landmark_name.lower()}_x"] = landmark_value[0]
+        feature_row[f"{landmark_name.lower()}_y"] = landmark_value[1]
+        feature_row[f"{landmark_name.lower()}_z"] = landmark_value[2]
+
+    for point_a, point_b, point_c in angle_triplets:
+        angle_name = f"angle_{point_a.lower()}_{point_b.lower()}_{point_c.lower()}"
+        feature_row[angle_name] = calculate_angle_degrees(
+            extracted_landmarks[point_a],
+            extracted_landmarks[point_b],
+            extracted_landmarks[point_c],
+        )
+
+    return feature_row
+
+# Process a single video file to extract per-frame features. For each frame, it runs pose estimation, extracts landmarks, checks validity, and builds feature rows for valid frames. It returns a list of feature rows for the entire video.
+def extract_features_from_video(
+    video_path,
+    exercise_label,
+    dataset_name,
+    pose_estimator,
+    landmark_indices,
+    coordinate_landmark_names,
+    angle_triplets,
+    min_visibility,
+):
+    video_capture = cv2.VideoCapture(str(video_path))
+    frame_index = 0
+    extracted_rows = []
+    video_identifier = f"{dataset_name}/{exercise_label}/{video_path.stem}"
+
+    while video_capture.isOpened():
+        frame_read_success, frame_bgr = video_capture.read()
+        if not frame_read_success:
+            break
+
+        frame_rgb = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
+        pose_results = pose_estimator.process(frame_rgb)
+        extracted_landmarks = extract_frame_landmarks(
+            pose_results,
+            landmark_indices,
+            coordinate_landmark_names,
+            min_visibility,
+        )
+
+        if extracted_landmarks:
+            left_side_is_valid = has_valid_body_side(extracted_landmarks, "LEFT")
+            right_side_is_valid = has_valid_body_side(extracted_landmarks, "RIGHT")
+
+            if left_side_is_valid or right_side_is_valid:
+                extracted_rows.append(
+                    build_feature_row(
+                        extracted_landmarks=extracted_landmarks,
+                        coordinate_landmark_names=coordinate_landmark_names,
+                        angle_triplets=angle_triplets,
+                        frame_index=frame_index,
+                        video_identifier=video_identifier,
+                        exercise_label=exercise_label,
+                    )
+                )
+
+        frame_index += 1
+
+    video_capture.release()
+    return extracted_rows
+
+# Save the list of feature row dictionaries to a CSV file using pandas, ensuring the output directory exists. Each row in the CSV corresponds to a single frame's extracted features.
+def save_rows_to_csv(rows, output_file_path):
+    output_file_path.parent.mkdir(parents=True, exist_ok=True)
+    dataframe = pd.DataFrame(rows)
+    dataframe.to_csv(output_file_path, index=False)
+
+# Sets up the main execution flow: parses arguments, initializes MediaPipe pose estimator, iterates through specified datasets and videos, extracts features for each video, and saves the results to CSV files in the output directory.
+def main():
+    args = parse_args()
+
+    dataset_root_path = Path(args.dataset_root)
+    input_dataset_names = args.input_datasets
+    output_directory_path = Path(args.output_dir)
+    minimum_landmark_visibility = args.min_visibility
+
+    media_pipe_pose = mp.solutions.pose
+    landmark_indices = build_landmark_indices(media_pipe_pose)
+    coordinate_landmark_names = get_coordinate_landmark_names()
+    angle_triplets = get_angle_triplets()
+
+    with media_pipe_pose.Pose(
+        static_image_mode=False,
+        model_complexity=1,
+        enable_segmentation=False,
+        min_detection_confidence=0.5,
+        min_tracking_confidence=0.5,
+    ) as pose_estimator:
+        for dataset_name in input_dataset_names:
+            dataset_path = dataset_root_path / dataset_name
+            if not dataset_path.exists():
+                print(f"Skipping missing dataset folder: {dataset_path}")
+                continue
+
+            print(f"Processing dataset: {dataset_name}")
+            dataset_rows = []
+            video_paths = list_video_files(dataset_path)
+
+            for video_path in tqdm(video_paths, desc=f"Videos in {dataset_name}"):
+                exercise_label = normalize_exercise_label(video_path.parent.name)
+                video_rows = extract_features_from_video(
+                    video_path=video_path,
+                    exercise_label=exercise_label,
+                    dataset_name=dataset_name,
+                    pose_estimator=pose_estimator,
+                    landmark_indices=landmark_indices,
+                    coordinate_landmark_names=coordinate_landmark_names,
+                    angle_triplets=angle_triplets,
+                    min_visibility=minimum_landmark_visibility,
+                )
+                dataset_rows.extend(video_rows)
+
+            output_file_path = output_directory_path / f"{dataset_name.replace('/', '_')}_frame_features.csv"
+            save_rows_to_csv(dataset_rows, output_file_path)
+            print(f"Saved: {output_file_path} ({len(dataset_rows)} rows)")
+
+
+if __name__ == "__main__":
+    main()

scripts/realtime_eval/evaluate_realtime_webcam.py

@@ -0,0 +1,412 @@
+import argparse
+import sys
+import time
+from pathlib import Path
+
+import cv2
+import joblib
+import mediapipe as mp
+import numpy as np
+import pandas as pd
+import torch
+from torch import nn
+
+PROJECT_ROOT = Path(__file__).resolve().parents[2]
+if str(PROJECT_ROOT) not in sys.path:
+    sys.path.insert(0, str(PROJECT_ROOT))
+
+from scripts.evaluate.rep_counting_methods import EXERCISE_CONFIGS, FixedThresholdFSMCounter, SmoothingBuffer, extract_primary_angle, normalize_exercise_name
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model-name", choices=["bilstm", "lstm", "gru", "tcn", "cnn_bilstm", "st_gcn"], required=True)
+    parser.add_argument("--models-root", default="models")
+    parser.add_argument("--output-dir", default="results/eval_realtime")
+    parser.add_argument("--sequence-length", type=int, default=30)
+    parser.add_argument("--feature-count", type=int, default=78)
+    parser.add_argument("--camera-index", type=int, default=0)
+    parser.add_argument("--run-seconds", type=int, default=75)
+    parser.add_argument("--prediction-interval", type=float, default=1.0)
+    return parser.parse_args()
+
+
+class BidirectionalLstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.bilstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size * 2, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.bilstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        return self.classifier(self.dropout(final_timestep_output))
+
+
+class LstmClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.lstm = nn.LSTM(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.lstm(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        return self.classifier(self.dropout(final_timestep_output))
+
+
+class GruClassifier(nn.Module):
+    def __init__(self, feature_count, hidden_size, class_count, dropout_probability):
+        super().__init__()
+        self.gru = nn.GRU(input_size=feature_count, hidden_size=hidden_size, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=False)
+        self.dropout = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(hidden_size, class_count)
+
+    def forward(self, input_sequence):
+        recurrent_output, _ = self.gru(input_sequence)
+        final_timestep_output = recurrent_output[:, -1, :]
+        return self.classifier(self.dropout(final_timestep_output))
+
+
+class Chomp1d(nn.Module):
+    def __init__(self, chomp_size):
+        super().__init__()
+        self.chomp_size = chomp_size
+
+    def forward(self, input_tensor):
+        if self.chomp_size == 0:
+            return input_tensor
+        return input_tensor[:, :, :-self.chomp_size].contiguous()
+
+
+class TemporalBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, kernel_size, dilation, dropout):
+        super().__init__()
+        padding = (kernel_size - 1) * dilation
+        self.conv1 = nn.Conv1d(input_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp1 = Chomp1d(padding)
+        self.relu1 = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout)
+        self.conv2 = nn.Conv1d(output_channels, output_channels, kernel_size, padding=padding, dilation=dilation)
+        self.chomp2 = Chomp1d(padding)
+        self.relu2 = nn.ReLU()
+        self.dropout2 = nn.Dropout(dropout)
+        self.downsample = nn.Conv1d(input_channels, output_channels, kernel_size=1) if input_channels != output_channels else None
+        self.final_relu = nn.ReLU()
+
+    def forward(self, input_tensor):
+        output_tensor = self.dropout1(self.relu1(self.chomp1(self.conv1(input_tensor))))
+        output_tensor = self.dropout2(self.relu2(self.chomp2(self.conv2(output_tensor))))
+        residual_tensor = input_tensor if self.downsample is None else self.downsample(input_tensor)
+        return self.final_relu(output_tensor + residual_tensor)
+
+
+class TcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, channel_width, kernel_size, dropout):
+        super().__init__()
+        self.input_projection = nn.Conv1d(feature_count, channel_width, kernel_size=1)
+        self.block1 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=1, dropout=dropout)
+        self.block2 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=2, dropout=dropout)
+        self.block3 = TemporalBlock(channel_width, channel_width, kernel_size, dilation=4, dropout=dropout)
+        self.classifier = nn.Linear(channel_width, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = input_sequence.transpose(1, 2)
+        temporal_tensor = self.block3(self.block2(self.block1(self.input_projection(temporal_tensor))))
+        return self.classifier(temporal_tensor[:, :, -1])
+
+
+class CnnBiLstmClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, cnn_filters, cnn_kernel_size, lstm_units, dropout_probability):
+        super().__init__()
+        self.conv1d = nn.Conv1d(feature_count, cnn_filters, kernel_size=cnn_kernel_size, padding=cnn_kernel_size // 2)
+        self.relu = nn.ReLU()
+        self.dropout1 = nn.Dropout(dropout_probability)
+        self.bilstm = nn.LSTM(input_size=cnn_filters, hidden_size=lstm_units, num_layers=2, batch_first=True, dropout=dropout_probability, bidirectional=True)
+        self.dropout2 = nn.Dropout(dropout_probability)
+        self.classifier = nn.Linear(lstm_units * 2, class_count)
+
+    def forward(self, input_sequence):
+        temporal_tensor = self.dropout1(self.relu(self.conv1d(input_sequence.transpose(1, 2)))).transpose(1, 2)
+        recurrent_output, _ = self.bilstm(temporal_tensor)
+        return self.classifier(self.dropout2(recurrent_output[:, -1, :]))
+
+
+class GraphConvolution(nn.Module):
+    def __init__(self, input_channels, output_channels):
+        super().__init__()
+        self.projection = nn.Conv2d(input_channels, output_channels, kernel_size=1)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        return torch.einsum("nctv,vw->nctw", self.projection(input_tensor), adjacency_matrix)
+
+
+class StGcnBlock(nn.Module):
+    def __init__(self, input_channels, output_channels, dropout):
+        super().__init__()
+        self.graph_convolution = GraphConvolution(input_channels, output_channels)
+        self.temporal_convolution = nn.Sequential(
+            nn.BatchNorm2d(output_channels),
+            nn.ReLU(inplace=True),
+            nn.Conv2d(output_channels, output_channels, kernel_size=(9, 1), padding=(4, 0)),
+            nn.BatchNorm2d(output_channels),
+            nn.Dropout(dropout),
+        )
+        self.residual = nn.Sequential(nn.Conv2d(input_channels, output_channels, kernel_size=1), nn.BatchNorm2d(output_channels)) if input_channels != output_channels else nn.Identity()
+        self.activation = nn.ReLU(inplace=True)
+
+    def forward(self, input_tensor, adjacency_matrix):
+        residual_tensor = self.residual(input_tensor)
+        output_tensor = self.temporal_convolution(self.graph_convolution(input_tensor, adjacency_matrix))
+        return self.activation(output_tensor + residual_tensor)
+
+
+class StGcnClassifier(nn.Module):
+    def __init__(self, feature_count, class_count, dropout):
+        super().__init__()
+        self.input_batch_norm = nn.BatchNorm1d(feature_count)
+        self.register_parameter("adjacency_logits", nn.Parameter(torch.eye(feature_count)))
+        self.block1 = StGcnBlock(1, 64, dropout)
+        self.block2 = StGcnBlock(64, 64, dropout)
+        self.block3 = StGcnBlock(64, 128, dropout)
+        self.classifier = nn.Linear(128, class_count)
+
+    def forward(self, input_sequence):
+        batch_size, sequence_length, feature_count = input_sequence.shape
+        normalized = self.input_batch_norm(input_sequence.reshape(batch_size * sequence_length, feature_count)).reshape(batch_size, sequence_length, feature_count)
+        graph_tensor = normalized.unsqueeze(1)
+        adjacency = torch.softmax(self.adjacency_logits, dim=1)
+        graph_tensor = self.block3(self.block2(self.block1(graph_tensor, adjacency), adjacency), adjacency)
+        pooled = graph_tensor.mean(dim=2).mean(dim=2)
+        return self.classifier(pooled)
+
+
+MODEL_SPECS = {
+    "bilstm": {"weight": "bidirectionallstm_model.pt", "scaler": "bidirectionallstm_scaler.pkl", "encoder": "bidirectionallstm_label_encoder.pkl", "builder": lambda f, c: BidirectionalLstmClassifier(f, 73, c, 0.2174)},
+    "lstm": {"weight": "lstm_model.pt", "scaler": "lstm_scaler.pkl", "encoder": "lstm_label_encoder.pkl", "builder": lambda f, c: LstmClassifier(f, 117, c, 0.3829)},
+    "gru": {"weight": "gru_model.pt", "scaler": "gru_scaler.pkl", "encoder": "gru_label_encoder.pkl", "builder": lambda f, c: GruClassifier(f, 96, c, 0.2)},
+    "tcn": {"weight": "tcn_model.pt", "scaler": "tcn_scaler.pkl", "encoder": "tcn_label_encoder.pkl", "builder": lambda f, c: TcnClassifier(f, c, 128, 3, 0.2)},
+    "cnn_bilstm": {"weight": "cnn_bilstm_model.pt", "scaler": "cnn_bilstm_scaler.pkl", "encoder": "cnn_bilstm_label_encoder.pkl", "builder": lambda f, c: CnnBiLstmClassifier(f, c, 128, 3, 73, 0.2)},
+    "st_gcn": {"weight": "st_gcn_model.pt", "scaler": "st_gcn_scaler.pkl", "encoder": "st_gcn_label_encoder.pkl", "builder": lambda f, c: StGcnClassifier(f, c, 0.2)},
+}
+
+
+def load_pose_module():
+    return mp.solutions.pose
+
+
+def build_landmark_indices(mp_pose):
+    names = [
+        "LEFT_SHOULDER", "RIGHT_SHOULDER", "LEFT_HIP", "RIGHT_HIP", "LEFT_KNEE", "RIGHT_KNEE",
+        "LEFT_ELBOW", "RIGHT_ELBOW", "LEFT_WRIST", "RIGHT_WRIST", "LEFT_ANKLE", "RIGHT_ANKLE",
+        "LEFT_HEEL", "RIGHT_HEEL", "LEFT_FOOT_INDEX", "RIGHT_FOOT_INDEX", "LEFT_PINKY", "RIGHT_PINKY",
+        "LEFT_INDEX", "RIGHT_INDEX", "LEFT_THUMB", "RIGHT_THUMB"
+    ]
+    return {name: mp_pose.PoseLandmark[name].value for name in names}
+
+
+def get_angle_triplets():
+    return [
+        ("LEFT_HIP", "LEFT_SHOULDER", "LEFT_ELBOW"), ("RIGHT_HIP", "RIGHT_SHOULDER", "RIGHT_ELBOW"),
+        ("LEFT_SHOULDER", "LEFT_ELBOW", "LEFT_WRIST"), ("RIGHT_SHOULDER", "RIGHT_ELBOW", "RIGHT_WRIST"),
+        ("LEFT_HIP", "LEFT_KNEE", "LEFT_ANKLE"), ("RIGHT_HIP", "RIGHT_KNEE", "RIGHT_ANKLE"),
+        ("LEFT_SHOULDER", "LEFT_HIP", "LEFT_KNEE"), ("RIGHT_SHOULDER", "RIGHT_HIP", "RIGHT_KNEE"),
+        ("LEFT_KNEE", "LEFT_ANKLE", "LEFT_HEEL"), ("RIGHT_KNEE", "RIGHT_ANKLE", "RIGHT_HEEL"),
+        ("LEFT_ANKLE", "LEFT_HEEL", "LEFT_FOOT_INDEX"), ("RIGHT_ANKLE", "RIGHT_HEEL", "RIGHT_FOOT_INDEX"),
+    ]
+
+
+def calculate_angle_degrees(point_a, point_b, point_c):
+    if np.allclose(point_a, 0.0) or np.allclose(point_b, 0.0) or np.allclose(point_c, 0.0):
+        return 0.0
+    vector_ab = point_a[:2] - point_b[:2]
+    vector_cb = point_c[:2] - point_b[:2]
+    denominator = np.linalg.norm(vector_ab) * np.linalg.norm(vector_cb)
+    if denominator == 0.0:
+        return 0.0
+    cosine_value = np.clip(np.dot(vector_ab, vector_cb) / denominator, -1.0, 1.0)
+    return float(np.degrees(np.arccos(cosine_value)))
+
+
+def extract_frame_features(results, landmark_indices, angle_triplets, min_visibility=0.5):
+    if not results.pose_landmarks:
+        return None
+    points = {}
+    for name, idx in landmark_indices.items():
+        lm = results.pose_landmarks.landmark[idx]
+        if lm.visibility >= min_visibility:
+            points[name] = np.array([lm.x, lm.y, lm.z], dtype=np.float32)
+        else:
+            points[name] = np.array([0.0, 0.0, 0.0], dtype=np.float32)
+
+    features = []
+    for name in landmark_indices:
+        point = points[name]
+        features.extend([point[0], point[1], point[2]])
+    for a, b, c in angle_triplets:
+        features.append(calculate_angle_degrees(points[a], points[b], points[c]))
+    return np.array(features, dtype=np.float32)
+
+
+def build_model_and_tools(args, device):
+    model_name = args.model_name
+    spec = MODEL_SPECS[model_name]
+    weights_dir = Path(args.models_root) / model_name / "weights"
+
+    scaler = joblib.load(weights_dir / spec["scaler"])
+    label_encoder = joblib.load(weights_dir / spec["encoder"])
+    class_count = len(label_encoder.classes_)
+
+    model = spec["builder"](args.feature_count, class_count).to(device)
+    model.load_state_dict(torch.load(weights_dir / spec["weight"], map_location=device))
+    model.eval()
+    return model, scaler, label_encoder
+
+
+def main():
+    args = parse_args()
+    output_dir = Path(args.output_dir)
+    output_dir.mkdir(parents=True, exist_ok=True)
+
+    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+    model, scaler, label_encoder = build_model_and_tools(args, device)
+
+    pose_module = load_pose_module()
+    landmark_indices = build_landmark_indices(pose_module)
+    angle_triplets = get_angle_triplets()
+
+    capture = cv2.VideoCapture(args.camera_index)
+    if not capture.isOpened():
+        raise RuntimeError("Could not open webcam.")
+
+    print("Realtime evaluation started.")
+    print("Protocol: 0-20s exercise A, 20-40s exercise B, 40-60s exercise C, 60-75s free.")
+
+    window = []
+    events = []
+    prediction_latencies_ms = []
+    frame_times = []
+    predicted_labels = []
+
+    last_prediction_time = 0.0
+    current_label = "none"
+    rep_counters = {}
+    rep_smoothers = {}
+
+    start_time = time.time()
+    with pose_module.Pose(static_image_mode=False, model_complexity=1, min_detection_confidence=0.5, min_tracking_confidence=0.5) as pose_estimator:
+        drawing_utils = mp.solutions.drawing_utils
+        drawing_spec_points = drawing_utils.DrawingSpec(color=(0, 0, 255), thickness=2, circle_radius=3)
+        drawing_spec_lines = drawing_utils.DrawingSpec(color=(0, 255, 0), thickness=2, circle_radius=1)
+        while True:
+            loop_start = time.time()
+            ok, frame_bgr = capture.read()
+            if not ok:
+                break
+
+            elapsed = time.time() - start_time
+            if elapsed >= args.run_seconds:
+                break
+
+            frame_rgb = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
+            results = pose_estimator.process(frame_rgb)
+
+            if results.pose_landmarks:
+                drawing_utils.draw_landmarks(
+                    frame_bgr,
+                    results.pose_landmarks,
+                    pose_module.POSE_CONNECTIONS,
+                    landmark_drawing_spec=drawing_spec_points,
+                    connection_drawing_spec=drawing_spec_lines,
+                )
+
+            frame_features = extract_frame_features(results, landmark_indices, angle_triplets)
+
+            if frame_features is not None:
+                window.append(frame_features)
+                if len(window) > args.sequence_length:
+                    window.pop(0)
+
+            normalized_label = normalize_exercise_name(current_label)
+            if results.pose_landmarks and normalized_label in EXERCISE_CONFIGS:
+                if normalized_label not in rep_counters:
+                    config = EXERCISE_CONFIGS[normalized_label]
+                    rep_counters[normalized_label] = FixedThresholdFSMCounter(config.fixed_low, config.fixed_high, config.min_state_frames)
+                    rep_smoothers[normalized_label] = SmoothingBuffer(config.smoothing_window)
+                landmarks = {}
+                for name, index in landmark_indices.items():
+                    lm = results.pose_landmarks.landmark[index]
+                    landmarks[name] = np.array([lm.x, lm.y, lm.z], dtype=np.float32) if lm.visibility >= 0.5 else np.array([0.0, 0.0, 0.0], dtype=np.float32)
+                current_config = EXERCISE_CONFIGS[normalized_label]
+                raw_angle = extract_primary_angle(landmarks, current_config)
+                smoothed_angle = rep_smoothers[normalized_label].update(raw_angle)
+                rep_counters[normalized_label].update(smoothed_angle)
+
+            if len(window) == args.sequence_length and (time.time() - last_prediction_time) >= args.prediction_interval:
+                infer_start = time.time()
+                sequence_array = np.array(window, dtype=np.float32).reshape(1, -1)
+                scaled = scaler.transform(sequence_array).reshape(1, args.sequence_length, args.feature_count)
+                input_tensor = torch.tensor(scaled, dtype=torch.float32, device=device)
+
+                with torch.inference_mode():
+                    logits = model(input_tensor)
+                    prediction_index = int(torch.argmax(logits, dim=1).item())
+                    current_label = label_encoder.classes_[prediction_index]
+
+                infer_ms = (time.time() - infer_start) * 1000.0
+                prediction_latencies_ms.append(infer_ms)
+                predicted_labels.append(current_label)
+                events.append({"timestamp_sec": elapsed, "predicted_label": current_label, "latency_ms": infer_ms})
+                last_prediction_time = time.time()
+
+            frame_times.append(time.time() - loop_start)
+
+            cv2.putText(frame_bgr, f"Model: {args.model_name}", (20, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 2)
+            cv2.putText(frame_bgr, f"Pred: {current_label}", (20, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 2)
+            current_reps = rep_counters[normalized_label].reps if normalized_label in rep_counters else 0
+            cv2.putText(frame_bgr, f"Reps: {current_reps}", (20, 90), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 2)
+            cv2.putText(frame_bgr, f"Time: {elapsed:5.1f}s", (20, 120), cv2.FONT_HERSHEY_SIMPLEX, 0.8, (0, 0, 255), 2)
+            cv2.imshow("Realtime Evaluation", frame_bgr)
+
+            if cv2.waitKey(1) & 0xFF == ord("q"):
+                break
+
+    capture.release()
+    cv2.destroyAllWindows()
+
+    fps = 1.0 / np.mean(frame_times) if frame_times else 0.0
+    mean_latency = float(np.mean(prediction_latencies_ms)) if prediction_latencies_ms else None
+    p95_latency = float(np.percentile(prediction_latencies_ms, 95)) if prediction_latencies_ms else None
+
+    flips = 0
+    for index in range(1, len(predicted_labels)):
+        if predicted_labels[index] != predicted_labels[index - 1]:
+            flips += 1
+    flip_rate = float(flips / max(1, len(predicted_labels) - 1))
+
+    summary = {
+        "model": args.model_name,
+        "device": str(device),
+        "run_seconds": args.run_seconds,
+        "prediction_count": len(predicted_labels),
+        "mean_latency_ms": mean_latency,
+        "p95_latency_ms": p95_latency,
+        "pipeline_fps": float(fps),
+        "prediction_flip_rate": flip_rate,
+    }
+
+    summary_path = output_dir / f"{args.model_name}_realtime_metrics.csv"
+    events_path = output_dir / f"{args.model_name}_realtime_events.csv"
+    pd.DataFrame([summary]).to_csv(summary_path, index=False)
+    pd.DataFrame(events).to_csv(events_path, index=False)
+
+    print("Realtime evaluation completed.")
+    print(summary)
+    print(f"Saved: {summary_path}")
+    print(f"Saved: {events_path}")
+
+
+if __name__ == "__main__":
+    main()