File size: 5,588 Bytes
166743f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | # Training and evaluation record
## v1 provenance
- Artifact type: trained XGBoost classifier on engineered pose features — **input is a 56-column pandas DataFrame, not images and not a CSV file at inference**
- Model: `xgboost.XGBClassifier` trained from scratch (no YOLO fine-tuning; YOLO26x-Pose used only upstream to generate the feature CSV for training)
- Upstream pose detector (data generation only, out of scope for classifier): Ultralytics YOLO-Pose (yolo26x-pose.pt), pretrained, batch_size=32, det conf 0.20, kp conf 0.25, PyTorch AMP
- CTSPL training run: Fall Detection v1 (Experiment 01)
- CTSPL fine-tuning run: none
- Training dataset: `pose_benchmark_priority1_dataset.csv` — 6,607 samples after 1:1 class balancing (**58 columns total = 51 pose features + 5 additional + 1 split + 1 label = 58; XGBoost uses only 51 + 5 = 56 classifier features, columns 1–56**)
- Raw image source for CSV generation (upstream): `/home/ctspl/model_training/fall/version3/data` (train/val/test with fall/normal subdirs)
- Training hardware: CPU-only XGBoost (no GPU required for classifier); upstream YOLO extraction used CUDA
- Training date: 2026-09-21
- Selected checkpoint: `models/xgboost_priority1_fall_model.pkl` (**input 56 features → output Fall/Normal**)
```
TRAINING: CCTV IMAGE → YOLO26x-Pose → 51 features → +5 → CSV (58 cols) → DataFrame → XGBoost
INFERENCE: upstream DataFrame (51 + bbox = 55 cols) → run.py: +5 → 56-col DataFrame → XGBoost (no CSV into model)
```
**Inference input contract (what goes IN to `run.py`):**
| Columns | Count | Source |
| --- | ---: | --- |
| `x_i, y_i, conf_i` for i = 0…16 | **51** | Upstream (input contract) |
| `x1, y1, x2, y2` | 4 | Upstream bbox (needed to compute the 5) |
| | **55** | **Total input to `run.py`** |
The **extra 5 features are extracted in `run.py`**, not received. Model input after extraction = **56 = 51 + 5**.
Upstream (data generation only, out of scope for classifier inference):
1. Extract 17 keypoints per detected person using YOLO-Pose (each with `x`, `y`, `confidence` → 51 features)
2. Training CSV has 58 columns total — 51 pose features + 5 additional Priority 1 features + 1 `split` + 1 `label` = **58 columns**
At inference, upstream delivers a **pandas DataFrame** with the 51 keypoint columns + bounding box; `scripts/run.py` computes the 5 Priority 1 features as columns and passes the **56-column DataFrame** to XGBoost.
In `pose_extract.py`: `row = norm_kpts + [aspect_ratio] + p1_features + [split, label]` where `norm_kpts` = 51 and `[aspect_ratio] + p1_features` = 5.
**Very important:** XGBoost classifier does **not** use `split` or `label` as input. It uses only **51 + 5 = 56 classifier features** (columns 1–56). Columns 57 → `split` (`train`/`val`/`test`) and 58 → `label` (`0` = Normal, `1` = Fall) are **metadata, not model input**.
The 2 metadata columns are:
- `split`: which dataset split the row came from (`train`, `val`, `test`)
- `label`: ground-truth class (`0` = Normal, `1` = Fall)
## Training configuration
**Upstream pose extraction (data generation only, not classifier training):**
- Model: yolo26x-pose.pt (Ultralytics)
- Batch size: 32
- Detection confidence threshold: 0.20
- Keypoint confidence threshold: 0.25 (for CSV generation; inference threshold for NaN masking is 0.50)
- Compute: PyTorch AMP mixed precision (CUDA) — only for extraction
**Feature engineering (output is classifier input — 51 + 5 = 56, with 58 columns total in CSV):**
- 51 pose features (cols 1–51): normalized keypoint features (17 COCO joints × 3: x_norm, y_norm, confidence) where `x_norm=(x-x1)/w`, `y_norm=(y-y1)/h`
- + 5 additional Priority 1 features (cols 52–56):
- `aspect_ratio` = w / h (col 52)
- `nose_relative_y` = (Y_nose - y1) / h (col 53)
- `torso_angle` = angle HipMid→ShoulderMid vs. Y-axis (col 54)
- `norm_com_y` = confidence-weighted center of mass Y (col 55)
- `head_hip_v_dist` = (Y_hip_mid - Y_nose) / h (col 56)
- = **56 classifier features (XGBoost input, columns 1–56)**
- + 1 `split` (col 57, `train`/`val`/`test`) + 1 `label` (col 58, `0`/`1`) = **58 columns total** in CSV
In `pose_extract.py`: `row = norm_kpts + [aspect_ratio] + p1_features + [split, label]` → 51 + 5 + 1 + 1 = 58. XGBoost uses only columns 1–56.
**XGBoost hyperparameters:**
- `n_estimators`: 150
- `max_depth`: 5
- `learning_rate`: 0.03
- `scale_pos_weight`: 1.0 (data pre-balanced)
- `missing`: np.nan
- `eval_metric`: logloss
- `random_state`: 42
**Classifier I/O:**
- Input: `(n, 56)` float **pandas DataFrame** (one row per person); CSV never enters the model at inference
- Output: `Fall` / `Normal` via `predict` / `predict_proba`; rule `P(Fall) >= 0.70 → Fall`
## Training performance record
**Standard test set evaluation (56-feature test vectors):**
Training completed successfully. Model saved to `models/xgboost_priority1_fall_model.pkl`.
```
--- XGBoost Classification Report (56 features → Fall/Normal) ---
precision recall f1-score support
Normal (0) 0.90 0.91 0.91 3207
Fall (1) 0.92 0.91 0.91 3400
accuracy 0.91 6607
macro avg 0.91 0.91 0.91 6607
weighted avg 0.91 0.91 0.91 6607
```
**Overall accuracy**: 91.00%
Grid search (classifier thresholds only: `CONF_THRESH` 0.25/0.50, `FALL_PROB_THRESH` 0.20–0.95) achieved best: Accuracy 92.67%, F1 90.91% at CONF=0.50, Fall_Prob=0.70. See `README.md` / `docs/spec.md` for full table.
|