| --- |
| license: other |
| tags: |
| - video |
| - regression |
| - speed-estimation |
| - optical-flow |
| - pytorch |
| datasets: |
| - x2q/onboard-speed-estimation |
| --- |
| |
| # SpeedNet — ego-speed from onboard video (private) |
|
|
| Optical flow → CNN → GRU speed regressor for onboard/POV footage (go-kart, |
| ATV, car, machinery). ~0.9 M parameters. Trained 2026-07-24 on the private |
| dataset `x2q/onboard-speed-estimation`. |
|
|
| ## Checkpoints |
|
|
| | File | Training | Use when | |
| |---|---|---| |
| | `speednet_v2.pt` | GPS per-frame loss + lap-time weak supervision (Racehall 1000 m laps) mixed from scratch | **Recommended.** Indoor kart + ATV + low/mid-speed car | |
| | `speednet_best.pt` | GPS per-frame loss only | Highway/high-speed car (indoor scale ~2× too low) | |
| | `speednet_v3b.pt` | **RECOMMENDED.** Context branch (RGB frame -> embedding -> GRU) + lap augmentation, trained on 7-domain corpus (own + L2D + comma2k19 + Skagen DK) | First checkpoint best-in-class everywhere: indoor laps 3.4 km/h MAE, ATV 0.65 m/s, highway 5.6 (beats baseline), urban 1.7, Skagen 2.1, L2D 2.7. Needs ctx frame input (see train_v3.py) | |
| | `speednet_v2b.pt` | v2 recipe retrained on expanded corpus (own 95 min + 72 L2D EU episodes + 120 comma2k19 segments, fps-normalized flow) | Best all-round GPS domains: ATV 0.73 m/s, L2D 4.2, comma 2.6, highway 8.7; indoor laps 6.0 km/h MAE (−6 bias — lap weight vs bigger corpus, tune LAMBDA to rebalance) | |
|
|
| ## Test metrics (leave-recording-out) |
|
|
| | Test set | best | v2 | |
| |---|---|---| |
| | ATV forest, different day (RMSE) | 0.77 m/s | 0.96 m/s | |
| | Car urban (RMSE) | 3.6 m/s | 2.3 m/s | |
| | Car highway 100–135 km/h (RMSE) | 5.8 m/s | 11.9 m/s | |
| | Indoor kart, holdout laps (lap-mean MAE vs official times) | ~42 km/h | **1.8 km/h** (full-lap window) / 4.6 km/h (streaming, +bias) | |
|
|
| Known limitation: monocular scale ambiguity — without scene context the model |
| trades highway accuracy for indoor accuracy (v2). |
|
|
| RGB context branch: TRIED (2026-07-24, v3 concat-embedding and v4 scalar |
| scale-gain). Both partially restored highway but learned session/scene quirks |
| instead of a generalizing scene→scale mapping, degrading other domains. Root |
| cause is data diversity (4 domains, one indoor venue), not architecture — |
| RESOLVED (2026-07-25): with 7 domains (added L2D EU, comma2k19 US, Skagen DK) |
| the context branch (v3b) learned a generalizing scene-scale mapping — single |
| checkpoint best-in-class on all test domains. Lap-forward augmentation also |
| required (prevents lap memorization). |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from modeling_speednet import SpeedNet, predict_streaming, compute_flow |
| |
| model = SpeedNet().cuda() |
| model.load_state_dict(torch.load('speednet_v2.pt')) |
| flow = compute_flow('clip_320x180_15fps.mp4') # [N, 72, 128, 2] |
| speed_mps = predict_streaming(model, flow) # per-frame, m/s |
| ``` |
|
|
| **Inference MUST be streaming** (hidden state carried across the clip, as in |
| `predict_streaming`). Windowed inference with reset state collapses |
| indoor-domain output to ~half scale. Post-smoothing with a ~1 s rolling |
| median is recommended. |
|
|
| Input contract: 320×180 @ 15 fps video → Farneback flow → resize 128×72. |
| Other resolutions/framerates change the flow distribution and will |
| mis-scale predictions. |
|
|
| ## Provenance |
|
|
| - GPS labels: Garmin Dash Cam 66W (car), DJI Osmo Action 5 Pro + GPS remote |
| (ATV/miniloader, djmd protobuf, position-derived 10 Hz). |
| - Lap labels: official SMS-Timing results (Racehall Aarhus, 1000 m track), |
| aligned to helmet-cam video; holdout laps never trained. |
| - The project's older fused indoor telemetry was rejected as ground truth. |
|
|
