Point card at the renamed GitHub repos (crowd-nav = package, crowd-nav-legacy = research)

README.md

@@ -1,131 +1,132 @@
----
-license: mit
-library_name: pytorch
-pipeline_tag: robotics
-tags:
-  - robotics
-  - social-navigation
-  - human-robot-interaction
-  - pedestrian-trajectory-prediction
-  - model-predictive-control
-  - convex-optimization
----
-
-# crowd-nav — human-response models for the Foresight controller
-
-Trained weights for **Foresight**, a convex receding-horizon controller that predicts how each
-nearby pedestrian will respond to the robot's *candidate plan*, and then chooses that plan by
-exact convex optimization. These are the response models the controller calls; the planner is
-plain code (no learned policy, no RL).
-
-| file | model | role |
-|---|---|---|
-| `linres_head.pt` | **Linear-response operator** (headline) | best closed-loop performance; makes planning an exact convex QP |
-| `residual_predictor_k1.pt` | Residual predictor | most accurate on real data; the safest variant in closed loop |
-| `residual_predictor.pt` | *legacy* | **deprecated** — trained with a robot-force prior of 0 but deployed at 1; superseded by `residual_predictor_k1.pt` |
-
-## Which one should I use?
-
-**`linres_head.pt`** unless you have a reason not to. It is a few centimetres less accurate as a
-pure forecaster, but its prediction is *exactly affine* in the robot's planned displacement, so the
-planner gets an exact constant Jacobian and the safety constraints stay linear — the whole plan
-selection becomes a convex QP that solves in ~17 ms with a checkable feasibility certificate.
-That trade wins where it counts: it leads our benchmark, and it is the only variant that crossed a
-90-pedestrian oncoming corridor with zero contacts.
-
-## Measured results
-
-Isaac Sim, 16 scenarios × 10 paired seeds (160 episodes per method), identical hardware:
-
-| model | goal reach | hard collisions / ep | plan latency (mean) |
-|---|---|---|---|
-| **Foresight-LinRes** (`linres_head.pt`) | **94.4 %** | 0.013 | **16.6 ms** |
-| Foresight-Residual (`residual_predictor_k1.pt`) | 90.0 % | **0.006** | 19.7 ms |
-| Foresight-SFM (physics only, ablation) | 93.8 % | 0.013 | 17.2 ms |
-| Foresight-CV (straight-line, ablation) | 90.0 % | 0.019 | 16.4 ms |
-| SICNav [T-RO '24] (nonconvex bilevel MPC) | 89.2 % | 0.042 | 1843 ms |
-
-Stress test — a 4.5 m × 105 m corridor with 90 pedestrians *all walking against* the robot and a
-continuous inflow so it never empties: LinRes reached the goal in 98.5 s with **zero contacts**;
-against pedestrian models it was never tuned on (SRFM, social-force) it reached in 85.0 s and
-91.7 s, also contact-free.
-
-Prediction accuracy on the held-out real split (factual ADE, metres — lower is better):
-
-| model | no robot | stationary robot | moving robot |
-|---|---|---|---|
-| Constant velocity | 0.240 | 0.253 | 0.354 |
-| Social force | 0.343 | 0.358 | 0.416 |
-| Residual (k1) | **0.192** | **0.217** | 0.285 |
-| Linear response | 0.204 | 0.248 | 0.319 |
-
-## Architecture
-
-Both models are a **physics prior plus a learned correction** — small by design, because the real
-moving-robot data is scarce.
-
-```
-prior (no parameters, per pedestrian per 0.25 s step)
-  f_goal  = (1.3·(g−p)/‖g−p‖ − v)/0.5
-  f_ped   = Σ 2.0·exp(−d/0.4)·d̂      (d < 2.0 m)
-  f_robot = k·exp(−d/0.5)·d̂          (d < 3.0 m)
-  v ← clip(v + f·Δt, 1.8);  p ← p + v·Δt
-
-residual_predictor_k1.pt     ŷ = prior(k=1) + f_θ(x)
-  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
-
-linres_head.pt               ŷ = prior(k=0) + f_θ(x) + G_φ(x)·ΔR
-  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
-  G_φ : MLP 22 → 64 → 96          (7,712 params) ⇒ rank-3 16×16 operator
-        G = Σ_{r=1..3} a_r b_rᵀ,  ΔR = (R − 1⊗r₀)/5.0
-        ⇒ ∂ŷ/∂R = G/5.0  — exact, constant, no differentiation needed
-```
-
-Input `x` (22-d, pedestrian local frame): 1 s of past positions (4 @ 0.25 s), velocity, goal
-direction, robot relative position, a robot-nearby flag, the two nearest neighbours, and a 3-way
-condition one-hot. Output: 8 × 2 future displacements — 2.0 s at 0.25 s.
-
-Training: supervised (Adam, lr 1e-3, batch 128, ~25 epochs), synthetic pre-train then fine-tune on
-the real **PeRoI** robot–pedestrian recordings. No reinforcement learning anywhere in this stack.
-Both run in well under 1 ms per call on CPU.
-
-## Load
-
-```bash
-pip install huggingface_hub torch
-hf download elmoghany/crowd-nav linres_head.pt residual_predictor_k1.pt --local-dir .
-```
-
-```python
-import torch
-ck = torch.load("linres_head.pt", map_location="cpu")
-ck["config"]
-# {'in_dim': 22, 'horizon_dim': 16, 'rank': 3, 'hidden': 64, 'dr_scale': 5.0,
-#  'sfm_k_robot': 0.0, 'convention': 'y = sfm(k=0) + f_resid(x_ctx) + head(x_ctx, dR)'}
-ck["f_resid"], ck["head"]   # two state_dicts
-```
-
-The `config` block is authoritative: `sfm_k_robot` **must** match the robot-force strength used in
-your prior at deployment. Mismatching them is a real bug we shipped once — the residual then
-corrects an error that is not there. `residual_predictor.pt` is the artifact of that mistake and is
-kept only for reproducibility.
-
-Model classes (`ResidualPredictor`, `LinearResponseHead`) and the full controller live in the
-research repo: **[github.com/elmoghany/crowd-nav](https://github.com/elmoghany/crowd-nav)**.
-A sim-free packaging of the controller for real robots is at
-[github.com/elmoghany/peroi-controller](https://github.com/elmoghany/peroi-controller).
-
-## Honest limitations
-
-- **Real-data forecasting is a tie.** On four real robot–pedestrian datasets (including JRDB),
-  every learned model here ties the analytic baselines on factual accuracy. Our measured
-  explanation: the counterfactual proxy those benchmarks rely on (constant-velocity extrapolation)
-  correlates only r ≈ 0.30 with exact simulated counterfactuals — the target is mostly noise. The
-  gains above are **closed-loop** gains, not forecasting gains.
-- **Compliance bias.** The training recordings contain cooperative pedestrians. On deliberately
-  stubborn crowds the k1 residual is the weakest learned variant; compliance-randomized training
-  data is the known fix, not yet applied.
-- **Cross-dataset transfer.** Moving `residual_predictor_k1.pt` to JRDB over-predicts robot-induced
-  deviation by ~6× (its k=1 prior is calibrated to our robot). It makes a controller conservative,
-  not unsafe — it still threaded real JRDB crowds contact-free — but retune `sfm_k_robot` for a
-  different platform.
+---
+license: mit
+library_name: pytorch
+pipeline_tag: robotics
+tags:
+  - robotics
+  - social-navigation
+  - human-robot-interaction
+  - pedestrian-trajectory-prediction
+  - model-predictive-control
+  - convex-optimization
+---
+
+# crowd-nav — human-response models for the Foresight controller
+
+Trained weights for **Foresight**, a convex receding-horizon controller that predicts how each
+nearby pedestrian will respond to the robot's *candidate plan*, and then chooses that plan by
+exact convex optimization. These are the response models the controller calls; the planner is
+plain code (no learned policy, no RL).
+
+| file | model | role |
+|---|---|---|
+| `linres_head.pt` | **Linear-response operator** (headline) | best closed-loop performance; makes planning an exact convex QP |
+| `residual_predictor_k1.pt` | Residual predictor | most accurate on real data; the safest variant in closed loop |
+| `residual_predictor.pt` | *legacy* | **deprecated** — trained with a robot-force prior of 0 but deployed at 1; superseded by `residual_predictor_k1.pt` |
+
+## Which one should I use?
+
+**`linres_head.pt`** unless you have a reason not to. It is a few centimetres less accurate as a
+pure forecaster, but its prediction is *exactly affine* in the robot's planned displacement, so the
+planner gets an exact constant Jacobian and the safety constraints stay linear — the whole plan
+selection becomes a convex QP that solves in ~17 ms with a checkable feasibility certificate.
+That trade wins where it counts: it leads our benchmark, and it is the only variant that crossed a
+90-pedestrian oncoming corridor with zero contacts.
+
+## Measured results
+
+Isaac Sim, 16 scenarios × 10 paired seeds (160 episodes per method), identical hardware:
+
+| model | goal reach | hard collisions / ep | plan latency (mean) |
+|---|---|---|---|
+| **Foresight-LinRes** (`linres_head.pt`) | **94.4 %** | 0.013 | **16.6 ms** |
+| Foresight-Residual (`residual_predictor_k1.pt`) | 90.0 % | **0.006** | 19.7 ms |
+| Foresight-SFM (physics only, ablation) | 93.8 % | 0.013 | 17.2 ms |
+| Foresight-CV (straight-line, ablation) | 90.0 % | 0.019 | 16.4 ms |
+| SICNav [T-RO '24] (nonconvex bilevel MPC) | 89.2 % | 0.042 | 1843 ms |
+
+Stress test — a 4.5 m × 105 m corridor with 90 pedestrians *all walking against* the robot and a
+continuous inflow so it never empties: LinRes reached the goal in 98.5 s with **zero contacts**;
+against pedestrian models it was never tuned on (SRFM, social-force) it reached in 85.0 s and
+91.7 s, also contact-free.
+
+Prediction accuracy on the held-out real split (factual ADE, metres — lower is better):
+
+| model | no robot | stationary robot | moving robot |
+|---|---|---|---|
+| Constant velocity | 0.240 | 0.253 | 0.354 |
+| Social force | 0.343 | 0.358 | 0.416 |
+| Residual (k1) | **0.192** | **0.217** | 0.285 |
+| Linear response | 0.204 | 0.248 | 0.319 |
+
+## Architecture
+
+Both models are a **physics prior plus a learned correction** — small by design, because the real
+moving-robot data is scarce.
+
+```
+prior (no parameters, per pedestrian per 0.25 s step)
+  f_goal  = (1.3·(g−p)/‖g−p‖ − v)/0.5
+  f_ped   = Σ 2.0·exp(−d/0.4)·d̂      (d < 2.0 m)
+  f_robot = k·exp(−d/0.5)·d̂          (d < 3.0 m)
+  v ← clip(v + f·Δt, 1.8);  p ← p + v·Δt
+
+residual_predictor_k1.pt     ŷ = prior(k=1) + f_θ(x)
+  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
+
+linres_head.pt               ŷ = prior(k=0) + f_θ(x) + G_φ(x)·ΔR
+  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
+  G_φ : MLP 22 → 64 → 96          (7,712 params) ⇒ rank-3 16×16 operator
+        G = Σ_{r=1..3} a_r b_rᵀ,  ΔR = (R − 1⊗r₀)/5.0
+        ⇒ ∂ŷ/∂R = G/5.0  — exact, constant, no differentiation needed
+```
+
+Input `x` (22-d, pedestrian local frame): 1 s of past positions (4 @ 0.25 s), velocity, goal
+direction, robot relative position, a robot-nearby flag, the two nearest neighbours, and a 3-way
+condition one-hot. Output: 8 × 2 future displacements — 2.0 s at 0.25 s.
+
+Training: supervised (Adam, lr 1e-3, batch 128, ~25 epochs), synthetic pre-train then fine-tune on
+the real **PeRoI** robot–pedestrian recordings. No reinforcement learning anywhere in this stack.
+Both run in well under 1 ms per call on CPU.
+
+## Load
+
+```bash
+pip install huggingface_hub torch
+hf download elmoghany/crowd-nav linres_head.pt residual_predictor_k1.pt --local-dir .
+```
+
+```python
+import torch
+ck = torch.load("linres_head.pt", map_location="cpu")
+ck["config"]
+# {'in_dim': 22, 'horizon_dim': 16, 'rank': 3, 'hidden': 64, 'dr_scale': 5.0,
+#  'sfm_k_robot': 0.0, 'convention': 'y = sfm(k=0) + f_resid(x_ctx) + head(x_ctx, dR)'}
+ck["f_resid"], ck["head"]   # two state_dicts
+```
+
+The `config` block is authoritative: `sfm_k_robot` **must** match the robot-force strength used in
+your prior at deployment. Mismatching them is a real bug we shipped once — the residual then
+corrects an error that is not there. `residual_predictor.pt` is the artifact of that mistake and is
+kept only for reproducibility.
+
+Model classes (`ResidualPredictor`, `LinearResponseHead`) and the full controller live in the
+**[github.com/elmoghany/crowd-nav](https://github.com/elmoghany/crowd-nav)** -- a one-file,
+simulator-free controller with a `quickstart.py` that fetches these weights and verifies your
+install in one command. The full research stack (convex-QP planner, Isaac benchmarks, paper) is at
+[github.com/elmoghany/crowd-nav-legacy](https://github.com/elmoghany/crowd-nav-legacy).
+
+## Honest limitations
+
+- **Real-data forecasting is a tie.** On four real robot–pedestrian datasets (including JRDB),
+  every learned model here ties the analytic baselines on factual accuracy. Our measured
+  explanation: the counterfactual proxy those benchmarks rely on (constant-velocity extrapolation)
+  correlates only r ≈ 0.30 with exact simulated counterfactuals — the target is mostly noise. The
+  gains above are **closed-loop** gains, not forecasting gains.
+- **Compliance bias.** The training recordings contain cooperative pedestrians. On deliberately
+  stubborn crowds the k1 residual is the weakest learned variant; compliance-randomized training
+  data is the known fix, not yet applied.
+- **Cross-dataset transfer.** Moving `residual_predictor_k1.pt` to JRDB over-predicts robot-induced
+  deviation by ~6× (its k=1 prior is calibrated to our robot). It makes a controller conservative,
+  not unsafe — it still threaded real JRDB crowds contact-free — but retune `sfm_k_robot` for a
+  different platform.