Add Foresight response models: linear-response operator (headline) + k1 residual; rewrite card

README.md

@@ -1,44 +1,131 @@
----
-license: mit
-library_name: pytorch
-pipeline_tag: robotics
-tags:
-  - robotics
-  - social-navigation
-  - human-robot-interaction
-  - pedestrian-trajectory-prediction
-  - model-predictive-control
----
-
-# PeRoI controller — trained weights
-
-`residual_predictor.pt` — the action-conditioned pedestrian-response predictor behind the **PeRoI
-social-navigation controller**. A NeuRoSFM residual (`ŷ = SocialForce + learned_correction`) trained on
-the real **PeRoI** robot–human interaction dataset; deployed on a robot as a velocity-grid MPC that
-anticipates how each nearby person will react to the robot and plans around them.
-
-**The controller code, real-robot integration guide, ROS node, and sanity check live in the private
-GitHub repo → `github.com/elmoghany/peroi-controller`.** This HF repo hosts only the weights.
-
-## Download the weights
-
-```bash
-pip install huggingface_hub
-hf download elmoghany/peroi-controller residual_predictor.pt --local-dir .
-# (private repo — run `hf auth login` first with an account that has access)
-```
-
-## Use
-
-```python
-from peroi_controller import PeRoIController   # from the GitHub repo
-ctrl = PeRoIController("residual_predictor.pt", robot_radius=0.30, v_max=0.6)
-vx, vy = ctrl.step(robot_xy, goal_xy, {track_id: (x, y), ...}, dt=loop_dt)
-```
-
-Architecture: 3-layer MLP residual on a Social-Force prior; input = pedestrian local-frame features
-(past 1 s + goal direction + robot relative state + nearest neighbours + robot-condition one-hot),
-output = 8×2 future deltas (2 s @ 0.25 s). ~90 KB, runs in <1 ms/step on CPU.
-
-Provenance + metrics + videos: **https://elmoghany.com/crowd-nav**. License: MIT.
-Contact: Mohamed Elmoghany (Cornell).
+---
+license: mit
+library_name: pytorch
+pipeline_tag: robotics
+tags:
+  - robotics
+  - social-navigation
+  - human-robot-interaction
+  - pedestrian-trajectory-prediction
+  - model-predictive-control
+  - convex-optimization
+---
+
+# crowd-nav — human-response models for the Foresight controller
+
+Trained weights for **Foresight**, a convex receding-horizon controller that predicts how each
+nearby pedestrian will respond to the robot's *candidate plan*, and then chooses that plan by
+exact convex optimization. These are the response models the controller calls; the planner is
+plain code (no learned policy, no RL).
+
+| file | model | role |
+|---|---|---|
+| `linres_head.pt` | **Linear-response operator** (headline) | best closed-loop performance; makes planning an exact convex QP |
+| `residual_predictor_k1.pt` | Residual predictor | most accurate on real data; the safest variant in closed loop |
+| `residual_predictor.pt` | *legacy* | **deprecated** — trained with a robot-force prior of 0 but deployed at 1; superseded by `residual_predictor_k1.pt` |
+
+## Which one should I use?
+
+**`linres_head.pt`** unless you have a reason not to. It is a few centimetres less accurate as a
+pure forecaster, but its prediction is *exactly affine* in the robot's planned displacement, so the
+planner gets an exact constant Jacobian and the safety constraints stay linear — the whole plan
+selection becomes a convex QP that solves in ~17 ms with a checkable feasibility certificate.
+That trade wins where it counts: it leads our benchmark, and it is the only variant that crossed a
+90-pedestrian oncoming corridor with zero contacts.
+
+## Measured results
+
+Isaac Sim, 16 scenarios × 10 paired seeds (160 episodes per method), identical hardware:
+
+| model | goal reach | hard collisions / ep | plan latency (mean) |
+|---|---|---|---|
+| **Foresight-LinRes** (`linres_head.pt`) | **94.4 %** | 0.013 | **16.6 ms** |
+| Foresight-Residual (`residual_predictor_k1.pt`) | 90.0 % | **0.006** | 19.7 ms |
+| Foresight-SFM (physics only, ablation) | 93.8 % | 0.013 | 17.2 ms |
+| Foresight-CV (straight-line, ablation) | 90.0 % | 0.019 | 16.4 ms |
+| SICNav [T-RO '24] (nonconvex bilevel MPC) | 89.2 % | 0.042 | 1843 ms |
+
+Stress test — a 4.5 m × 105 m corridor with 90 pedestrians *all walking against* the robot and a
+continuous inflow so it never empties: LinRes reached the goal in 98.5 s with **zero contacts**;
+against pedestrian models it was never tuned on (SRFM, social-force) it reached in 85.0 s and
+91.7 s, also contact-free.
+
+Prediction accuracy on the held-out real split (factual ADE, metres — lower is better):
+
+| model | no robot | stationary robot | moving robot |
+|---|---|---|---|
+| Constant velocity | 0.240 | 0.253 | 0.354 |
+| Social force | 0.343 | 0.358 | 0.416 |
+| Residual (k1) | **0.192** | **0.217** | 0.285 |
+| Linear response | 0.204 | 0.248 | 0.319 |
+
+## Architecture
+
+Both models are a **physics prior plus a learned correction** — small by design, because the real
+moving-robot data is scarce.
+
+```
+prior (no parameters, per pedestrian per 0.25 s step)
+  f_goal  = (1.3·(g−p)/‖g−p‖ − v)/0.5
+  f_ped   = Σ 2.0·exp(−d/0.4)·d̂      (d < 2.0 m)
+  f_robot = k·exp(−d/0.5)·d̂          (d < 3.0 m)
+  v ← clip(v + f·Δt, 1.8);  p ← p + v·Δt
+
+residual_predictor_k1.pt     ŷ = prior(k=1) + f_θ(x)
+  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
+
+linres_head.pt               ŷ = prior(k=0) + f_θ(x) + G_φ(x)·ΔR
+  f_θ : MLP 22 → 128 → 128 → 16   (21,520 params)
+  G_φ : MLP 22 → 64 → 96          (7,712 params) ⇒ rank-3 16×16 operator
+        G = Σ_{r=1..3} a_r b_rᵀ,  ΔR = (R − 1⊗r₀)/5.0
+        ⇒ ∂ŷ/∂R = G/5.0  — exact, constant, no differentiation needed
+```
+
+Input `x` (22-d, pedestrian local frame): 1 s of past positions (4 @ 0.25 s), velocity, goal
+direction, robot relative position, a robot-nearby flag, the two nearest neighbours, and a 3-way
+condition one-hot. Output: 8 × 2 future displacements — 2.0 s at 0.25 s.
+
+Training: supervised (Adam, lr 1e-3, batch 128, ~25 epochs), synthetic pre-train then fine-tune on
+the real **PeRoI** robot–pedestrian recordings. No reinforcement learning anywhere in this stack.
+Both run in well under 1 ms per call on CPU.
+
+## Load
+
+```bash
+pip install huggingface_hub torch
+hf download elmoghany/crowd-nav linres_head.pt residual_predictor_k1.pt --local-dir .
+```
+
+```python
+import torch
+ck = torch.load("linres_head.pt", map_location="cpu")
+ck["config"]
+# {'in_dim': 22, 'horizon_dim': 16, 'rank': 3, 'hidden': 64, 'dr_scale': 5.0,
+#  'sfm_k_robot': 0.0, 'convention': 'y = sfm(k=0) + f_resid(x_ctx) + head(x_ctx, dR)'}
+ck["f_resid"], ck["head"]   # two state_dicts
+```
+
+The `config` block is authoritative: `sfm_k_robot` **must** match the robot-force strength used in
+your prior at deployment. Mismatching them is a real bug we shipped once — the residual then
+corrects an error that is not there. `residual_predictor.pt` is the artifact of that mistake and is
+kept only for reproducibility.
+
+Model classes (`ResidualPredictor`, `LinearResponseHead`) and the full controller live in the
+research repo: **[github.com/elmoghany/crowd-nav](https://github.com/elmoghany/crowd-nav)**.
+A sim-free packaging of the controller for real robots is at
+[github.com/elmoghany/peroi-controller](https://github.com/elmoghany/peroi-controller).
+
+## Honest limitations
+
+- **Real-data forecasting is a tie.** On four real robot–pedestrian datasets (including JRDB),
+  every learned model here ties the analytic baselines on factual accuracy. Our measured
+  explanation: the counterfactual proxy those benchmarks rely on (constant-velocity extrapolation)
+  correlates only r ≈ 0.30 with exact simulated counterfactuals — the target is mostly noise. The
+  gains above are **closed-loop** gains, not forecasting gains.
+- **Compliance bias.** The training recordings contain cooperative pedestrians. On deliberately
+  stubborn crowds the k1 residual is the weakest learned variant; compliance-randomized training
+  data is the known fix, not yet applied.
+- **Cross-dataset transfer.** Moving `residual_predictor_k1.pt` to JRDB over-predicts robot-induced
+  deviation by ~6× (its k=1 prior is calibrated to our robot). It makes a controller conservative,
+  not unsafe — it still threaded real JRDB crowds contact-free — but retune `sfm_k_robot` for a
+  different platform.

linres_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b62533c21dc5914bb9d146cbb09be99717ec1e6be7f7fa90a8d8025e9f33539
+size 121136

residual_predictor_k1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c775aa7966703671bb7b75f6d53a5a4543d4b92a2e49724a8e5c86e68b9ae547
+size 88980