|
Download applications/avoiding/avoiding_baselines/eval/RESULTS.md from GlobalMotionPlanning/GlobMP: direct link, hf CLI and curl.
- Browser
- Download file 2.82 kB
-
https://huggingface.co/GlobalMotionPlanning/GlobMP/resolve/main/applications/avoiding/avoiding_baselines/eval/RESULTS.md
- Command line
-
hf download hf://GlobalMotionPlanning/GlobMP/applications/avoiding/avoiding_baselines/eval/RESULTS.md
-
curl -L -o RESULTS.md https://huggingface.co/GlobalMotionPlanning/GlobMP/resolve/main/applications/avoiding/avoiding_baselines/eval/RESULTS.md
2.82 kB
D3IL avoiding-v0 results
Measured with this folder's code and checkpoints/model_ema_10.pth. These are
the numbers to reproduce, and the ones the paper's Table 3 reports.
- 100 episodes per method, seed 0, at most 100 environment steps, planning horizon H = 16, replanning every 4 executed steps.
- Twelve constraints (
D3IL_HARD=1): the nine of the shippednovelset plus the three keep-outs upstream ships commented out. - All ten policies sample from the same planner trained here (500k steps, batch 32), so differences are the planners', not the model's.
- SR is the safety rate, TS the task-success rate. "Steps (safe)" averages the episode length over the safe episodes only, so a method with SR 0 has none.
| Method | SR ↑ | TS ↑ | Steps (safe) ↓ | Violations ↓ | Time (s) ↓ |
|---|---|---|---|---|---|
| Diffuser (unconstrained) | 0.01 | 0.89 | 61.00 ± 0.00 | 14.31 | 0.09 |
| Classifier Guidance | 0.00 | 0.01 | — | 5.50 | 0.17 |
| Gradient Guidance | 0.24 | 0.97 | 62.96 ± 7.27 | 4.35 | 1.16 |
| Projected Diffusion | 0.45 | 0.79 | 65.18 ± 8.09 | 2.19 | 7.16 |
| [Primal-Dual] | 0.05 | 0.00 | 100.00 ± 0.00 | 10.21 | 0.74 |
| [Augmented Lagrangian] | 0.00 | 0.00 | — | 25.38 | 0.79 |
| SafeDiffuser-RoS | 0.06 | 0.20 | 67.50 ± 6.55 | 3.73 | 7.34 |
| SafeDiffuser-ReS | 0.24 | 0.65 | 62.17 ± 5.10 | 3.97 | 5.44 |
| SafeDiffuser-TVS | 0.02 | 0.95 | 58.50 ± 1.50 | 13.54 | 5.26 |
| DiRecT | 0.87 | 0.97 | 61.38 ± 4.84 | 0.34 | 3.72 |
Reproducing
data_prep/fit_dynamics.sh # seconds, CPU
run/run_baseline.sh direct 0 100 # one policy
run/run_all.sh 0 # all ten, Table 3
Compare runs/avoiding-v0/eval/<exp-name>/summary.json against the table.
Agreement should be close but not bit-exact across machines: avg_computation_time
is wall clock, and SR at 100 episodes still carries a standard error of about
0.05 at mid-range rates.
What the brackets mean
[Primal-Dual] and [Augmented Lagrangian] need --guidance-last-steps 50.
At the default both crash the simulator, so their rows are bracketed in the
paper: they are reported, but under a setting the other methods did not need.
run/run_baseline.sh passes the flag for those two only.
Traps
D3IL_HARD=1is the whole difference between this table and the nine-constraint runs. Without it the same code gives a different table -- e.g. DiRecT reads safety 0.97 rather than 0.87.--diffusion-cp 10selectsmodel_ema_10.pth. Another checkpoint index is a different model and will not reproduce these numbers.WANDB_MODE=disabled, ortrain.pywaits for a login.- The dataset is vendored in
DiRecT/d3il/environments/dataset/data/avoiding/(96 expert episodes); nothing is downloaded.