# D3IL avoiding-v0 results

Measured with this folder's code and `checkpoints/model_ema_10.pth`. **These are
the numbers to reproduce**, and the ones the paper's Table 3 reports.

- **100 episodes per method**, seed 0, at most 100 environment steps, planning
  horizon H = 16, replanning every 4 executed steps.
- **Twelve constraints** (`D3IL_HARD=1`): the nine of the shipped `novel` set
  plus the three keep-outs upstream ships commented out.
- All ten policies sample from the **same** planner trained here (500k steps,
  batch 32), so differences are the planners', not the model's.
- SR is the safety rate, TS the task-success rate. "Steps (safe)" averages the
  episode length over the safe episodes only, so a method with SR 0 has none.

| Method | SR ↑ | TS ↑ | Steps (safe) ↓ | Violations ↓ | Time (s) ↓ |
|---|---|---|---|---|---|
| Diffuser (unconstrained) | 0.01 | 0.89 | 61.00 ± 0.00 | 14.31 | 0.09 |
| Classifier Guidance | 0.00 | 0.01 | — | 5.50 | 0.17 |
| Gradient Guidance | 0.24 | 0.97 | 62.96 ± 7.27 | 4.35 | 1.16 |
| Projected Diffusion | 0.45 | 0.79 | 65.18 ± 8.09 | 2.19 | 7.16 |
| [Primal-Dual] | 0.05 | 0.00 | 100.00 ± 0.00 | 10.21 | 0.74 |
| [Augmented Lagrangian] | 0.00 | 0.00 | — | 25.38 | 0.79 |
| SafeDiffuser-RoS | 0.06 | 0.20 | 67.50 ± 6.55 | 3.73 | 7.34 |
| SafeDiffuser-ReS | 0.24 | 0.65 | 62.17 ± 5.10 | 3.97 | 5.44 |
| SafeDiffuser-TVS | 0.02 | 0.95 | 58.50 ± 1.50 | 13.54 | 5.26 |
| DiRecT | 0.87 | 0.97 | 61.38 ± 4.84 | 0.34 | 3.72 |

## Reproducing

```bash
data_prep/fit_dynamics.sh                 # seconds, CPU
run/run_baseline.sh direct 0 100          # one policy
run/run_all.sh 0                          # all ten, Table 3
```

Compare `runs/avoiding-v0/eval/<exp-name>/summary.json` against the table.
Agreement should be close but not bit-exact across machines: `avg_computation_time`
is wall clock, and SR at 100 episodes still carries a standard error of about
0.05 at mid-range rates.

## What the brackets mean

`[Primal-Dual]` and `[Augmented Lagrangian]` need `--guidance-last-steps 50`.
At the default both crash the simulator, so their rows are bracketed in the
paper: they are reported, but under a setting the other methods did not need.
`run/run_baseline.sh` passes the flag for those two only.

## Traps

1. **`D3IL_HARD=1` is the whole difference** between this table and the
   nine-constraint runs. Without it the same code gives a different table --
   e.g. DiRecT reads safety 0.97 rather than 0.87.
2. **`--diffusion-cp 10`** selects `model_ema_10.pth`. Another checkpoint index
   is a different model and will not reproduce these numbers.
3. **`WANDB_MODE=disabled`**, or `train.py` waits for a login.
4. The dataset is vendored in `DiRecT/d3il/environments/dataset/data/avoiding/`
   (96 expert episodes); nothing is downloaded.