v3/README.md

# Speedup Patch reproduction bundle (v3)

This bundle audits all six registered claims from **Speedup Patch: Learning a
Plug-and-Play Policy to Accelerate Embodied Manipulation**.

## Evidence levels

The authors do not link or release SuP scheduler code, trained schedulers, raw
rollouts, or robot checkpoints in the paper source as of 2026-07-29. We do not
substitute an unrelated implementation and call it exact.

The bundle therefore combines:

1. an exact audit of the arXiv TeX tables and annotated Figure 7;
2. a real-data experiment using the exact `ARModel` class from the
   author-linked official ADMPO repository on the paper-linked LIBERO-Spatial
   demonstrations; and
3. explicit boundaries for simulator and robot claims that cannot be rerun.

## Real LIBERO experiment

- Dataset: `openvla/modified_libero_rlds`, pinned revision
  `6ce6aaaaabdbe590b1eef5cd29c0d33f14a08551`.
- Shard: `libero_spatial-train.tfrecord-00000-of-00016`, 27 episodes, SHA-256
  `6d6360559a4b71df3bcabd9b55a75451f04375449ca5267616cbc1c21528eac5`.
- Split by episode: 19 train / 4 validation / 4 test.
- Models: official ADM `ARModel` (321,928 parameters) versus a
  parameter-matched MLP (319,308).
- Variable action horizons: 1–10; 4,840 held-out any-step examples.
- Seeds: 42, 43, 44; 1,200 optimizer updates per model and seed.
- Runtime: 123.19 CPU seconds; peak RSS 515.14 MiB.

Mean results:

| Model | State MSE | EEF RMSE | Spearman(predicted, true displacement) |
|---|---:|---:|---:|
| Official ADM | 0.00013650 | 0.013378 | 0.97627 |
| Matched MLP | 0.00014654 | 0.013835 | 0.97300 |

ADM reduces mean state MSE by 6.86% and EEF RMSE by 3.31%. It has lower EEF
RMSE at every long horizon 7–10. This tests the recurrent model choice on real
demonstrations; it does **not** reproduce the Figure 7 violation/success
correlations because success rollouts were not released.

We also executed a bounded SuP-style three-phase path on the same real
demonstrations: fit the official ADM, synthesize both `k=1` and `k=2`
counterfactual transitions using endpoint deviation, and train expectile-IQL
Q/V networks with the paper's LIBERO settings (`epsilon=0.02`, `tau=0.95`,
`gamma=0.1`, penalty `-2`). All 462 held-out `k=2` choices exceeded the
threshold under this endpoint metric, and IQL consequently selected `k=1`
everywhere: zero predicted violations but no acceleration. This is an
informative negative result showing that the exact unreleased trajectory
deviation implementation is material.

## Exact artifact findings

- BiGym ACT+SuP success recomputes from all 20 appendix tasks as
  0.6725 → 0.67. The rounded displayed step lengths give either 1.979
  (ratio of sums) or 2.052 (mean task ratio), so the reported 2.01× cannot be
  uniquely regenerated without raw rollouts.
- LIBERO `pi0.5`+SuP recomputes to 0.973 success and 1.3515× → 1.35×.
- LIBERO VLA-Adapter+SuP recomputes to 0.9365 → 0.937 success and
  1.3361× → 1.34×.
- Real SuP counts recompute to 0.6111 → 0.611 success; displayed lengths give
  2.1693× → 2.17×.
- The 5.12M figure is total SuP trainable parameters in Table 3, not
  scheduler-only parameters and not Table 4.
- Figure 7 annotations show stronger negative ADM correlation in all four
  suites: ADM `[-.20,-.34,-.30,-.05]`, MLP
  `[-.03,-.22,-.25,-.03]`.

## Rerun

```bash
python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
bash download_data.sh data
python reproduce_libero_adm.py \
  --tfrecord data/libero_spatial_no_noops/1.0.0/libero_spatial-train.tfrecord-00000-of-00016 \
  --seeds 42 43 44 --steps 1200 --batch-size 256
python run_offline_iql_mechanism.py \
  --tfrecord data/libero_spatial_no_noops/1.0.0/libero_spatial-train.tfrecord-00000-of-00016 \
  --world-steps 800 --iql-steps 1500 --epsilon 0.02 --seed 42
python audit_paper_artifacts.py
python analyze_release.py
python test_release.py
```

The full source archive is included, as are the precise official ADM snapshot,
raw outputs, figure, and integrity manifest.