SabaPivot/repro-speedup-patch-upgrades/v3 - Storage Bucket

Speedup Patch reproduction bundle (v3)

This bundle audits all six registered claims from Speedup Patch: Learning a Plug-and-Play Policy to Accelerate Embodied Manipulation.

Evidence levels

The authors do not link or release SuP scheduler code, trained schedulers, raw rollouts, or robot checkpoints in the paper source as of 2026-07-29. We do not substitute an unrelated implementation and call it exact.

The bundle therefore combines:

1. an exact audit of the arXiv TeX tables and annotated Figure 7;
2. a real-data experiment using the exact ARModel class from the author-linked official ADMPO repository on the paper-linked LIBERO-Spatial demonstrations; and
3. explicit boundaries for simulator and robot claims that cannot be rerun.

Real LIBERO experiment

• Dataset: openvla/modified_libero_rlds, pinned revision 6ce6aaaaabdbe590b1eef5cd29c0d33f14a08551.
• Shard: libero_spatial-train.tfrecord-00000-of-00016, 27 episodes, SHA-256 6d6360559a4b71df3bcabd9b55a75451f04375449ca5267616cbc1c21528eac5.
• Split by episode: 19 train / 4 validation / 4 test.
• Models: official ADM ARModel (321,928 parameters) versus a parameter-matched MLP (319,308).
• Variable action horizons: 1–10; 4,840 held-out any-step examples.
• Seeds: 42, 43, 44; 1,200 optimizer updates per model and seed.
• Runtime: 123.19 CPU seconds; peak RSS 515.14 MiB.

Mean results:

Model	State MSE	EEF RMSE	Spearman(predicted, true displacement)
Official ADM	0.00013650	0.013378	0.97627
Matched MLP	0.00014654	0.013835	0.97300

ADM reduces mean state MSE by 6.86% and EEF RMSE by 3.31%. It has lower EEF RMSE at every long horizon 7–10. This tests the recurrent model choice on real demonstrations; it does not reproduce the Figure 7 violation/success correlations because success rollouts were not released.

We also executed a bounded SuP-style three-phase path on the same real demonstrations: fit the official ADM, synthesize both k=1 and k=2 counterfactual transitions using endpoint deviation, and train expectile-IQL Q/V networks with the paper's LIBERO settings (epsilon=0.02, tau=0.95, gamma=0.1, penalty -2). All 462 held-out k=2 choices exceeded the threshold under this endpoint metric, and IQL consequently selected k=1 everywhere: zero predicted violations but no acceleration. This is an informative negative result showing that the exact unreleased trajectory deviation implementation is material.

Exact artifact findings

• BiGym ACT+SuP success recomputes from all 20 appendix tasks as 0.6725 → 0.67. The rounded displayed step lengths give either 1.979 (ratio of sums) or 2.052 (mean task ratio), so the reported 2.01× cannot be uniquely regenerated without raw rollouts.
• LIBERO pi0.5+SuP recomputes to 0.973 success and 1.3515× → 1.35×.
• LIBERO VLA-Adapter+SuP recomputes to 0.9365 → 0.937 success and 1.3361× → 1.34×.
• Real SuP counts recompute to 0.6111 → 0.611 success; displayed lengths give 2.1693× → 2.17×.
• The 5.12M figure is total SuP trainable parameters in Table 3, not scheduler-only parameters and not Table 4.
• Figure 7 annotations show stronger negative ADM correlation in all four suites: ADM [-.20,-.34,-.30,-.05], MLP [-.03,-.22,-.25,-.03].

Rerun

python -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
bash download_data.sh data
python reproduce_libero_adm.py \
  --tfrecord data/libero_spatial_no_noops/1.0.0/libero_spatial-train.tfrecord-00000-of-00016 \
  --seeds 42 43 44 --steps 1200 --batch-size 256
python run_offline_iql_mechanism.py \
  --tfrecord data/libero_spatial_no_noops/1.0.0/libero_spatial-train.tfrecord-00000-of-00016 \
  --world-steps 800 --iql-steps 1500 --epsilon 0.02 --seed 42
python audit_paper_artifacts.py
python analyze_release.py
python test_release.py

The full source archive is included, as are the precise official ADM snapshot, raw outputs, figure, and integrity manifest.