| --- |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - lerobot |
| - robotics |
| - imitation-learning |
| - x7s |
| - act |
| - offline-evaluation |
| datasets: |
| - LightwheelAI/Lightwheel-Tasks-X7S |
| --- |
| |
| # LW-Bench ACT 5K for X7s |
|
|
| LeRobot ACT policy trained from scratch on the audited X7s strict compositional split in LW-Compositional-Bench. |
|
|
| ## Training contract |
|
|
| - Dataset: `LightwheelAI/Lightwheel-Tasks-X7S` |
| - Train split: 91 clean tasks; tasks 7, 119, and 149 held out |
| - Inputs: three RGB cameras plus 25D robot state |
| - Output: native 21D X7s absolute action |
| - Hardware: 4 GPUs |
| - Seed: 1000 |
| - Effective global batch: 64 |
| - Checkpoint: 5,000 global optimizer updates |
| - Evaluation horizon: first 16 valid action steps |
|
|
| ## Offline 5K results |
|
|
| | Metric | Value | |
| | --- | ---: | |
| | H16 MAE | 0.077566 | |
| | H16 RMSE | 0.163998 | |
| | Gripper balanced accuracy | 94.73% | |
| | Boundary-proxy MAE | 0.109228 | |
| | Interior MAE | 0.071501 | |
| | Boundary / interior MAE | 1.528x | |
|
|
| These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation. |
|
|
| ## Load with LeRobot |
|
|
| ```python |
| from lerobot.policies.act.modeling_act import ACTPolicy |
| |
| policy = ACTPolicy.from_pretrained("Stevenshuqing/LW-Bench-ACT") |
| policy.eval() |
| ``` |
|
|
| Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete `pretrained_model` directory produced by LeRobot. |
|
|
| ## Provenance |
|
|
| - Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench |
| - Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md |
| - Code revision: `2d0e26dd2fe2e91046f416bd65bc369f8568d7f0` |
| - `model.safetensors` SHA256: `dffc7c1801b77a53c18fe3f78627aa742ec0d158e0ece5b78737ca4edab6dfa4` |
|
|
| ## Limitations |
|
|
| This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations. |
|
|
