File size: 2,259 Bytes
2248be6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | ---
library_name: lerobot
pipeline_tag: robotics
tags:
- lerobot
- robotics
- imitation-learning
- x7s
- diffusion-policy
- offline-evaluation
datasets:
- LightwheelAI/Lightwheel-Tasks-X7S
---
# LW-Bench Diffusion Policy 5K for X7s
LeRobot Diffusion Policy trained from scratch on the audited X7s strict compositional split in LW-Compositional-Bench.
## Training contract
- Dataset: `LightwheelAI/Lightwheel-Tasks-X7S`
- Train split: 91 clean tasks; tasks 7, 119, and 149 held out
- Inputs: three RGB cameras plus 25D robot state
- Output: native 21D X7s absolute action
- Architecture: ResNet18 plus conditional diffusion
- Training horizon: 24 with 2 observations and 16 future action steps
- Hardware: 4 GPUs
- Seed: 1000
- Effective global batch: 64
- Checkpoint: 5,000 global optimizer updates
## Offline 5K results
| Metric | Value |
| --- | ---: |
| H16 MAE | 0.100714 |
| H16 RMSE | 0.208515 |
| Gripper balanced accuracy | 91.61% |
| Boundary-proxy MAE | 0.129451 |
| Interior MAE | 0.095209 |
| Boundary / interior MAE | 1.360x |
These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation.
## Load with LeRobot
```python
from lerobot.policies.diffusion.modeling_diffusion import DiffusionPolicy
policy = DiffusionPolicy.from_pretrained("Stevenshuqing/LW-Bench-Diffusion")
policy.eval()
```
Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete `pretrained_model` directory produced by LeRobot.
## Provenance
- Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench
- Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md
- Code revision: `2d0e26dd2fe2e91046f416bd65bc369f8568d7f0`
- `model.safetensors` SHA256: `e1540ffe39b85bd762b4ab90c52a64d16e031b048957a0d02cd664663395849b`
## Limitations
This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations.
|