---
library_name: lerobot
pipeline_tag: robotics
tags:
- lerobot
- robotics
- imitation-learning
- x7s
- smolvla
- vision-language-action
- offline-evaluation
datasets:
- LightwheelAI/Lightwheel-Tasks-X7S
base_model:
- lerobot/smolvla_base
---

# LW-Bench SmolVLA 5K for X7s

LeRobot SmolVLA policy fine-tuned from `lerobot/smolvla_base` on the audited X7s strict compositional split in LW-Compositional-Bench.

## Training contract

- Dataset: `LightwheelAI/Lightwheel-Tasks-X7S`
- Train split: 91 clean tasks; tasks 7, 119, and 149 held out
- Inputs: three RGB cameras, 25D robot state, and canonical task language
- Output: native 21D X7s absolute action
- Hardware: 4 GPUs
- Seed: 1000
- Effective global batch: 64
- Checkpoint: 5,000 global optimizer updates
- Evaluation horizon: first 16 valid action steps

## Offline 5K results

| Metric | Value |
| --- | ---: |
| H16 MAE | 0.117617 |
| H16 RMSE | 0.227890 |
| Gripper balanced accuracy | 87.08% |
| Boundary-proxy MAE | 0.147732 |
| Interior MAE | 0.111848 |
| Boundary / interior MAE | 1.321x |

These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation.

## Load with LeRobot

```python
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy

policy = SmolVLAPolicy.from_pretrained("Stevenshuqing/LW-Bench-SmolVLA")
policy.eval()
```

Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete `pretrained_model` directory produced by LeRobot.

## Provenance

- Base model: https://huggingface.co/lerobot/smolvla_base
- Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench
- Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md
- Code revision: `2d0e26dd2fe2e91046f416bd65bc369f8568d7f0`
- `model.safetensors` SHA256: `f70959474aad109134f799e49dd313abf209380c8e1c808d8650e6d92287e242`

## Limitations

This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations.