| --- |
| license: apache-2.0 |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - lerobot |
| - act |
| - bimanual |
| - deformable-object-manipulation |
| - cloth |
| --- |
| |
| # ACT — flatten_tshirt (bimanual cloth flattening) |
| |
| An [ACT](https://arxiv.org/abs/2304.13705) policy trained on the `flatten_tshirt` task of a |
| bimanual deformable-object (cloth / bag) manipulation benchmark. The robot is a dual-arm |
| Piper; the task is to flatten a crumpled t-shirt on a table. |
|
|
| Simulation uses a GPU cloth solver co-simulated with the robot in a single model, and |
| observations are rendered with a photorealistic renderer. |
|
|
| ## Model |
|
|
| | | | |
| |---|---| |
| | Architecture | ACT, ResNet-18 vision backbone | |
| | Observation | 3 × RGB `720×1280` (`static_cam`, `left_hand_cam`, `right_hand_cam`) + 14-D joint state | |
| | Action | 14-D (left 6 joints + gripper, right 6 joints + gripper) | |
| | Chunk size / action steps | 100 / 100 | |
| | `n_obs_steps` | 1 | |
|
|
| ## Training |
|
|
| | | | |
| |---|---| |
| | Dataset | `flatten_tshirt_200` — 200 episodes / 41,464 frames, LeRobot v3.0, 25 fps | |
| | Steps | 30,000 | |
| | Batch size | 16 (single A100-80G) | |
| | Learning rate | 1e-5 | |
| | Seed | 1000 | |
| | Image augmentation | enabled, max 3 random transforms per sample | |
|
|
| Augmentation follows a tuned recipe (brightness / contrast / saturation / hue / sharpness / |
| small affine) rather than the LeRobot default, which is disabled. The brightness range is |
| deliberately asymmetric toward the darker side: simulation lighting is idealized while real |
| RealSense D435i footage tends to be darker, so biasing the augmentation toward darker samples |
| is the right direction for sim-to-real transfer. |
|
|
| ## Status |
|
|
| ⚠️ **This checkpoint has not yet been formally evaluated.** It has only been through a |
| small smoke-level closed-loop run, not the benchmark's standard N=100 protocol. Success-rate |
| numbers are deliberately not published here yet; they will be added once the full evaluation |
| has been run. Treat this as a training artifact, not a reported result. |
|
|
| ## Usage |
|
|
| ```python |
| from lerobot.policies.act.modeling_act import ACTPolicy |
| |
| policy = ACTPolicy.from_pretrained("hwk0809/act-flatten-tshirt") |
| ``` |
|
|
| The policy expects the three camera streams named exactly as listed above, plus a 14-D |
| `observation.state`, and returns a 14-D action. It runs in-process (no policy server needed). |
|
|
| ## License |
|
|
| Apache-2.0. |
|
|
