README.md

---
license: apache-2.0
library_name: lerobot
pipeline_tag: robotics
tags:
  - robotics
  - lerobot
  - act
  - bimanual
  - deformable-object-manipulation
  - cloth
---

# ACT — flatten_tshirt (bimanual cloth flattening)

An [ACT](https://arxiv.org/abs/2304.13705) policy trained on the `flatten_tshirt` task of a
bimanual deformable-object (cloth / bag) manipulation benchmark. The robot is a dual-arm
Piper; the task is to flatten a crumpled t-shirt on a table.

Simulation uses a GPU cloth solver co-simulated with the robot in a single model, and
observations are rendered with a photorealistic renderer.

## Model

| | |
|---|---|
| Architecture | ACT, ResNet-18 vision backbone |
| Observation | 3 × RGB `720×1280` (`static_cam`, `left_hand_cam`, `right_hand_cam`) + 14-D joint state |
| Action | 14-D (left 6 joints + gripper, right 6 joints + gripper) |
| Chunk size / action steps | 100 / 100 |
| `n_obs_steps` | 1 |

## Training

| | |
|---|---|
| Dataset | `flatten_tshirt_200` — 200 episodes / 41,464 frames, LeRobot v3.0, 25 fps |
| Steps | 30,000 |
| Batch size | 16 (single A100-80G) |
| Learning rate | 1e-5 |
| Seed | 1000 |
| Image augmentation | enabled, max 3 random transforms per sample |

Augmentation follows a tuned recipe (brightness / contrast / saturation / hue / sharpness /
small affine) rather than the LeRobot default, which is disabled. The brightness range is
deliberately asymmetric toward the darker side: simulation lighting is idealized while real
RealSense D435i footage tends to be darker, so biasing the augmentation toward darker samples
is the right direction for sim-to-real transfer.

## Status

⚠️ **This checkpoint has not yet been formally evaluated.** It has only been through a
small smoke-level closed-loop run, not the benchmark's standard N=100 protocol. Success-rate
numbers are deliberately not published here yet; they will be added once the full evaluation
has been run. Treat this as a training artifact, not a reported result.

## Usage

```python
from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("hwk0809/act-flatten-tshirt")
```

The policy expects the three camera streams named exactly as listed above, plus a 14-D
`observation.state`, and returns a 14-D action. It runs in-process (no policy server needed).

## License

Apache-2.0.