| --- |
| license: apache-2.0 |
| library_name: lerobot |
| tags: [robotics, lerobot, act, tactile, manipulation] |
| datasets: [aryankakad/tactile_charger_inserting] |
| --- |
| |
| # act_vision_charger_50ep |
| |
| **ACT** policy — **vision only** — trained on **50 episodes**. |
| |
| Task: *grab and remove the charger from the socket and put it in the black box* |
| |
| Trained with [LeFlexiTac](https://github.com/TNA001-AI/lerobot_tactile), a LeRobot fork adding |
| FlexiTac tactile sensing. Docs: <https://tna001-ai.github.io/LeFlexiTac/docs.html> |
| |
| ## Training data |
| |
| | | | |
| |---|---| |
| | dataset | [`aryankakad/tactile_charger_inserting`](https://huggingface.co/datasets/aryankakad/tactile_charger_inserting) | |
| | episodes | 50 (all) | |
| | frames | 27,973 @ 30 fps | |
| | cameras | `observation.images.top`, `observation.images.gripper` (224×224) | |
| | tactile | **not used** — vision-only baseline | |
| | state / action | 6-DoF SO-100 follower | |
| |
| ## Configuration |
| |
| | | | |
| |---|---| |
| | steps | 100,000 | |
| | batch size | 32 | |
| | **epochs** | **114.4** | |
| | `chunk_size` | `100` | |
| | `n_action_steps` | `100` | |
| | `vision_backbone` | `resnet18` | |
| | `dim_model` | `512` | |
| | `n_encoder_layers` | `4` | |
| | `n_decoder_layers` | `1` | |
| | `use_vae` | `True` | |
| | `kl_weight` | `10.0` | |
| | `optimizer_lr` | `1e-05` | |
| | `optimizer_weight_decay` | `0.0001` | |
|
|
| Every model in this series is **epoch-matched at ~114.4 epochs**, so dataset size and |
| sensor modality are the only variables across the set. |
|
|
| ## Training command actually used |
|
|
| Run on 1× AMD Instinct MI300X (ROCm 6.2.4). `HIP_VISIBLE_DEVICES` selected the GPU, |
| so `--policy.device=cuda` refers to that single card. |
|
|
| ```bash |
| python -u -m lerobot.scripts.lerobot_train \ |
| --dataset.repo_id=aryankakad/tactile_charger_inserting \ |
| --policy.type=act \ |
| --policy.repo_id=Dimios45/act_vision_charger_50ep \ |
| --policy.private=true --policy.device=cuda \ |
| --output_dir=outputs/train/A_act_vision_50ep --job_name=A_act_vision_50ep \ |
| --batch_size=32 --num_workers=8 --steps=100000 --save_freq=20000 --wandb.enable=true |
| ``` |
|
|
| ## Evaluation / rollout |
|
|
| Not run here — this machine has no robot attached. To evaluate, run on the machine with the |
| SO-100 and sensors, loading the policy with `--policy.path=Dimios45/act_vision_charger_50ep`. |
|
|
| Reference: the `lerobot-record` eval invocations in |
| [`tactile_cmd.txt`](https://github.com/TNA001-AI/lerobot_tactile/blob/main/tactile_cmd.txt) |
| and the [project docs](https://tna001-ai.github.io/LeFlexiTac/docs.html). You will need to |
| supply your own robot port, camera serials. |
|
|
| ## Notes |
|
|
| - Two ROCm-specific fixes were required in the fork: `persistent_workers=True` on the |
| dataloader (epoch boundaries otherwise stalled ~410 s each), and keeping |
| `cudnn.benchmark` **off** (on ROCm it triggers an exhaustive MIOpen search that can |
| precede step 1 by hours). |
| - Training loss is **not** a proxy for task success. Compare policies by rollout success |
| rate, especially on contact-rich phases. |
| - The source dataset's task string is labelled `stack cup` — a mislabel carried over from |
| an earlier session. It does not affect ACT, which is not language-conditioned. |
|
|
