---
library_name: lerobot
license: apache-2.0
pipeline_tag: robotics
tags:
- lerobot
- robotics
- act
- robosynchallenge
- handle_basket
datasets: []
model_name: act
---

# ACT_sim_handle_basket (RoboSynChallenge submission checkpoint)

ACT (Action Chunking with Transformers) checkpoint for the RoboSynChallenge
**`handle_basket`** simulation task, re-packaged in the **LeRobot 0.4.4** layout so
that it loads without any legacy code path.

This checkpoint is the imitation-learning component of our preliminary-round
submission. It is used both

* as a stand-alone ACT baseline for `handle_basket`, and
* as the action-chunk generator that the submitted hybrid policy hands over to
  after its analytic grasp/place stage.

## Provenance

| Item | Value |
| --- | --- |
| Upstream weights | [`RoboSynChallenge/ACT_sim_handle_basket`](https://huggingface.co/RoboSynChallenge/ACT_sim_handle_basket) |
| Upstream format | legacy LeRobot `b883328` (normalization buffers stored inside `model.safetensors`) |
| This repository | same weights, migrated to the LeRobot 0.4.4 external pre/post-processor layout |
| Migration tool | `lerobot.processor.migrate_policy_normalization` with a key-restoration pass (see below) |

The migration is **format-only**: no weight is retrained, re-initialised or
fine-tuned. `model.safetensors` is byte-for-byte the upstream tensor payload.

### Why a key-restoration pass was necessary

LeRobot's official migrator renames feature keys with
`".".join(parts[:-1]).replace("_", ".")`, which turns
`observation_images_cam_high` into `observation.images.cam.high` instead of
`observation.images.cam_high`. Those keys never match the policy features, so
the **image branch would silently stay un-normalised** (a mid-grey image stays
`0.5` instead of becoming roughly `0.066` after ImageNet normalisation).

The migrator was therefore followed by a pass that rewrites the state-file keys
back to the real feature names declared in `config.json`, and by an assertion
that normalisation actually applies:

```
image 0.5 -> ~0.066   (must NOT be ~0.5)
```

If you re-derive these files yourself, please re-run that assertion — a
checkpoint that looks fine can score 0 % purely because of this key mismatch.

## Files

| File | Size | Role |
| --- | ---: | --- |
| `model.safetensors` | 206,560,504 B | ACT weights (upstream payload) |
| `config.json` | 2,117 B | ACT policy config, `chunk_size=50`, `n_action_steps=50` |
| `policy_preprocessor.json` | 1,780 B | LeRobot 0.4.4 preprocessing pipeline |
| `policy_preprocessor_step_3_normalizer_processor.safetensors` | 1,704 B | normalisation statistics |
| `policy_postprocessor.json` | 661 B | LeRobot 0.4.4 postprocessing pipeline |
| `policy_postprocessor_step_0_unnormalizer_processor.safetensors` | 1,704 B | de-normalisation statistics |

## Interface

| | |
| --- | --- |
| Observation state | `observation.state`, shape `[14]` (`obs["robot"]["qpos"]`) |
| Observation images | `observation.images.cam_high`, `observation.images.cam_right_wrist`, `observation.images.cam_left_wrist`, each `[3, 480, 640]`, `float32` in `[0, 1]` |
| Action | `action`, shape `[14]`: `[left_arm(0-5), left_gripper(6), right_arm(7-12), right_gripper(13)]` |
| Action type | absolute joint positions |
| Chunk size | `50` predicted, `50` executed per inference (`n_action_steps`) |

## Usage

```python
from lerobot.policies.act.modeling_act import ACTPolicy
from lerobot.policies.factory import make_pre_post_processors

policy = ACTPolicy.from_pretrained("<this-repo>")
pre, post = make_pre_post_processors(
    policy_cfg=policy.config,
    pretrained_path="<this-repo>",
    preprocessor_overrides={"device_processor": {"device": policy.config.device}},
    postprocessor_overrides={"device_processor": {"device": "cpu"}},
)
batch = {
    "observation.state": state,                 # [1, 14] float32
    "observation.images.cam_high": img_high,    # [1, 3, 480, 640] float32 in [0, 1]
    "observation.images.cam_right_wrist": img_rw,
    "observation.images.cam_left_wrist": img_lw,
}
action = post(policy.select_action(pre(batch)))  # [1, 14]
```

A ready-made adapter that handles both this 0.4.4 layout and the legacy
`b883328` layout is shipped in the submission repository alongside this card.

## Evaluation protocol

* Task: `handle_basket`, setting `random` (domain randomisation on)
* Episode limit: 500 environment steps (`configs/handle_basket/random/gym_config.json`)
* Success criterion (official): the milk carton is inside the basket, the basket
  is lifted and displaced more than 0.15 m along `+y`, and all of that holds for
  55 consecutive steps.

## Limitations

* Simulation only. This checkpoint was trained on the released simulated
  `handle_basket` demonstrations; it is not a real-robot policy.
* It is **not** a stand-alone solution for the whole task. On its own it
  performs the grasp/place/carry sequence at the base rate of the released ACT
  baseline; the submitted hybrid policy adds an analytic, RGB-only
  grasp-and-place stage in front of it.
* Training data, seeds and hyper-parameters are those of the upstream released
  checkpoint; only the on-disk format was changed here.