Instructions to use OpenRAL/rskill-act-aloha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use OpenRAL/rskill-act-aloha with LeRobot:
- Notebooks
- Google Colab
- Kaggle
chore: publish rSkill OpenRAL/rskill-act-aloha v0.1.0
Browse files- README.md +4 -4
- eval/README.md +1 -1
- eval/aloha_transfer_cube.json +3 -3
- rskill.yaml +49 -18
README.md
CHANGED
|
@@ -58,7 +58,7 @@ wrist or third-person view.
|
|
| 58 |
|
| 59 |
| Field | Value |
|
| 60 |
| --- | --- |
|
| 61 |
-
| `name` | `
|
| 62 |
| `version` | `0.1.0` |
|
| 63 |
| `license` | `mit` |
|
| 64 |
| `role` | `s1` |
|
|
@@ -76,13 +76,13 @@ Full schema: `openral_core.RSkillManifest` β
|
|
| 76 |
## Reproduction
|
| 77 |
|
| 78 |
```bash
|
| 79 |
-
git clone https://github.com/
|
| 80 |
just bootstrap && uv sync --all-packages --group sim
|
| 81 |
|
| 82 |
# End-to-end via the canonical SimEnvironment config:
|
| 83 |
just sim-act-aloha
|
| 84 |
# which runs:
|
| 85 |
-
#
|
| 86 |
|
| 87 |
# Sim test (real gym-aloha MuJoCo with contact dynamics):
|
| 88 |
uv run pytest tests/sim/test_aloha_bimanual_act_aloha.py -v -m sim
|
|
@@ -97,5 +97,5 @@ upstream weights. Commercial use is allowed
|
|
| 97 |
## See also
|
| 98 |
|
| 99 |
- [`robots/aloha_bimanual/README.md`](../../robots/aloha_bimanual/README.md) β RobotDescription manifest.
|
| 100 |
-
- [`
|
| 101 |
- [`docs/reference/vla_compatibility.md`](../../docs/reference/vla_compatibility.md) β VLA Γ Robot Γ Sim matrix.
|
|
|
|
| 58 |
|
| 59 |
| Field | Value |
|
| 60 |
| --- | --- |
|
| 61 |
+
| `name` | `OpenRAL/rskill-act-aloha` |
|
| 62 |
| `version` | `0.1.0` |
|
| 63 |
| `license` | `mit` |
|
| 64 |
| `role` | `s1` |
|
|
|
|
| 76 |
## Reproduction
|
| 77 |
|
| 78 |
```bash
|
| 79 |
+
git clone https://github.com/OpenRAL/openral && cd OpenRAL
|
| 80 |
just bootstrap && uv sync --all-packages --group sim
|
| 81 |
|
| 82 |
# End-to-end via the canonical SimEnvironment config:
|
| 83 |
just sim-act-aloha
|
| 84 |
# which runs:
|
| 85 |
+
# openral sim run --config scenes/benchmarks/act_aloha_transfer_cube.yaml --save-video
|
| 86 |
|
| 87 |
# Sim test (real gym-aloha MuJoCo with contact dynamics):
|
| 88 |
uv run pytest tests/sim/test_aloha_bimanual_act_aloha.py -v -m sim
|
|
|
|
| 97 |
## See also
|
| 98 |
|
| 99 |
- [`robots/aloha_bimanual/README.md`](../../robots/aloha_bimanual/README.md) β RobotDescription manifest.
|
| 100 |
+
- [`scenes/benchmarks/act_aloha_transfer_cube.yaml`](../../scenes/benchmarks/act_aloha_transfer_cube.yaml) β paired SimEnvironment config.
|
| 101 |
- [`docs/reference/vla_compatibility.md`](../../docs/reference/vla_compatibility.md) β VLA Γ Robot Γ Sim matrix.
|
eval/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
| 3 |
`aloha_transfer_cube.json` is the ALOHA bimanual cube-transfer benchmark
|
| 4 |
result block for this rSkill. Validated against
|
| 5 |
[`openral_core.RSkillEvalResult`](../../../docs/reference/schemas/RSkillEvalResult.json)
|
| 6 |
-
at load time by the `rSkill` loader and surfaced by `
|
| 7 |
|
| 8 |
| Field | Value |
|
| 9 |
| --- | --- |
|
|
|
|
| 3 |
`aloha_transfer_cube.json` is the ALOHA bimanual cube-transfer benchmark
|
| 4 |
result block for this rSkill. Validated against
|
| 5 |
[`openral_core.RSkillEvalResult`](../../../docs/reference/schemas/RSkillEvalResult.json)
|
| 6 |
+
at load time by the `rSkill` loader and surfaced by `openral benchmark report`.
|
| 7 |
|
| 8 |
| Field | Value |
|
| 9 |
| --- | --- |
|
eval/aloha_transfer_cube.json
CHANGED
|
@@ -1,13 +1,13 @@
|
|
| 1 |
{
|
| 2 |
-
"schema_version": "1",
|
| 3 |
"source": {
|
| 4 |
"paper": "https://arxiv.org/abs/2304.13705",
|
| 5 |
"arxiv": "https://arxiv.org/abs/2304.13705",
|
| 6 |
"model_variant": "act",
|
| 7 |
-
"evaluated_by": "OpenRAL:
|
| 8 |
"reproduced_locally": true,
|
| 9 |
"reproduction_planned": null,
|
| 10 |
-
"reproduction_cli": "
|
| 11 |
"table": null,
|
| 12 |
"status": "reproduced"
|
| 13 |
},
|
|
|
|
| 1 |
{
|
| 2 |
+
"schema_version": "0.1",
|
| 3 |
"source": {
|
| 4 |
"paper": "https://arxiv.org/abs/2304.13705",
|
| 5 |
"arxiv": "https://arxiv.org/abs/2304.13705",
|
| 6 |
"model_variant": "act",
|
| 7 |
+
"evaluated_by": "OpenRAL:openral benchmark run",
|
| 8 |
"reproduced_locally": true,
|
| 9 |
"reproduction_planned": null,
|
| 10 |
+
"reproduction_cli": "openral benchmark run --suite aloha_transfer_cube --rskill rskill://act-aloha",
|
| 11 |
"table": null,
|
| 12 |
"status": "reproduced"
|
| 13 |
},
|
rskill.yaml
CHANGED
|
@@ -1,23 +1,32 @@
|
|
| 1 |
# rSkill manifest β OpenRAL packaging format V1 (CLAUDE.md Β§6.4)
|
| 2 |
# Wraps: lerobot/act_aloha_sim_transfer_cube_human (MIT)
|
| 3 |
# Paper: Zhao et al., 2023 β Action Chunking Transformer.
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
version: "0.1.0"
|
| 9 |
license: "mit"
|
| 10 |
role: "s1"
|
|
|
|
| 11 |
|
|
|
|
| 12 |
model_family: "act"
|
| 13 |
|
|
|
|
| 14 |
# Bimanual ALOHA (2 Γ 7-DoF arms = 14-DoF action space). Used by
|
| 15 |
# tests/sim/test_aloha_bimanual_act_aloha.py (gym-aloha MuJoCo).
|
| 16 |
embodiment_tags:
|
| 17 |
- "aloha"
|
| 18 |
|
| 19 |
-
capabilities_required: {}
|
| 20 |
-
|
| 21 |
# ACT for ALOHA cube-transfer ships with a single top-down 480Γ640 RGB stream.
|
| 22 |
sensors_required:
|
| 23 |
- modality: "rgb"
|
|
@@ -30,17 +39,25 @@ sensors_required:
|
|
| 30 |
# robots/aloha_bimanual/robot.yaml.
|
| 31 |
actuators_required:
|
| 32 |
- kind: "joint_position"
|
|
|
|
|
|
|
| 33 |
|
|
|
|
| 34 |
runtime: "pytorch"
|
| 35 |
-
|
| 36 |
quantization:
|
| 37 |
dtype: "fp32"
|
| 38 |
backend: "pytorch"
|
| 39 |
-
|
| 40 |
weights_uri: "hf://lerobot/act_aloha_sim_transfer_cube_human"
|
| 41 |
|
| 42 |
-
|
|
|
|
|
|
|
|
|
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
latency_budget:
|
| 45 |
# Reference-host measurement (RTX 4070 Laptop, CUDA 12.8, PyTorch 2.10)
|
| 46 |
# of the warm full-chunk inference is 16 ms; bf16 autocast is ~12 ms.
|
|
@@ -48,20 +65,13 @@ latency_budget:
|
|
| 48 |
# "tolerance_pct=100 β 2Γ ceiling" pattern (giving a 50 ms test ceiling,
|
| 49 |
# matching the previous _WARM_CHUNK_CEILING_S = 0.050).
|
| 50 |
per_chunk_ms: 25.0
|
| 51 |
-
warmup_ms: 5000.0
|
| 52 |
-
load_ms: 10000.0
|
| 53 |
-
|
| 54 |
-
fallback_skill_id: null
|
| 55 |
|
|
|
|
| 56 |
# Headline success rate from skills/act-aloha/eval/aloha_transfer_cube.json
|
| 57 |
-
# (50 episodes via `
|
| 58 |
benchmarks:
|
| 59 |
aloha_transfer_cube: 0.82
|
| 60 |
|
| 61 |
-
# Policy-class identity; ACT manages its own preprocessing / state
|
| 62 |
-
# contract inside the lerobot ACTPolicy so nothing else needs to move.
|
| 63 |
-
policy_id: "act"
|
| 64 |
-
|
| 65 |
paper_url: "https://arxiv.org/abs/2304.13705"
|
| 66 |
source_repo: "hf://lerobot/act_aloha_sim_transfer_cube_human"
|
| 67 |
|
|
@@ -72,3 +82,24 @@ description: >
|
|
| 72 |
PolicyProcessorPipeline migration and ships without normalisation
|
| 73 |
buffers β see tests/sim/test_aloha_bimanual_act_aloha.py for the
|
| 74 |
resulting numerical-contract caveats.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# rSkill manifest β OpenRAL packaging format V1 (CLAUDE.md Β§6.4)
|
| 2 |
# Wraps: lerobot/act_aloha_sim_transfer_cube_human (MIT)
|
| 3 |
# Paper: Zhao et al., 2023 β Action Chunking Transformer.
|
| 4 |
+
#
|
| 5 |
+
# LEGACY PROCESSOR PATH: this checkpoint pre-dates lerobot's
|
| 6 |
+
# PolicyProcessorPipeline migration and ships its norm stats inside
|
| 7 |
+
# model.safetensors. The schema's processors block is therefore omitted;
|
| 8 |
+
# the ACT adapter dispatches on manifest.processors is None and falls
|
| 9 |
+
# back to the snapshot_download + _try_load_act_norm_stats path. Migrating
|
| 10 |
+
# to per-file URIs would require re-publishing the upstream checkpoint
|
| 11 |
+
# and is tracked as a follow-up.
|
| 12 |
+
|
| 13 |
+
# ββ Identity βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 14 |
+
schema_version: "0.1"
|
| 15 |
+
name: "OpenRAL/rskill-act-aloha"
|
| 16 |
version: "0.1.0"
|
| 17 |
license: "mit"
|
| 18 |
role: "s1"
|
| 19 |
+
kind: "vla" # ADR-00XX: rSkill kind discriminator. "vla" = learnable Vision-Language-Action policy.
|
| 20 |
|
| 21 |
+
# ββ Policy identity ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 22 |
model_family: "act"
|
| 23 |
|
| 24 |
+
# ββ Compatibility contract βββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 25 |
# Bimanual ALOHA (2 Γ 7-DoF arms = 14-DoF action space). Used by
|
| 26 |
# tests/sim/test_aloha_bimanual_act_aloha.py (gym-aloha MuJoCo).
|
| 27 |
embodiment_tags:
|
| 28 |
- "aloha"
|
| 29 |
|
|
|
|
|
|
|
| 30 |
# ACT for ALOHA cube-transfer ships with a single top-down 480Γ640 RGB stream.
|
| 31 |
sensors_required:
|
| 32 |
- modality: "rgb"
|
|
|
|
| 39 |
# robots/aloha_bimanual/robot.yaml.
|
| 40 |
actuators_required:
|
| 41 |
- kind: "joint_position"
|
| 42 |
+
control_mode_semantics:
|
| 43 |
+
mode: "absolute"
|
| 44 |
|
| 45 |
+
# ββ Runtime / weights ββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 46 |
runtime: "pytorch"
|
|
|
|
| 47 |
quantization:
|
| 48 |
dtype: "fp32"
|
| 49 |
backend: "pytorch"
|
|
|
|
| 50 |
weights_uri: "hf://lerobot/act_aloha_sim_transfer_cube_human"
|
| 51 |
|
| 52 |
+
# ββ Preprocessing (all knobs needed to interpret IO) βββββββββββββββββββββββ
|
| 53 |
+
# processors omitted β legacy path; norm stats live inside model.safetensors.
|
| 54 |
+
# ACT manages its own preprocessing / state contract inside the lerobot
|
| 55 |
+
# ACTPolicy so nothing else needs to move.
|
| 56 |
|
| 57 |
+
# ββ Execution semantics ββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 58 |
+
chunk_size: 100
|
| 59 |
+
# n_action_steps omitted β ACT default is 1 (per-step re-inference +
|
| 60 |
+
# temporal ensembling, paper-faithful).
|
| 61 |
latency_budget:
|
| 62 |
# Reference-host measurement (RTX 4070 Laptop, CUDA 12.8, PyTorch 2.10)
|
| 63 |
# of the warm full-chunk inference is 16 ms; bf16 autocast is ~12 ms.
|
|
|
|
| 65 |
# "tolerance_pct=100 β 2Γ ceiling" pattern (giving a 50 ms test ceiling,
|
| 66 |
# matching the previous _WARM_CHUNK_CEILING_S = 0.050).
|
| 67 |
per_chunk_ms: 25.0
|
|
|
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
+
# ββ Provenance βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 70 |
# Headline success rate from skills/act-aloha/eval/aloha_transfer_cube.json
|
| 71 |
+
# (50 episodes via `openral benchmark run`).
|
| 72 |
benchmarks:
|
| 73 |
aloha_transfer_cube: 0.82
|
| 74 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
paper_url: "https://arxiv.org/abs/2304.13705"
|
| 76 |
source_repo: "hf://lerobot/act_aloha_sim_transfer_cube_human"
|
| 77 |
|
|
|
|
| 82 |
PolicyProcessorPipeline migration and ships without normalisation
|
| 83 |
buffers β see tests/sim/test_aloha_bimanual_act_aloha.py for the
|
| 84 |
resulting numerical-contract caveats.
|
| 85 |
+
|
| 86 |
+
# ADR-0022 β action vocabulary surfaced to the reasoner LLM tool
|
| 87 |
+
# palette so it can pick this skill by what it does (action verb +
|
| 88 |
+
# object + scene), not just by its slug.
|
| 89 |
+
actions:
|
| 90 |
+
- "transfer"
|
| 91 |
+
- "pick"
|
| 92 |
+
- "place"
|
| 93 |
+
objects:
|
| 94 |
+
- "cube"
|
| 95 |
+
scenes:
|
| 96 |
+
- "tabletop"
|
| 97 |
+
|
| 98 |
+
# ADR-0019 β per-checkpoint action contract (consumed by the dataset bridge
|
| 99 |
+
# to bind the LeRobot v3 `action` feature shape).
|
| 100 |
+
action_contract:
|
| 101 |
+
dim: 14
|
| 102 |
+
|
| 103 |
+
# ADR-0019 β per-checkpoint state contract.
|
| 104 |
+
state_contract:
|
| 105 |
+
dim: 14
|