| ---
|
| library_name: physicalai-train
|
| tags:
|
| - robotics-ai-suite
|
| - vision-language-action
|
| - robotics
|
| - physicalai
|
| - physical-ai
|
| - "chipset:ptl"
|
| license: apache-2.0
|
| title: "Action Chunking Transformer"
|
| subtitle: "Baseline imitation-learning policy"
|
| category: "Physical AI"
|
| model_type:
|
| - "Imitation Learning"
|
| primary_type: "Vision Language Action"
|
| secondary_types:
|
| - Robotics
|
| related_models:
|
| - pi05-libero-fp16-ov-catalog
|
| - smolvla-libero-fp16-ov-catalog
|
| key_novelty: "Baseline imitation-learning model that maps camera images and robot state to an executable chunk of actions for the simplest imitation tasks."
|
| thumbnail: assets/thumbnail.png
|
| image_overview: assets/architecture-overview.svg
|
| image_detailed: assets/architecture-detailed.svg
|
| code: "https://github.com/openvinotoolkit/physicalai"
|
| ---
|
|
|
| # Action Chunking Transformer
|
|
|
| That's a basic model for solving simplest imitation learning tasks.
|
| The original implementations can be found [here](https://github.com/tonyzhaozh/act/tree/main).
|
|
|
| The model takes images from one or multiple cameras and robot state and produces
|
| a chunk of actions, which robot can execute as a sequence of movements in real world.
|
|
|
| The model weights are random and provided only for testing purposes.
|
|
|
| To fine-tune your model with a custom dataset, you can use [Physical AI Studio](https://github.com/open-edge-platform/physical-ai-studio).
|
|
|
| # How to Use
|
|
|
| ## Installation
|
|
|
| ```bash
|
| uv pip install physicalai numpy
|
| ```
|
|
|
| ## Running Inference with [OpenVINO Physical AI framework](https://github.com/openvinotoolkit/physicalai).
|
|
|
| The following API example showcases inference API for this model:
|
|
|
| ```python
|
| import numpy as np
|
| from physicalai.inference import InferenceModel
|
|
|
| model = InferenceModel("act-fp16-ov", device="CPU")
|
|
|
| # Build a dummy LIBERO-style observation.
|
| # LIBERO provides two cameras (agentview + wrist) and an 8-dim robot state.
|
| # Images use the LeRobot convention: float32 in [0, 1], shape (C, H, W).
|
| observation = {
|
| "images.image": np.random.rand(1, 3, 256, 256).astype(np.float32),
|
| "images.image2": np.random.rand(1, 3, 256, 256).astype(np.float32),
|
| "state": np.zeros((1, 8), dtype=np.float32),
|
| }
|
|
|
| chunk = model.predict_action_chunk(observation)
|
| ```
|
|
|
| Note that the model should be downloaded and saved to the `act-fp16-ov` folder prior to running this script.
|
|
|
| ## Legal information
|
|
|
| The original model is distributed under [Apache 2.0](https://choosealicense.com/licenses/apache-2.0/) license.
|
|
|
| ## Disclaimer
|
|
|
| Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See [Intel’s Global Human Rights Principles](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf). Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.
|
|
|
