File size: 2,972 Bytes
d41de60 efaedac d41de60 efaedac d41de60 3b58c02 d41de60 2e46a2b d41de60 3b58c02 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
library_name: physicalai-train
tags:
- robotics-ai-suite
- vision-language-action
- robotics
- physicalai
- physical-ai
- "chipset:ptl"
license: apache-2.0
title: "Action Chunking Transformer"
subtitle: "Baseline imitation-learning policy"
category: "Physical AI"
model_type:
- "Imitation Learning"
primary_type: "Vision Language Action"
secondary_types:
- Robotics
related_models:
- pi05-libero-fp16-ov-catalog
- smolvla-libero-fp16-ov-catalog
key_novelty: "Baseline imitation-learning model that maps camera images and robot state to an executable chunk of actions for the simplest imitation tasks."
thumbnail: assets/thumbnail.png
image_overview: assets/architecture-overview.svg
image_detailed: assets/architecture-detailed.svg
code: "https://github.com/openvinotoolkit/physicalai"
---
# Action Chunking Transformer
That's a basic model for solving simplest imitation learning tasks.
The original implementations can be found [here](https://github.com/tonyzhaozh/act/tree/main).
The model takes images from one or multiple cameras and robot state and produces
a chunk of actions, which robot can execute as a sequence of movements in real world.
The model weights are random and provided only for testing purposes.
To fine-tune your model with a custom dataset, you can use [Physical AI Studio](https://github.com/open-edge-platform/physical-ai-studio).
# How to Use
## Installation
```bash
uv pip install physicalai numpy
```
## Running Inference with [OpenVINO Physical AI framework](https://github.com/openvinotoolkit/physicalai).
The following API example showcases inference API for this model:
```python
import numpy as np
from physicalai.inference import InferenceModel
model = InferenceModel("act-fp16-ov", device="CPU")
# Build a dummy LIBERO-style observation.
# LIBERO provides two cameras (agentview + wrist) and an 8-dim robot state.
# Images use the LeRobot convention: float32 in [0, 1], shape (C, H, W).
observation = {
"images.image": np.random.rand(1, 3, 256, 256).astype(np.float32),
"images.image2": np.random.rand(1, 3, 256, 256).astype(np.float32),
"state": np.zeros((1, 8), dtype=np.float32),
}
chunk = model.predict_action_chunk(observation)
```
Note that the model should be downloaded and saved to the `act-fp16-ov` folder prior to running this script.
## Legal information
The original model is distributed under [Apache 2.0](https://choosealicense.com/licenses/apache-2.0/) license.
## Disclaimer
Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See [Intel’s Global Human Rights Principles](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf). Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.
|