README.md

---
library_name: physicalai-train
tags:
  - vision-language-action
  - robotics
  - physicalai
license: apache-2.0
---

# Action Chunking Transformer

That's a basic model for solving simplest imitation learning tasks.
The original implementations can be found [here](https://github.com/tonyzhaozh/act/tree/main).

The model takes images from one or multiple cameras and robot state and produces
a chunk of actions, which robot can execute as a sequence of movements in real world.

The model weights are random and provided only for testing purposes.

To fine-tune your model with a custom dataset, you can use [Physical AI Studio](https://github.com/open-edge-platform/physical-ai-studio).

# How to Use

## Installation

```bash
uv pip install physicalai numpy
```

## Running Inference with [OpenVINO Physical AI framework](https://github.com/openvinotoolkit/physicalai).

The following API example showcases inference API for this model:

```python
import numpy as np
from physicalai.inference import InferenceModel

model = InferenceModel("act-fp16-ov", device="CPU")

# Build a dummy LIBERO-style observation.
# LIBERO provides two cameras (agentview + wrist) and an 8-dim robot state.
# Images use the LeRobot convention: float32 in [0, 1], shape (C, H, W).
observation = {
    "images.image": np.random.rand(1, 3, 256, 256).astype(np.float32),
    "images.image2": np.random.rand(1, 3, 256, 256).astype(np.float32),
    "state": np.zeros((1, 8), dtype=np.float32),
}

chunk = model.predict_action_chunk(observation)
```

Note that the model should be downloaded and saved to the `act-fp16-ov` folder prior to running this script.

## Legal information

The original model is distributed under [Apache 2.0](https://choosealicense.com/licenses/apache-2.0/) license.

## Disclaimer

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See [Intel’s Global Human Rights Principles](https://www.intel.com/content/dam/www/central-libraries/us/en/documents/policy-human-rights.pdf). Intel’s products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.