| --- |
| library_name: lerobot |
| license: apache-2.0 |
| tags: |
| - foundation |
| - amd |
| - rocm |
| - robotics |
| pipeline_tag: robotics |
| --- |
| |
|  |
|
|
| # ACT: Optimized for AMD ROCm |
|
|
| ACT (Action Chunking Transformer, Zhao et al.) is a vision-only behavior-cloning policy from HuggingFace LeRobot for 6-DOF robot arm control. There is no language input, no task prompt, and no flow-matching denoiser. This repository packages evaluation/inference for robot arm action prediction using PyTorch, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs and CPUs. |
|
|
| This is based on the implementation of ACT found [here](https://huggingface.co/docs/lerobot/en/act). |
| This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act) to reproduce results or export with custom configurations. More details on model performance can be found [here](#performance-summary). |
|
|
| --- |
|
|
| ## Task Overview |
|
|
| **Task:** Robot arm action prediction (behavior cloning) |
|
|
| **Dataset:** BlankHead/so101_redcube_greencloth_3cams (LeRobot format) |
| |
| **Output metrics:** MAE, RMSE (per-joint and per-episode) |
| |
| > **Backend note:** CPU runs FP32; GPU runs BF16. No NPU (VitisAI) path is available — NPU targets print an informational note and exit cleanly. |
| |
| --- |
| |
| ## AMD ROCm Optimization |
| |
| This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs. Key points: |
| |
| - Validated backends: **PyTorch** (native ROCm HIP kernels) — CPU (FP32) and GPU (BF16). |
| - No code changes required versus the upstream ACT/LeRobot implementation — only environment/runtime configuration differs. |
| - CPU fallback path supported for environments without a ROCm-capable GPU. |
| - No NPU (VitisAI) path is available for this model. |
| |
| | Runtime | Precision | Backend | Hardware | Notes | |
| |---|---|---|---|---| |
| | CPU | FP32 | PyTorch | AMD CPU | `make benchmark-cpu` / `make evaluate-cpu` | |
| | GPU | BF16 | PyTorch (ROCm) | AMD Instinct™ / Radeon™ GPU | `make benchmark-gpu` / `make evaluate-gpu` | |
| |
| --- |
| |
| ## Getting Started |
| |
| For setup instructions, evaluation scripts, and custom configuration options, see the [Act on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act). |
| |
| --- |
| |
| ## Model Details |
| |
| **Model Type:** Vision-only behavior-cloning policy (Action Chunking Transformer) |
| |
| **Base Model:** (ACT — Action Chunking Transformer, Zhao et al.) |
| |
| **Model Stats:** |
| - Model variant: act-3cams-val (fine-tuned checkpoint, 3-camera SO-101 setup) |
| - Vision-only input (side, up, wrist cameras) — no language input, no task prompt, no flow-matching denoiser |
| - `chunk_size=100`, `n_action_steps=100`, `temporal_ensemble_coeff=null` (one chunk, all 100 actions consumed, no temporal ensemble) |
| - Number of parameters: `80M` |
| - Precision tested: FP32 (CPU), BF16 (GPU) |
|
|
| --- |
|
|
| ## Performance Summary |
|
|
| Lower MAE/RMSE indicates predicted joint actions more closely match the recorded ground-truth trajectory; both are computed per-joint and averaged across episodes. |
|
|
| ### Metrics Explained |
|
|
| | Metric | Description | |
| |--------|-------------| |
| | MAE | Mean Absolute Error — average absolute difference between predicted and ground-truth joint positions across all timesteps. Lower is better. | |
| | RMSE | Root Mean Squared Error — penalizes large deviations more heavily than MAE. Lower is better. | |
|
|
| ### Accuracy Results |
|
|
| **Full Dataset Evaluation (BlankHead/so101_redcube_greencloth_3cams, 1 episode, chunked mode, no temporal ensemble)** — filled from `runs/eval/<dataset_tag>/loss.json`; run `make evaluate-<device>` to refresh: |
| |
| <!-- accuracy-table-start --> |
| | Device | Backend | Precision | Variant | Avg MAE | Avg RMSE | |
| |--------|---------|-----------|---------|---------|----------| |
| | CPU/GPU | PyTorch | FP32/BF16 | act-3cams-val | 1.6267 | 4.4198 | |
| <!-- accuracy-table-end --> |
| |
| Per-joint breakdown: |
| |
| | Joint | Avg MAE | Avg RMSE | |
| |-------|---------|----------| |
| | shoulder_pan | 1.6908 | 2.4138 | |
| | shoulder_lift | 3.6184 | 8.7557 | |
| | elbow_flex | 2.0769 | 5.3892 | |
| | wrist_flex | 1.0396 | 1.7437 | |
| | wrist_roll | 0.7511 | 1.0216 | |
| | gripper | 0.5832 | 1.2611 | |
| |
| --- |
| |
| ## Dig Deeper |
| |
| Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples? |
| |
| 📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act)** |
|
|
| The GitHub repository includes: |
| - Setup and prerequisites for ROCm environments |
| - Scripts for the supported runners (`pipeline.py`, `evaluate_lerobot.py`, `benchmark_lerobot.py`) |
| - Additional model variants and datasets |
| - Benchmarking and reproduction instructions |
|
|
| --- |
|
|
|
|
