File size: 4,779 Bytes
04448a3 c33e267 04448a3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | ---
library_name: lerobot
license: apache-2.0
tags:
- foundation
- amd
- rocm
- robotics
pipeline_tag: robotics
---
# ACT: Optimized for AMD ROCm
ACT (Action Chunking Transformer, Zhao et al.) is a vision-only behavior-cloning policy from HuggingFace LeRobot for 6-DOF robot arm control. There is no language input, no task prompt, and no flow-matching denoiser. This repository packages evaluation/inference for robot arm action prediction using PyTorch, exported and validated for **AMD ROCm** so it runs efficiently on AMD GPUs and CPUs.
This is based on the implementation of ACT found [here](https://huggingface.co/docs/lerobot/en/act).
This repository contains configurations and scripts optimized for **AMD® ROCm™** platforms. You can use the [AMD scripts](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act) to reproduce results or export with custom configurations. More details on model performance can be found [here](#performance-summary).
---
## Task Overview
**Task:** Robot arm action prediction (behavior cloning)
**Dataset:** BlankHead/so101_redcube_greencloth_3cams (LeRobot format)
**Output metrics:** MAE, RMSE (per-joint and per-episode)
> **Backend note:** CPU runs FP32; GPU runs BF16. No NPU (VitisAI) path is available — NPU targets print an informational note and exit cleanly.
---
## AMD ROCm Optimization
This model export has been adapted and validated for **AMD Instinct™ / Radeon™ GPUs** running **ROCm**, as well as AMD CPUs. Key points:
- Validated backends: **PyTorch** (native ROCm HIP kernels) — CPU (FP32) and GPU (BF16).
- No code changes required versus the upstream ACT/LeRobot implementation — only environment/runtime configuration differs.
- CPU fallback path supported for environments without a ROCm-capable GPU.
- No NPU (VitisAI) path is available for this model.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| CPU | FP32 | PyTorch | AMD CPU | `make benchmark-cpu` / `make evaluate-cpu` |
| GPU | BF16 | PyTorch (ROCm) | AMD Instinct™ / Radeon™ GPU | `make benchmark-gpu` / `make evaluate-gpu` |
---
## Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the [Act on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act).
---
## Model Details
**Model Type:** Vision-only behavior-cloning policy (Action Chunking Transformer)
**Base Model:** (ACT — Action Chunking Transformer, Zhao et al.)
**Model Stats:**
- Model variant: act-3cams-val (fine-tuned checkpoint, 3-camera SO-101 setup)
- Vision-only input (side, up, wrist cameras) — no language input, no task prompt, no flow-matching denoiser
- `chunk_size=100`, `n_action_steps=100`, `temporal_ensemble_coeff=null` (one chunk, all 100 actions consumed, no temporal ensemble)
- Number of parameters: `80M`
- Precision tested: FP32 (CPU), BF16 (GPU)
---
## Performance Summary
Lower MAE/RMSE indicates predicted joint actions more closely match the recorded ground-truth trajectory; both are computed per-joint and averaged across episodes.
### Metrics Explained
| Metric | Description |
|--------|-------------|
| MAE | Mean Absolute Error — average absolute difference between predicted and ground-truth joint positions across all timesteps. Lower is better. |
| RMSE | Root Mean Squared Error — penalizes large deviations more heavily than MAE. Lower is better. |
### Accuracy Results
**Full Dataset Evaluation (BlankHead/so101_redcube_greencloth_3cams, 1 episode, chunked mode, no temporal ensemble)** — filled from `runs/eval/<dataset_tag>/loss.json`; run `make evaluate-<device>` to refresh:
<!-- accuracy-table-start -->
| Device | Backend | Precision | Variant | Avg MAE | Avg RMSE |
|--------|---------|-----------|---------|---------|----------|
| CPU/GPU | PyTorch | FP32/BF16 | act-3cams-val | 1.6267 | 4.4198 |
<!-- accuracy-table-end -->
Per-joint breakdown:
| Joint | Avg MAE | Avg RMSE |
|-------|---------|----------|
| shoulder_pan | 1.6908 | 2.4138 |
| shoulder_lift | 3.6184 | 8.7557 |
| elbow_flex | 2.0769 | 5.3892 |
| wrist_flex | 1.0396 | 1.7437 |
| wrist_roll | 0.7511 | 1.0216 |
| gripper | 0.5832 | 1.2611 |
---
## Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 **[View the full project on GitHub](https://github.com/AMD-PAVS/physical_ai_sdk/blob/main/models/Act)**
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for the supported runners (`pipeline.py`, `evaluate_lerobot.py`, `benchmark_lerobot.py`)
- Additional model variants and datasets
- Benchmarking and reproduction instructions
---
|