ACT: Optimized for AMD ROCm
ACT (Action Chunking Transformer, Zhao et al.) is a vision-only behavior-cloning policy from HuggingFace LeRobot for 6-DOF robot arm control. There is no language input, no task prompt, and no flow-matching denoiser. This repository packages evaluation/inference for robot arm action prediction using PyTorch, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.
This is based on the implementation of ACT found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.
Task Overview
Task: Robot arm action prediction (behavior cloning)
Dataset: BlankHead/so101_redcube_greencloth_3cams (LeRobot format)
Output metrics: MAE, RMSE (per-joint and per-episode)
Backend note: CPU runs FP32; GPU runs BF16. No NPU (VitisAI) path is available — NPU targets print an informational note and exit cleanly.
AMD ROCm Optimization
This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:
- Validated backends: PyTorch (native ROCm HIP kernels) — CPU (FP32) and GPU (BF16).
- No code changes required versus the upstream ACT/LeRobot implementation — only environment/runtime configuration differs.
- CPU fallback path supported for environments without a ROCm-capable GPU.
- No NPU (VitisAI) path is available for this model.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| CPU | FP32 | PyTorch | AMD CPU | make benchmark-cpu / make evaluate-cpu |
| GPU | BF16 | PyTorch (ROCm) | AMD Instinct™ / Radeon™ GPU | make benchmark-gpu / make evaluate-gpu |
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the Act on GitHub.
Model Details
Model Type: Vision-only behavior-cloning policy (Action Chunking Transformer)
Base Model: (ACT — Action Chunking Transformer, Zhao et al.)
Model Stats:
- Model variant: act-3cams-val (fine-tuned checkpoint, 3-camera SO-101 setup)
- Vision-only input (side, up, wrist cameras) — no language input, no task prompt, no flow-matching denoiser
chunk_size=100,n_action_steps=100,temporal_ensemble_coeff=null(one chunk, all 100 actions consumed, no temporal ensemble)- Number of parameters:
80M - Precision tested: FP32 (CPU), BF16 (GPU)
Performance Summary
Lower MAE/RMSE indicates predicted joint actions more closely match the recorded ground-truth trajectory; both are computed per-joint and averaged across episodes.
Metrics Explained
| Metric | Description |
|---|---|
| MAE | Mean Absolute Error — average absolute difference between predicted and ground-truth joint positions across all timesteps. Lower is better. |
| RMSE | Root Mean Squared Error — penalizes large deviations more heavily than MAE. Lower is better. |
Accuracy Results
Full Dataset Evaluation (BlankHead/so101_redcube_greencloth_3cams, 1 episode, chunked mode, no temporal ensemble) — filled from runs/eval/<dataset_tag>/loss.json; run make evaluate-<device> to refresh:
| Device | Backend | Precision | Variant | Avg MAE | Avg RMSE |
|---|---|---|---|---|---|
| CPU/GPU | PyTorch | FP32/BF16 | act-3cams-val | 1.6267 | 4.4198 |
Per-joint breakdown:
| Joint | Avg MAE | Avg RMSE |
|---|---|---|
| shoulder_pan | 1.6908 | 2.4138 |
| shoulder_lift | 3.6184 | 8.7557 |
| elbow_flex | 2.0769 | 5.3892 |
| wrist_flex | 1.0396 | 1.7437 |
| wrist_roll | 0.7511 | 1.0216 |
| gripper | 0.5832 | 1.2611 |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Scripts for the supported runners (
pipeline.py,evaluate_lerobot.py,benchmark_lerobot.py) - Additional model variants and datasets
- Benchmarking and reproduction instructions