File size: 6,447 Bytes
3a2d185 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 | ---
license: apache-2.0
library_name: transformers
base_model:
- Qwen/Qwen3.5-9B
- Qwen/Qwen3.8-27B
pipeline_tag: video-text-to-text
tags:
- robotics
- atomic-actions
- temporal-action-segmentation
- video
---
# RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning
**RoboAtom-Anno · 9B and 27B video annotation models**
[Models](#models) · [Download](#download) · [Inference](#inference) · [Output and evaluation](#output-and-evaluation) · [License and citation](#license-and-citation)
## Overview
RoboAtom-Anno predicts atomic actions from robot videos and task instructions. Each predicted segment contains a **skill label**, an **action description**, and **frame boundaries**. The release provides merged 9B and 27B weights, with tokenizers, video processors, and generation settings.
## Models
| Model | Weights | Architecture | Precision |
| --------------------- | ------------------------------------------------------------------------------------------------ | --------------------------- | --------- |
| **RoboAtom-Anno-9B** | [RoboAtom-Anno-9B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-9B) | Fine-tuned from Qwen3.5-9B | BF16 |
| **RoboAtom-Anno-27B** | [RoboAtom-Anno-27B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-27B) | Fine-tuned from Qwen3.8-27B | BF16 |
Use Python **3.11+**, `transformers==5.8.0`, and compatible CUDA-enabled PyTorch. The [RoboAtom inference implementation](https://github.com/InternRobotics/RoboAtom#quick-start-benchmark) handles synchronized camera composition, frame sampling, and frame-index prompts. GPU requirements depend on the model and video input.
## Download
These examples use `./model` for weights and `./data` for the benchmark:
```bash
hf download InternRobotics/RoboAtom --repo-type model \
--include 'RoboAtom-Anno-9B/**' --local-dir ./model
hf download InternRobotics/RoboAtom --repo-type dataset \
--include 'RoboAtom-Bench/**' --local-dir ./data
```
Replace `RoboAtom-Anno-9B` with `RoboAtom-Anno-27B` to select the larger model. The Hugging Face CLI (`hf`) is required for these download commands.
## Inference
### 1. Install the code
From your [RoboAtom code checkout](https://github.com/InternRobotics/RoboAtom):
```bash
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install -e '.[inference]'
```
### 2. Start the model service
The following paths match the download layout above; replace them if you chose another location:
```bash
roboatom serve \
--model ./model/RoboAtom-Anno-9B \
--media-root ./data/RoboAtom-Bench
```
The service listens on `http://127.0.0.1:3000`. Change the model directory to select the 27B weights.
### 3. Generate predictions
In another terminal with the same environment activated:
```bash
roboatom infer \
--input ./data/RoboAtom-Bench/benchmark.jsonl \
--output outputs/roboatom_predictions.jsonl \
--base-url http://127.0.0.1:3000
```
Choose a new output filename for each run; the inference client refuses to overwrite an existing prediction file.
| Setting | Released inference behavior |
| --------------- | ------------------------------------------------------------------- |
| Sampling | Uniform, target 2 FPS, including both endpoints, at most 448 frames |
| Precision | BF16 |
| Generation | Deterministic, at most 4,096 new tokens |
| Time convention | Original-video frame indices, zero-based, both endpoints included |
## Output and evaluation
The client writes one prediction record per benchmark input, with an identity key and ordered action segments. A segment covers `start_frame` through `end_frame` inclusively; frames 0–74 contain 75 frames. Keep ground truth separate from model inputs.
From the code checkout, score the predictions:
```bash
roboatom evaluate \
--gt ./data/RoboAtom-Bench/gt.jsonl \
--benchmark-input ./data/RoboAtom-Bench/benchmark.jsonl \
--prediction outputs/roboatom_predictions.jsonl \
--protocol configs/roboatom_bench.yaml \
--output-dir outputs/roboatom_evaluation
```
The scorer writes `metrics.json`, `summary.tsv`, and `episodes.jsonl`. See the [benchmark card](https://huggingface.co/datasets/InternRobotics/RoboAtom) for data fields and the [code README](https://github.com/InternRobotics/RoboAtom#evaluation) for the evaluation protocol.
## License and citation
The model directories contain the [9B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-9B/LICENSE) and [27B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-27B/LICENSE). See the project's [third-party notices](https://github.com/InternRobotics/RoboAtom/blob/main/THIRD_PARTY_NOTICES.md) for component attribution and [citation file](https://github.com/InternRobotics/RoboAtom/blob/main/CITATION.cff) for publication metadata.
If you use RoboAtom in your research, please cite:
```bibtex
@misc{wei2026roboatomlargescaleatomicactionsupervision,
title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning},
author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu},
year={2026},
eprint={2610.40207},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2610.40207},
}
```
|