| --- |
| license: apache-2.0 |
| library_name: transformers |
| base_model: |
| - Qwen/Qwen3.5-9B |
| - Qwen/Qwen3.8-27B |
| pipeline_tag: video-text-to-text |
| tags: |
| - robotics |
| - atomic-actions |
| - temporal-action-segmentation |
| - video |
| --- |
| |
| # RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning |
|
|
|  |
|
|
| **RoboAtom-Anno · 9B and 27B video annotation models** |
|
|
|  |
|
|
| [Models](#models) · [Download](#download) · [Inference](#inference) · [Output and evaluation](#output-and-evaluation) · [License and citation](#license-and-citation) |
|
|
| ## Overview |
|
|
| RoboAtom-Anno predicts atomic actions from robot videos and task instructions. Each predicted segment contains a **skill label**, an **action description**, and **frame boundaries**. The release provides merged 9B and 27B weights, with tokenizers, video processors, and generation settings. |
|
|
| ## Models |
|
|
|
|
| | Model | Weights | Architecture | Precision | |
| | --------------------- | ------------------------------------------------------------------------------------------------ | --------------------------- | --------- | |
| | **RoboAtom-Anno-9B** | [RoboAtom-Anno-9B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-9B) | Fine-tuned from Qwen3.5-9B | BF16 | |
| | **RoboAtom-Anno-27B** | [RoboAtom-Anno-27B/](https://huggingface.co/InternRobotics/RoboAtom/tree/main/RoboAtom-Anno-27B) | Fine-tuned from Qwen3.8-27B | BF16 | |
|
|
|
|
| Use Python **3.11+**, `transformers==5.8.0`, and compatible CUDA-enabled PyTorch. The [RoboAtom inference implementation](https://github.com/InternRobotics/RoboAtom#quick-start-benchmark) handles synchronized camera composition, frame sampling, and frame-index prompts. GPU requirements depend on the model and video input. |
|
|
| ## Download |
|
|
| These examples use `./model` for weights and `./data` for the benchmark: |
|
|
| ```bash |
| hf download InternRobotics/RoboAtom --repo-type model \ |
| --include 'RoboAtom-Anno-9B/**' --local-dir ./model |
| hf download InternRobotics/RoboAtom --repo-type dataset \ |
| --include 'RoboAtom-Bench/**' --local-dir ./data |
| ``` |
|
|
| Replace `RoboAtom-Anno-9B` with `RoboAtom-Anno-27B` to select the larger model. The Hugging Face CLI (`hf`) is required for these download commands. |
|
|
| ## Inference |
|
|
|
|
|
|
| ### 1. Install the code |
|
|
| From your [RoboAtom code checkout](https://github.com/InternRobotics/RoboAtom): |
|
|
| ```bash |
| python3.11 -m venv .venv |
| source .venv/bin/activate |
| python -m pip install -e '.[inference]' |
| ``` |
|
|
|
|
|
|
| ### 2. Start the model service |
|
|
| The following paths match the download layout above; replace them if you chose another location: |
|
|
| ```bash |
| roboatom serve \ |
| --model ./model/RoboAtom-Anno-9B \ |
| --media-root ./data/RoboAtom-Bench |
| ``` |
|
|
| The service listens on `http://127.0.0.1:3000`. Change the model directory to select the 27B weights. |
|
|
| ### 3. Generate predictions |
|
|
| In another terminal with the same environment activated: |
|
|
| ```bash |
| roboatom infer \ |
| --input ./data/RoboAtom-Bench/benchmark.jsonl \ |
| --output outputs/roboatom_predictions.jsonl \ |
| --base-url http://127.0.0.1:3000 |
| ``` |
|
|
| Choose a new output filename for each run; the inference client refuses to overwrite an existing prediction file. |
|
|
|
|
| | Setting | Released inference behavior | |
| | --------------- | ------------------------------------------------------------------- | |
| | Sampling | Uniform, target 2 FPS, including both endpoints, at most 448 frames | |
| | Precision | BF16 | |
| | Generation | Deterministic, at most 4,096 new tokens | |
| | Time convention | Original-video frame indices, zero-based, both endpoints included | |
|
|
|
|
|
|
|
|
| ## Output and evaluation |
|
|
| The client writes one prediction record per benchmark input, with an identity key and ordered action segments. A segment covers `start_frame` through `end_frame` inclusively; frames 0–74 contain 75 frames. Keep ground truth separate from model inputs. |
|
|
| From the code checkout, score the predictions: |
|
|
| ```bash |
| roboatom evaluate \ |
| --gt ./data/RoboAtom-Bench/gt.jsonl \ |
| --benchmark-input ./data/RoboAtom-Bench/benchmark.jsonl \ |
| --prediction outputs/roboatom_predictions.jsonl \ |
| --protocol configs/roboatom_bench.yaml \ |
| --output-dir outputs/roboatom_evaluation |
| ``` |
|
|
| The scorer writes `metrics.json`, `summary.tsv`, and `episodes.jsonl`. See the [benchmark card](https://huggingface.co/datasets/InternRobotics/RoboAtom) for data fields and the [code README](https://github.com/InternRobotics/RoboAtom#evaluation) for the evaluation protocol. |
|
|
| ## License and citation |
|
|
| The model directories contain the [9B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-9B/LICENSE) and [27B Apache-2.0 license](https://huggingface.co/InternRobotics/RoboAtom/blob/main/RoboAtom-Anno-27B/LICENSE). See the project's [third-party notices](https://github.com/InternRobotics/RoboAtom/blob/main/THIRD_PARTY_NOTICES.md) for component attribution and [citation file](https://github.com/InternRobotics/RoboAtom/blob/main/CITATION.cff) for publication metadata. |
|
|
| If you use RoboAtom in your research, please cite: |
|
|
| ```bibtex |
| @misc{wei2026roboatomlargescaleatomicactionsupervision, |
| title={RoboAtom: Large-scale Atomic-Action Supervision for Robot Learning}, |
| author={Xueyuan Wei and Xuanxu Wang and Xudong Li and Yuping He and Wanshui Gan and Jiaqi Peng and Wenqi Guo and Jianjun Zhou and Junting Dong and Chunhua Shen and Haoyu Guo and Mulin Yu and Tao Lu}, |
| year={2026}, |
| eprint={2610.40207}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.RO}, |
| url={https://arxiv.org/abs/2610.40207}, |
| } |
| ``` |
|
|
|
|
