| --- |
| license: other |
| license_name: see-model-card |
| license_link: https://huggingface.co/LogosRoboticsLab/roboreel_models#licensing |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - imitation-learning |
| - learning-from-observation |
| - manipulation |
| - benchmark |
| --- |
| |
| # RoboReel β evaluated policy checkpoints |
|
|
| Policy checkpoints for **"Monkey See, Can Monkey Do? A Benchmark for Evaluating |
| Robot Skill Learning by Observation"** ([arXiv:2609.08209](https://arxiv.org/abs/2609.08209)), |
| benchmark **RoboReel** Β· [project page](https://roboreel.github.io). |
|
|
| Each folder holds **exactly the checkpoint the paper's numbers were computed |
| from** β not the last checkpoint a run happened to write. Training code, |
| evaluation harness and configs live in the `roboreel_models` repository; the |
| simulation environments live in the benchmark environment repository. |
|
|
| ## Layout |
|
|
| ``` |
| <condition>/<method>/ |
| βββ train_config.json # the run's own config (see "Paths" below) |
| βββ <sidecar> # norm_stats.json | action_codec.json | demonstration_config.txt |
| βββ ckpt/ | ckpt.pt | ckpt.pth # the evaluated checkpoint |
| ``` |
|
|
| Every method's weights are called **`ckpt`** whichever trainer wrote them β a |
| directory for the JAX/orbax and LeRobot policies, a single file for Vid2Robot |
| (`ckpt.pt`) and UniSkill (`ckpt.pth`). |
|
|
| `<condition>` is the training suite, and it is the **top** level: **`nd/`** = no |
| distraction (clean scenes), **`wd/`** = with distraction (distractors in both |
| train and eval). So the keyframe policy trained with distractors is |
| `wd/vid_cond_pi05_keyframes/ckpt/`. |
|
|
| The paper's third suite, **ED** (clean training, distracted evaluation), needs |
| no extra weights β it evaluates everything under `nd/` against the WD scene |
| bank. |
|
|
| ## Checkpoints |
|
|
| | Paper label | Method dir | Weights | Size each | |
| |---|---|---|---| |
| | VLM + Ο0.5 | `vlm_pi05` | `ckpt/` | 9.35 GB | |
| | Vid. Cond. Ο0.5 (Uniform) | `vid_cond_pi05_uniform` | `ckpt/` | 6.20 GB | |
| | Vid. Cond. Ο0.5 (Keyframes) | `vid_cond_pi05_keyframes` | `ckpt/` | 6.20 GB | |
| | Vid2Robot | `vid2robot` | `ckpt.pt` | 2.37 GB | |
| | UniSkill | `uniskill_policy` | `ckpt.pth` | 0.74 GB | |
|
|
| UniSkill is a Robomimic diffusion policy over skill representations produced by |
| an **off-the-shelf, un-finetuned UniSkill inverse-skill-dynamics model**. That |
| ISD is deliberately *not* redistributed here β take it from the UniSkill |
| authors' own release and use it unmodified. |
|
|
| ## Training setup |
|
|
| Common to all runs: 10 tasks (`bowl_in_plate`, `close_drawer`, `empty_basket`, |
| `open_lid`, `pick_cube`, `press_toaster`, `push_cube`, `stack_cups`, |
| `toys_in_basket`, `toys_in_drawer`), 80 demonstrations per task |
| (`max_demo_index: 79`), 8-D state and action. |
|
|
| | Method | Setup | |
| |---|---| |
| | VLM + Ο0.5 | Ο0.5 full fine-tune from `lerobot/pi05_base`, expert-only; AdamW, peak LR 2.5e-5 cosineβ0.1Γ, batch 32, bf16, gradient checkpointing, 50k steps, action chunk 20, seed 42. Instructions generated by Qwen3-VL-4B from the human video. | |
| | Vid. Cond. Ο0.5 (Uniform) | openpi Ο0.5 from `pi05_base`, JAX FSDP; AdamW (Ξ² 0.9/0.95, Ξ΅ 1e-8, wd 1e-10), grad clip 1.0, peak LR 5e-5, 1k warmup, EMA 0.999, batch 64, 50k steps, seed 42. Demonstration tokens: 512 tokens / 32 frames / 16 tokens per frame, uniform frame sampling. Demo camera `front`, robot cameras `base_camera` + `hand_camera`. | |
| | Vid. Cond. Ο0.5 (Keyframes) | Identical to Uniform β **only frame selection differs**: GPT-5.4-mini best-point-prompt keyframes over the human video downsampled to 2 fps. | |
| | Vid2Robot | ViT-B/16 encoders, 2-layer Perceiver Resamplers (64 latents), 4-layer decoders d=768, 16 prompt / 8 state frames, action chunk 4 over 256 bins; AdamW peak 3e-4 β 1e-6, wd 0.05, 2k warmup, batch 32, 50k steps, seed 42. | |
| | UniSkill | Robomimic diffusion policy only; Adam 1e-4, batch 256, 1000 epochs Γ 100 iters = 100k steps, EMA power 0.75, 100 DDPM train / 10 DDIM inference steps, `skill_aug` aug_num 5, seed 0. | |
| |
| Every value above is also recorded verbatim in each folder's |
| `train_config.json`. |
|
|
| ## Paths |
|
|
| `train_config.json` and the UniSkill `config.json` are the configs the runs |
| were actually launched with, with one edit: **host-specific absolute paths were |
| rewritten to placeholders**, since they pointed at cluster mounts that nobody |
| else has. Everything else is untouched. |
|
|
| | Placeholder | Set it to | |
| |---|---| |
| | `${ROBOREEL_DATA}` | your RoboReel dataset root | |
| | `${ROBOREEL_CKPT}` | where you downloaded this repo | |
| | `${OPENPI_DATA_HOME}` | your openpi asset cache (holds `openpi-assets/checkpoints/pi05_base`) | |
|
|
| The evaluation harness reads `data_root` / `data_dir` out of |
| `train_config.json` to locate the human demonstration videos, so these must be |
| pointed at a real dataset root before evaluating. |
|
|
| ## Download |
|
|
| ```bash |
| pip install -U huggingface_hub |
| |
| # one method + condition |
| hf download LogosRoboticsLab/roboreel_models \ |
| --include "nd/vid_cond_pi05_keyframes/*" --local-dir ./roboreel_ckpt |
| |
| # one condition, all five methods |
| hf download LogosRoboticsLab/roboreel_models \ |
| --include "nd/*" --local-dir ./roboreel_ckpt |
| |
| # everything (β50 GB) |
| hf download LogosRoboticsLab/roboreel_models --local-dir ./roboreel_ckpt |
| ``` |
|
|
| ## Not in this repo |
|
|
| - **The RoboReel dataset** (human videos, robot trajectories, keyframe |
| annotations, skill representations) β released separately. |
| - **SeeDo** and **PointPolicy** checkpoints β two of the paper's seven methods, |
| to be added. |
| - **BC-Z** β discussed in related work only, never evaluated. |
| - Base/pretrained weights that are not ours to redistribute: `pi05_base`, |
| PaliGemma, CLIP, Depth-Anything-V2, and the UniSkill ISD. |
|
|
| ## Licensing |
|
|
| Mixed, because the checkpoints are derived from different base models: |
|
|
| - `vlm_pi05`, `vid_cond_pi05_uniform` and `vid_cond_pi05_keyframes` are |
| fine-tunes of **Ο0.5** (openpi, Apache-2.0), whose backbone is **PaliGemma / |
| Gemma** β these weights are therefore also subject to the |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms). |
| - `uniskill_policy` is a Robomimic diffusion policy trained on |
| representations from **UniSkill**; see the UniSkill release for its terms. |
| - `vid2robot` is trained from scratch on the RoboReel dataset apart from its |
| ImageNet-pretrained ViT-B/16 and SigLIP text encoder. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{roboreel2026, |
| title = {Monkey See, Can Monkey Do? A Benchmark for Evaluating Robot Skill Learning by Observation}, |
| journal = {arXiv preprint arXiv:2609.08209}, |
| year = {2026} |
| } |
| ``` |
|
|
