README.md

metadata

license: other
license_name: see-model-card
license_link: https://huggingface.co/LogosRoboticsLab/roboreel_models#licensing
pipeline_tag: robotics
tags:
  - robotics
  - imitation-learning
  - learning-from-observation
  - manipulation
  - benchmark

RoboReel — evaluated policy checkpoints

Policy checkpoints for "Monkey See, Can Monkey Do? A Benchmark for Evaluating Robot Skill Learning by Observation" (arXiv:2609.08209), benchmark RoboReel · project page.

Each folder holds exactly the checkpoint the paper's numbers were computed from — not the last checkpoint a run happened to write. Training code, evaluation harness and configs live in the roboreel_models repository; the simulation environments live in the benchmark environment repository.

Layout

<condition>/<method>/
├── train_config.json      # the run's own config (see "Paths" below)
├── <sidecar>              # norm_stats.json | action_codec.json | demonstration_config.txt
└── ckpt/ | ckpt.pt | ckpt.pth    # the evaluated checkpoint

Every method's weights are called ckpt whichever trainer wrote them — a directory for the JAX/orbax and LeRobot policies, a single file for Vid2Robot (ckpt.pt) and UniSkill (ckpt.pth).

<condition> is the training suite, and it is the top level: nd/ = no distraction (clean scenes), wd/ = with distraction (distractors in both train and eval). So the keyframe policy trained with distractors is wd/vid_cond_pi05_keyframes/ckpt/.

The paper's third suite, ED (clean training, distracted evaluation), needs no extra weights — it evaluates everything under nd/ against the WD scene bank.

Checkpoints

Paper label	Method dir	Weights	Size each
VLM + π0.5	vlm_pi05	ckpt/	9.35 GB
Vid. Cond. π0.5 (Uniform)	vid_cond_pi05_uniform	ckpt/	6.20 GB
Vid. Cond. π0.5 (Keyframes)	vid_cond_pi05_keyframes	ckpt/	6.20 GB
Vid2Robot	vid2robot	ckpt.pt	2.37 GB
UniSkill	uniskill_policy	ckpt.pth	0.74 GB

UniSkill is a Robomimic diffusion policy over skill representations produced by an off-the-shelf, un-finetuned UniSkill inverse-skill-dynamics model. That ISD is deliberately not redistributed here — take it from the UniSkill authors' own release and use it unmodified.

Training setup

Common to all runs: 10 tasks (bowl_in_plate, close_drawer, empty_basket, open_lid, pick_cube, press_toaster, push_cube, stack_cups, toys_in_basket, toys_in_drawer), 80 demonstrations per task (max_demo_index: 79), 8-D state and action.

Method	Setup
VLM + π0.5	π0.5 full fine-tune from lerobot/pi05_base, expert-only; AdamW, peak LR 2.5e-5 cosine→0.1×, batch 32, bf16, gradient checkpointing, 50k steps, action chunk 20, seed 42. Instructions generated by Qwen3-VL-4B from the human video.
Vid. Cond. π0.5 (Uniform)	openpi π0.5 from pi05_base, JAX FSDP; AdamW (β 0.9/0.95, ε 1e-8, wd 1e-10), grad clip 1.0, peak LR 5e-5, 1k warmup, EMA 0.999, batch 64, 50k steps, seed 42. Demonstration tokens: 512 tokens / 32 frames / 16 tokens per frame, uniform frame sampling. Demo camera front, robot cameras base_camera + hand_camera.
Vid. Cond. π0.5 (Keyframes)	Identical to Uniform — only frame selection differs: GPT-5.4-mini best-point-prompt keyframes over the human video downsampled to 2 fps.
Vid2Robot	ViT-B/16 encoders, 2-layer Perceiver Resamplers (64 latents), 4-layer decoders d=768, 16 prompt / 8 state frames, action chunk 4 over 256 bins; AdamW peak 3e-4 → 1e-6, wd 0.05, 2k warmup, batch 32, 50k steps, seed 42.
UniSkill	Robomimic diffusion policy only; Adam 1e-4, batch 256, 1000 epochs × 100 iters = 100k steps, EMA power 0.75, 100 DDPM train / 10 DDIM inference steps, skill_aug aug_num 5, seed 0.

Every value above is also recorded verbatim in each folder's train_config.json.

Paths

train_config.json and the UniSkill config.json are the configs the runs were actually launched with, with one edit: host-specific absolute paths were rewritten to placeholders, since they pointed at cluster mounts that nobody else has. Everything else is untouched.

Placeholder	Set it to
${ROBOREEL_DATA}	your RoboReel dataset root
${ROBOREEL_CKPT}	where you downloaded this repo
${OPENPI_DATA_HOME}	your openpi asset cache (holds openpi-assets/checkpoints/pi05_base)

The evaluation harness reads data_root / data_dir out of train_config.json to locate the human demonstration videos, so these must be pointed at a real dataset root before evaluating.

Download

pip install -U huggingface_hub

# one method + condition
hf download LogosRoboticsLab/roboreel_models \
  --include "nd/vid_cond_pi05_keyframes/*" --local-dir ./roboreel_ckpt

# one condition, all five methods
hf download LogosRoboticsLab/roboreel_models \
  --include "nd/*" --local-dir ./roboreel_ckpt

# everything (≈50 GB)
hf download LogosRoboticsLab/roboreel_models --local-dir ./roboreel_ckpt

Not in this repo

• The RoboReel dataset (human videos, robot trajectories, keyframe annotations, skill representations) — released separately.
• SeeDo and PointPolicy checkpoints — two of the paper's seven methods, to be added.
• BC-Z — discussed in related work only, never evaluated.
• Base/pretrained weights that are not ours to redistribute: pi05_base, PaliGemma, CLIP, Depth-Anything-V2, and the UniSkill ISD.

Licensing

Mixed, because the checkpoints are derived from different base models:

• vlm_pi05, vid_cond_pi05_uniform and vid_cond_pi05_keyframes are fine-tunes of π0.5 (openpi, Apache-2.0), whose backbone is PaliGemma / Gemma — these weights are therefore also subject to the Gemma Terms of Use.
• uniskill_policy is a Robomimic diffusion policy trained on representations from UniSkill; see the UniSkill release for its terms.
• vid2robot is trained from scratch on the RoboReel dataset apart from its ImageNet-pretrained ViT-B/16 and SigLIP text encoder.

Citation

@article{roboreel2026,
  title   = {Monkey See, Can Monkey Do? A Benchmark for Evaluating Robot Skill Learning by Observation},
  journal = {arXiv preprint arXiv:2609.08209},
  year    = {2026}
}