| # Environment Notes |
|
|
| This benchmark bundle was validated on a Linux GPU machine with: |
|
|
| - CoppeliaSim v4.1.0 available at `COPPELIASIM_ROOT` |
| - `xvfb` and `xauth` installed |
| - Qt xcb runtime libraries installed: |
| - `libxrender1` |
| - `libxkbcommon0` |
| - `libxkbcommon-x11-0` |
| - `libxcb-icccm4` |
| - `libxcb-image0` |
| - `libxcb-keysyms1` |
| - `libxcb-randr0` |
| - `libxcb-render-util0` |
| - `libxcb-shape0` |
| - `libxcb-shm0` |
| - `libxcb-sync1` |
| - `libxcb-xfixes0` |
| - `libxcb-xinerama0` |
| - `libxcb-xkb1` |
|
|
| The successful RLBench2 oven evaluation was run with: |
|
|
| ```bash |
| xvfb-run -a -s "-screen 0 1400x900x24" python \ |
| online_evaluation_rlbench/evaluate_policy.py \ |
| --checkpoint models/3dfa_peract2/3dfa_peract2.pth \ |
| --task bimanual_take_tray_out_of_oven \ |
| --data_dir data/3dfa/peract2_test \ |
| --dataset Peract2_3dfront_3dwrist \ |
| --image_size 256,256 \ |
| --model_type denoise3d \ |
| --bimanual true \ |
| --prediction_len 1 \ |
| --backbone clip \ |
| --fps_subsampling_factor 4 \ |
| --embedding_dim 120 \ |
| --num_attn_heads 8 \ |
| --num_vis_instr_attn_layers 3 \ |
| --num_history 3 \ |
| --num_shared_attn_layers 4 \ |
| --relative_action false \ |
| --rotation_format quat_xyzw \ |
| --denoise_timesteps 5 \ |
| --denoise_model rectified_flow |
| ``` |
|
|
| The `take shoes out of box` validation path was run from the bundled |
| PointFlowMatch source on a Blackwell GPU machine after upgrading the RLBench |
| environment to Torch `2.11.0+cu128` / torchvision `0.26.0+cu128` / |
| torchaudio `2.11.0+cu128`. The bundled PointFlowMatch tree also contains two |
| local compatibility fixes required for this workspace: |
|
|
| - `pfp/envs/rlbench_env.py` |
| - adapts PointFlowMatch to the local RLBench camera naming and observation API |
| - broadens motion-planning failure recovery to handle simulator-side runtime failures |
| - `pfp/policy/fm_policy.py` |
| - adds an inference-only fallback when legacy `composer` cannot import on modern Torch |
| - loads checkpoints with `weights_only=False` for PyTorch 2.6+ |
|
|
| The shoes evaluation command used here was: |
|
|
| ```bash |
| scripts/run_pointflowmatch_take_shoes_out_of_box.sh 10 50 |
| ``` |
|
|
| That wrapper expands to the equivalent raw command: |
|
|
| ```bash |
| export PYTHONPATH=third_party/diffusion_policy:third_party/PointFlowMatch:${PYTHONPATH:-} |
| export COPPELIASIM_ROOT=/path/to/CoppeliaSim |
| export LD_LIBRARY_PATH=${LD_LIBRARY_PATH:-}:$COPPELIASIM_ROOT |
| export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT |
| xvfb-run -a -s "-screen 0 1400x900x24" python \ |
| third_party/PointFlowMatch/scripts/evaluate.py \ |
| log_wandb=False \ |
| env_runner.env_config.vis=False \ |
| env_runner.num_episodes=10 \ |
| env_runner.max_episode_length=200 \ |
| policy.ckpt_name=1717447341-indigo-quokka/1717447341-indigo-quokka \ |
| policy.num_k_infer=50 |
| ``` |
|
|
| Result note for shoes: |
|
|
| - `reports/pointflowmatch_take_shoes_out_of_box_ep10_k50_gpu/summary.json` |
| records a verified non-zero result before a later RLBench/PyRep crash in the |
| same longer rollout. |
|
|
