| --- |
| tags: |
| - robotics |
| - rlbench |
| - benchmarking |
| - label-validation |
| --- |
| |
| # VLAdaptorBench |
|
|
| This repository contains the benchmark setup, metric code, debug history, and validation artifacts for the proposed VLA + adaptor label study on `bimanual_take_tray_out_of_oven`. |
|
|
| This is still a label-validation repository, not a policy repository. No `pi0.5` integration is included here. |
|
|
| ## Current Status |
|
|
| The latest work behind this upload produced: |
|
|
| - `metric_iter30_full100_single_pass_full_logging_fixed_templates_merged` |
| - merged 100-episode dense/fuller-logging result tree from the single-pass fixed-template run |
|
|
| The current Hub upload includes: |
|
|
| - `artifacts/results/metric_iter31_sample10_all_metrics_verify/` |
| - compact 10-episode verification subset with `all_metrics` GIFs only |
| - the fast `all_metrics`-only render path in: |
| - `code/scripts/render_oven_metric_frame.py` |
| - `code/scripts/render_oven_metric_gifs.py` |
|
|
| The new sample verification bundle is meant to be the quickest remote sanity-check entry point. It includes the sampled dense/keyframe tables, per-episode metrics, fuller debug sidecars, fixed templates, selection metadata, and one compact full-metrics GIF per sampled episode. |
|
|
| The earlier `metric_iter29_ep0_single_pass_full_logging_fixed_templates` validation pass for episode 0 remains the detailed single-episode reference for the fuller debug logging and the debug-aware GIF renderer. |
|
|
| That run keeps the trusted `iter24` template bundle fixed, adds the fuller dense/debug logging in a single pass, and regenerates the episode-0 visualization suite from the richer artifact. It is the current reference for: |
|
|
| - the `episode0.debug.jsonl` sidecar with per-frame `p_pre` and `p_ext` internals |
| - the single-pass dense CSV with fuller logged sub-metrics |
| - the updated `path_quality_focus` GIF that now exposes the `p_ext` milestone search, milestone scores, and planner outcomes directly in the visualization |
|
|
| The earlier `metric_iter24_*_door_contact_geom` reruns for episodes 0 and 1 remain the trusted baseline for the repaired oven metrics. |
|
|
| That rerun fixes the main simulator-state bugs that were still contaminating the oven metrics: |
|
|
| 1. The reveal-to-retrieve transition used to occur too late, effectively at grasp time. |
| 2. The visibility metric used to drop to zero around frame 232 even when the tray grasp region was clearly visible in `wrist_left`. |
| 3. `p_pre` stayed near zero until grasp. |
| 4. Extraction labels could flicker or drift because oracle rollouts were not restoring the simulator state exactly. |
| 5. The old dense runner's restore-heavy path could still bias later frames after an oracle call. |
|
|
| The current code addresses those issues by: |
|
|
| - decoding RLBench mask PNGs correctly before converting them back to simulator handles |
| - scoring visibility directly from mask-handle agreement instead of the old depth/z heuristic |
| - inferring tray mask handles from grasp-region projections |
| - deriving a late-window pregrasp approach template instead of accidentally including frame-8 arm poses |
| - adding explicit `pregrasp_progress`, `pregrasp_distance`, `pregrasp_speed`, and `phase_score` |
| - making the repair path batch frames sequentially per worker so late-frame rows do not drift |
| - snapshotting and restoring exact arm joints, gripper joints, and the full grasped-object subtree |
| - supporting and now preferring `--independent-replay` for the authoritative dense study |
| - tightening `y_pre` so it stays on once the retriever is clearly inside the pregrasp corridor |
| - retuning `phase_score` so it tracks the reveal-to-retrieve handoff instead of generic early motion |
| - recomputing intervention validity from isolated per-frame env replays instead of the old live-cache path |
| - sampling intervention states earlier in the reveal phase so pre-ready extract checks are not contaminated by borderline near-ready states |
| - confirming extraction feasibility with repeated planner checks inside the extract oracle so one lucky planner sample is less likely to flip a label |
|
|
| The old `iter4_*`, `iter6_*`, `iter19_*`, and `iter22_*` outputs are still useful historical checkpoints, but the current authoritative outputs are: |
|
|
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/` |
| - `artifacts/results/metric_iter29_ep0_single_pass_full_logging_fixed_templates/` |
|
|
| The main new fix in `iter24` is the assisted-door contact scoring inside `p_pre`: |
|
|
| - the old `ignore_collisions=True` branch treated oven-door contact as name-whitelisted and only checked the final door angle change |
| - the new scorer traces door contacts step-by-step, estimates the local door-surface normal from simulator geometry, scores whether the retriever is sliding along the door or pushing it open, and penalizes direct head-on contact or door-closing motion |
| - this specifically removes the false closed-door `p_pre` spike in episode 0 around frames `43-56` without collapsing the later pregrasp rise once the door is actually opening |
|
|
| The current repo state should therefore be treated as the repaired benchmark snapshot with geometry-aware door assistance, not the final metric design. |
|
|
| Brief caveat: the current `y_ready` label still gates on low oven-door angular speed after extraction feasibility persists. In this task, the retriever arm can legitimately nudge the door while already committing to retrieval, so `y_ready` can still switch later than the true reveal-to-retrieve boundary. For the current oven benchmark, `y_ready` should therefore not be treated as a decisive validation metric or a trusted phase-switch target. |
|
|
| The oven task also has a highly structured reveal-to-retrieve handoff in the expert demos: both arms reposition, the revealer opens and clears the door, then the retriever commits. Because that phase pattern is so standardized, good results on this task are most useful as a task-specific smoke test or a "does the adaptor beat a base finetune here?" check, not as strong evidence of general reveal-and-retrieve reasoning. |
|
|
| ## What Is In This Upload |
|
|
| - `code/rr_label_study/` |
| - Core metric code. |
| - Dense replay, visibility scoring, pregrasp/extraction oracles, keyframe extraction, intervention checks, and summary metric computation. |
| - `code/scripts/` |
| - Study runners and helpers. |
| - `run_oven_label_study.py`: dense/keyframe study runner. |
| - `launch_parallel_oven_label_study.py`: multi-display worker launcher. |
| - `recompute_oven_pregrasp_parallel.py`: targeted dense rerun for repaired `p_pre` labels. |
| - `run_oven_pregrasp_batch.py`: sequential per-worker pregrasp recomputation helper. |
| - `refresh_saved_oven_study.py`: recompute keyframes, per-episode metrics, intervention stats, and summary JSONs from saved dense CSVs after metric-code changes. |
| - `run_oven_single_frame.py`: single-frame recomputation helper. |
| - `run_oven_frame_batch.py`: new sequential batch recomputation helper used to avoid late-frame drift. |
| - `repair_oven_episode_dense.py`: batched repair pass for suspicious dense rows. |
| - `render_oven_metric_frame.py`: per-frame visualization renderer. |
| - `render_oven_metric_gifs.py`: GIF renderer. |
| - The visualization renderer now accepts either legacy `templates.pkl` files or the newer authoritative `templates.json` bundles. |
| - `artifacts/results/` |
| - Full debug history, including stale runs and current validation outputs. |
| - `runtime_assets/` |
| - Archived runtime assets needed to recreate this setup on another machine. |
| - Includes the local oven-task dataset snapshot and the local `coppelia_sim` extraction used on this machine. |
| - `environment/` |
| - Machine snapshot, env export, pip freeze, setup helpers, and dataset notes. |
| - `external/` |
| - Local source snapshots of RLBench, PyRep, PerAct bimanual, and YARR used for this work. |
| - `MANIFEST.txt` |
| - Flat file listing of the upload contents. |
|
|
| ## Latest Metric Fixes |
|
|
| The latest code changes are in: |
|
|
| - `code/rr_label_study/oven_study.py` |
| - `code/scripts/recompute_oven_pregrasp_parallel.py` |
| - `code/scripts/run_oven_pregrasp_batch.py` |
| - `code/scripts/repair_oven_episode_dense.py` |
| - `code/scripts/run_oven_frame_batch.py` |
| - `code/scripts/render_oven_metric_frame.py` |
|
|
| The important changes are: |
|
|
| ### 1. Visibility metric repair |
|
|
| - `_load_mask()` now rescales stored mask PNGs back to `[0, 1]` before calling `rgb_handles_to_mask`. |
| - Visibility is now computed by projecting grasp-region or whole-tray points into each camera and checking whether the decoded mask handle at the projected pixel matches the inferred tray handles. |
| - Template derivation now infers `mask_handle_ids` from reference frames near the actual pregrasp/grasp window. |
|
|
| This fixes the old failure where visibility dropped to zero even when the tray lip was visibly present in the wrist camera. |
|
|
| ### 2. Pregrasp/path metric repair |
|
|
| - Template extraction now detects the pregrasp approach onset in a bounded late window before grasp instead of taking the first small negative slope in the entire episode. |
| - The current template approach frames for episode 0 are now: |
| - `177, 187, 197, 208, 218, 229, 232` |
| - `p_pre` now uses the last few approach templates plus explicit geometric progress toward the pregrasp pose instead of only brittle planner success. |
| - `y_pre` now treats "already inside the pregrasp corridor" as success, which is appropriate for this oracle study. |
| - The assisted pregrasp branch no longer treats oven-door collisions as a binary whitelist: |
| - it traces per-step door contacts under `ignore_collisions=True` |
| - estimates a local door-surface normal from the contacted simulator shape |
| - rewards tangential or door-opening contact |
| - penalizes head-on or door-closing contact |
| - requires a minimum geometry-aware door-contact quality before assisted `p_pre` credit is given |
|
|
| ### 3. Replay/repair correctness |
|
|
| - The old isolated repair path replayed every suspicious frame from a fresh reset, which could corrupt late rows. |
| - The new helper `run_oven_frame_batch.py` computes frame rows sequentially inside a single env per worker. |
| - `repair_oven_episode_dense.py` now distributes frame batches, not individual frames, across displays. |
| - `SimulatorSnapshot` now restores: |
| - arm joint trees and explicit joint positions |
| - gripper joint trees and explicit joint positions |
| - the full subtree under any grasped object |
| - grasp attachments with the original release parent |
| - `ReplayCache` now keeps retrying stable grasp attachment while the demo gripper remains closed. |
|
|
| This fixed the major replay bug where post-oracle restores could leave the arm, gripper, or grasped tray in a subtly different state than the true demo frame. |
|
|
| ### 4. Earlier phase signal |
|
|
| - The code now records: |
| - `pregrasp_progress` |
| - `pregrasp_distance` |
| - `pregrasp_speed` |
| - `phase_score` |
| - `phase_score` is now dominated by actual approach progress and `p_pre`, with a stricter threshold (`0.5`) so it no longer flips during the early reveal phase. |
| - `y_retrieve` is still oracle-like and monotone, but the metric side now has a cleaner approach-sensitive signal for early switching. |
|
|
| ### 5. Independent replay |
|
|
| - `run_oven_label_study.py` already exposed `--independent-replay`. |
| - `launch_parallel_oven_label_study.py` now passes that flag through to worker runs. |
| - For the current oven study, independent replay is the trustworthy dense mode because it avoids cross-frame contamination from oracle rollouts. |
|
|
| ### 6. Intervention validity repair |
|
|
| - The old intervention summary reused the dense-study replay cache, which could still corrupt post-ready extract checks. |
| - `_interventional_validity()` now evaluates each sampled intervention state from a fresh env/replay instance. |
| - `refresh_saved_oven_study.py` now supports `--dataset-root` so intervention metrics can be recomputed instead of copied forward from stale JSON. |
| - The refined intervention protocol now samples pre-ready states at `ready_onset-20` and `ready_onset-10` instead of `ready_onset-10` and `ready_onset-5`, which avoids counting borderline almost-ready states as generic reveal-phase interventions. |
|
|
| ### 7. Extraction-oracle hardening |
|
|
| - `_extract_score_and_success()` now uses repeated planner checks before marking a milestone as feasible. |
| - The current configuration is intentionally modest: |
| - `DEFAULT_PLAN_ATTEMPTS = 2` |
| - `DEFAULT_PLAN_MIN_SUCCESSES = 2` |
| - This only hardens the extraction oracle, not the pregrasp score, so the dense study remains tractable while the noisy pre-ready extract successes are suppressed. |
|
|
| ## Latest Validated Artifacts |
|
|
| The current trustworthy artifacts are: |
|
|
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/episode0.dense.csv` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/episode0.keyframes.csv` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/episode0.metrics.json` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/summary.json` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/visualizations/episode0_all_metrics.gif` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/visualizations/episode0_visibility_focus.gif` |
| - `artifacts/results/metric_iter24_ep0_door_contact_geom/visualizations/episode0_path_quality_focus.gif` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/episode1.dense.csv` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/episode1.keyframes.csv` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/episode1.metrics.json` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/summary.json` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/visualizations/episode1_all_metrics.gif` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/visualizations/episode1_visibility_focus.gif` |
| - `artifacts/results/metric_iter24_ep1_door_contact_geom/visualizations/episode1_path_quality_focus.gif` |
|
|
| - `artifacts/results/oven_episode0_iter4_templates/templates.json` |
| - `artifacts/results/oven_episode0_iter4_templates/templates.pkl` |
| - `artifacts/results/oven_episode0_iter4_batch/iter4_batch_comparison.csv` |
| - `artifacts/results/oven_episode0_iter4_batch/frames/` |
| - `artifacts/results/oven_episode0_iter4_clean/iter4_targeted_comparison.csv` |
| - `artifacts/results/oven_episode0_iter4_dense_geom_170_234.csv` |
| - `artifacts/results/oven_episode0_iter6_visual_checks/boundary_rgb_contact_sheet.png` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.dense.csv` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.keyframes.csv` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.metrics.json` |
| - `artifacts/results/oven_episode0_iter6_independent_full/summary.json` |
| - `artifacts/results/oven_episode0_iter6_visual_checks/early_visibility_contact_sheet.png` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/episode0.dense.csv` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/episode0.metrics.json` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_all_metrics.gif` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_visibility_focus.gif` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_path_quality_focus.gif` |
| - `artifacts/results/manual_metric_checks/episode0_frame210_visibility.png` |
| - `artifacts/results/manual_metric_checks/episode0_frame232_visibility.png` |
| - `artifacts/results/manual_metric_checks/episode0_frame210_path.png` |
| - `artifacts/results/manual_metric_checks/episode6_frame230_path.png` |
| - `artifacts/results/iter12_parallel_smoke_8ep_refined/parallel_summary.json` |
| - `artifacts/results/iter12_parallel_smoke_8ep_refined/parallel_workers.json` |
|
|
| The `iter6_independent_full` CSVs and JSON summaries have been refreshed with the latest `phase_score` logic via `code/scripts/refresh_saved_oven_study.py`. |
|
|
| ## Key Verified Findings |
|
|
| From the current independent-replay validation on episode 0: |
|
|
| - Visibility over the dense 170-234 window is clean: |
| - min `three_view_visibility = 1.0` |
| - min `full_view_visibility = 1.0` |
| - Pregrasp progress now rises well before grasp and stays predictive: |
| - frame `210`: `pregrasp_progress ≈ 0.451`, `p_pre ≈ 0.185`, `y_pre = 0` |
| - frame `215`: `pregrasp_progress ≈ 0.568`, `p_pre ≈ 0.375`, `y_pre = 1` |
| - frame `220`: `pregrasp_progress ≈ 0.702`, `p_pre ≈ 0.496`, `y_pre = 1` |
| - frame `225`: `pregrasp_progress ≈ 0.847`, `p_pre ≈ 0.559`, `y_pre = 1` |
| - frame `230`: `pregrasp_progress ≈ 0.950`, `p_pre ≈ 0.654`, `y_pre = 1` |
| - Extraction feasibility is now separated from pregrasp: |
| - frame `230`: `p_ext ≈ 0.0007`, `y_ext = 0` |
| - frame `232`: `p_ext = 1.0`, `y_ext = 1` |
| - frame `234`: `p_ext = 1.0`, `y_ext = 1` |
| - In the refreshed full independent episode-0 run: |
| - `ppre_cross_frame = 216` |
| - `pext_cross_frame = 232` |
| - `phase_cross_frame = 214` |
| - `retrieve_cross_frame = 215` |
| - `ready_cross_frame = 234` |
| - `single_switch_rate = 1.0` |
| - `reversion_rate = 0.0` |
| - `auroc_ppre_ypre ≈ 0.761` |
| - `auprc_ppre_ypre ≈ 0.903` |
| - `auroc_pext_yext = 1.0` |
| - `auprc_pext_yext = 1.0` |
| - `auroc_phase_yretrieve = 1.0` |
| - `auprc_phase_yretrieve = 1.0` |
| - `f1_phase_yretrieve ≈ 0.996` |
| - `auroc_phase_yready ≈ 0.998` |
| - `f1_phase_yready ≈ 0.905` |
| - In the refreshed isolated intervention check on episode 0: |
| - pre-ready `open_more` increases `p_ext` on `2/2` sampled states |
| - pre-ready `extract` succeeds on `0/2` |
| - post-ready `extract` succeeds on `2/2` |
| - post-ready `open_more` and `hold_open` both have low marginal gain on `2/2` |
| - The refreshed phase columns now place: |
| - `first phase_switch` at frame `214` |
| - `first y_retrieve` at frame `215` |
| - `first y_ready` at frame `234` |
| - The refined 8-episode independent-replay smoke in `artifacts/results/iter12_parallel_smoke_8ep_refined/` shows: |
| - `single_switch_rate = 1.0` |
| - `reversion_rate = 0.0` |
| - mean `auroc_ppre_ypre ≈ 0.809` |
| - mean `auprc_ppre_ypre ≈ 0.924` |
| - mean `auroc_pext_yext = 1.0` |
| - mean `auprc_pext_yext = 1.0` |
| - mean `f1_phase_yretrieve ≈ 0.996` |
| - mean `f1_phase_yready ≈ 0.906` |
| - mean dense boundary error to `y_retrieve ≈ 0.88` frames |
| - mean pre-ready extract success `= 0.0/2.0` |
| - mean pre-ready wait extract success `= 0.0/2.0` |
| - mean post-ready extract success `≈ 1.625/2.0` |
| - The main remaining limitation on this oven task is not a broken metric but task structure: |
| - the grasp-region visibility metric is visually faithful but only weakly predictive because the tray lip is already visible early in many demos |
| - time remains a very strong trivial baseline for `y_ext` on expert demos |
| - `open_more` improves `p_ext` mainly near the reveal/retrieve boundary, not uniformly throughout the whole pre-ready window |
|
|
| See: |
|
|
| - `artifacts/results/oven_episode0_iter4_batch/iter4_batch_comparison.csv` |
| - `artifacts/results/oven_episode0_iter4_dense_geom_170_234.csv` |
| - `artifacts/results/oven_episode0_iter6_visual_checks/boundary_rgb_contact_sheet.png` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.dense.csv` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.metrics.json` |
| - `artifacts/results/manual_metric_checks/episode0_frame210_visibility.png` |
| - `artifacts/results/manual_metric_checks/episode0_frame232_visibility.png` |
| - `artifacts/results/manual_metric_checks/episode0_frame210_path.png` |
| - `artifacts/results/manual_metric_checks/episode6_frame230_path.png` |
| - `artifacts/results/iter12_parallel_smoke_8ep_refined/parallel_summary.json` |
|
|
| ## Artifact Guide |
|
|
| ### Current artifacts |
|
|
| - `oven_episode0_iter3_templates/` |
| - First regenerated template bundle after the mask/approach fixes. |
| - `oven_episode0_iter4_templates/` |
| - Current template bundle with the corrected late-window approach onset. |
| - `oven_episode0_iter4_clean/` |
| - Isolated targeted frame checks used while diagnosing the old per-frame repair drift. |
| - `oven_episode0_iter4_batch/` |
| - Current batched sequential repair validation. |
| - `oven_episode0_iter4_dense_geom_170_234.csv` |
| - Dense sequential geometry and visibility sweep across the reveal-to-retrieve boundary. |
|
|
| ### Historical artifacts |
|
|
| - `oven_episode0_repaired_v1/` |
| - Useful historical reference, but not the current authoritative artifact. |
| - It still contains the old late transition and old visibility/path issues. |
| - `oven_episode0_full*/`, `oven_to240_*/`, `oven_episode0_independent_v*/` |
| - Debugging history from earlier iterations. |
| - `parallel_smoke_2x10/` |
| - Xvfb/worker parallelization smoke test. |
| - `oven_smoke_*` |
| - Early smoke runs. |
|
|
| ## Repository Map |
|
|
| Relevant entry points: |
|
|
| - `code/rr_label_study/oven_study.py` |
| - `code/scripts/run_oven_label_study.py` |
| - `code/scripts/launch_parallel_oven_label_study.py` |
| - `code/scripts/run_oven_single_frame.py` |
| - `code/scripts/run_oven_frame_batch.py` |
| - `code/scripts/repair_oven_episode_dense.py` |
| - `code/scripts/render_oven_metric_frame.py` |
| - `code/scripts/render_oven_metric_gifs.py` |
|
|
| Relevant current artifacts: |
|
|
| - `artifacts/results/oven_episode0_iter4_templates/templates.json` |
| - `artifacts/results/oven_episode0_iter4_batch/iter4_batch_comparison.csv` |
| - `artifacts/results/oven_episode0_iter4_dense_geom_170_234.csv` |
| - `artifacts/results/oven_episode0_iter6_independent_full/episode0.dense.csv` |
| - `artifacts/results/oven_episode0_iter6_independent_full/summary.json` |
| - `artifacts/results/oven_episode0_iter6_visual_checks/boundary_rgb_contact_sheet.png` |
| - `artifacts/results/oven_episode0_iter6_visual_checks/early_visibility_contact_sheet.png` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/episode0.dense.csv` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/episode0.metrics.json` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_all_metrics.gif` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_visibility_focus.gif` |
| - `artifacts/results/oven_episode0_iter16_gif_suite/visualizations/episode0_path_quality_focus.gif` |
|
|
| ## Environment |
|
|
| This was run on: |
|
|
| - Ubuntu `22.04.5` |
| - Kernel `6.8.0-65-generic` |
| - `96` CPU cores visible |
| - `503 GiB` RAM visible |
| - `NVIDIA A40` |
|
|
| See: |
|
|
| - `environment/system_info.txt` |
| - `environment/repo_revisions.txt` |
| - `environment/conda_env_rlbench.yml` |
| - `environment/pip_freeze_rlbench.txt` |
|
|
| ## Upstream Repos Used |
|
|
| Exact revisions are recorded in `environment/repo_revisions.txt`. |
|
|
| The local run used: |
|
|
| - `markusgrotz/RLBench` |
| - `markusgrotz/PyRep` |
| - `markusgrotz/peract_bimanual` |
| - `markusgrotz/YARR` |
|
|
| Those source snapshots are included under `external/`. |
|
|
| ## Reproducing On The Same Hardware Class |
|
|
| 1. Read `environment/dataset_notes.txt`. |
| 2. Run `environment/setup_same_hardware.sh /workspace`. |
| 3. Source `environment/activate_rlbench_runtime.sh /workspace`. |
| 4. Run the dense study: |
|
|
| ```bash |
| python /workspace/VLAdaptorBench_upload/code/scripts/run_oven_label_study.py \ |
| --dataset-root /workspace/data/bimanual_take_tray_out_of_oven_train_128 \ |
| --result-dir /workspace/tmp_run \ |
| --max-episodes 1 \ |
| --checkpoint-stride 16 \ |
| --template-episode-index 0 \ |
| --independent-replay |
| ``` |
|
|
| 5. If you want to repair suspicious frames in parallel with the new batched path: |
|
|
| ```bash |
| python /workspace/VLAdaptorBench_upload/code/scripts/repair_oven_episode_dense.py \ |
| --dataset-root /workspace/data/bimanual_take_tray_out_of_oven_train_128 \ |
| --episode-dir /workspace/data/bimanual_take_tray_out_of_oven_train_128/all_variations/episodes/episode0 \ |
| --input-dense-csv /workspace/tmp_run/episode0.dense.csv \ |
| --output-dir /workspace/tmp_run_repaired \ |
| --checkpoint-stride 16 \ |
| --num-workers 4 \ |
| --base-display 170 |
| ``` |
|
|
| ## Important Note |
|
|
| The full 100-episode independent-replay run is not yet the authoritative artifact in this upload. The current repository state documents the repaired metric code, the exact snapshot/restore fixes, and the episode-0 independent validation that is required before scaling to the full study. |
|
|
| ## Dataset Note |
|
|
| The RLBench demonstration dataset itself is not re-uploaded here. This repository contains the study code and generated artifacts only. The expected dataset path is documented in `environment/dataset_notes.txt`. |
|
|
| CoppeliaSim binaries are not included. The setup helpers expect a local extraction at `/workspace/coppelia_sim`. |
|
|
