Update reference outputs to prediction-only preview videos
Browse files- examples/README.md +0 -26
- examples/reference_outputs/README.md +0 -13
- examples/reference_outputs/example_01.jsonl +1 -1
- examples/reference_outputs/{summary_gt_vs_pred.png → example_01_pred_progress.mp4} +2 -2
- examples/reference_outputs/example_02.jsonl +1 -1
- examples/reference_outputs/{example_01_gt_vs_pred.png → example_02_pred_progress.mp4} +2 -2
- examples/reference_outputs/example_03.jsonl +1 -0
- examples/reference_outputs/{example_02_gt_vs_pred.png → example_03_pred_progress.mp4} +2 -2
examples/README.md
DELETED
|
@@ -1,26 +0,0 @@
|
|
| 1 |
-
# Examples
|
| 2 |
-
|
| 3 |
-
This directory contains `3` bundled demo episodes for quick smoke testing and simple progress inference demos.
|
| 4 |
-
|
| 5 |
-
Included examples:
|
| 6 |
-
|
| 7 |
-
- `example_01`
|
| 8 |
-
- bucket: `test_nonexpert_seen`
|
| 9 |
-
- task: `抓取紫色方块使其从方形洞口落入积木桶中`
|
| 10 |
-
- note: `contains rollback / grasp-failure regression`
|
| 11 |
-
- `example_02`
|
| 12 |
-
- bucket: `test_nonexpert_unseen`
|
| 13 |
-
- task: `把洋葱放进快递箱里。`
|
| 14 |
-
- note: `contains rollback / object-drop regression`
|
| 15 |
-
- `example_03`
|
| 16 |
-
- bucket: `test_expert_seen`
|
| 17 |
-
- task: `将三角烧杯放在三脚架上。`
|
| 18 |
-
|
| 19 |
-
Each example directory contains:
|
| 20 |
-
|
| 21 |
-
- `episode.mp4`: demo video
|
| 22 |
-
- `metadata.json`: source bucket and task metadata, plus bundled benchmark GT progress
|
| 23 |
-
|
| 24 |
-
Reference outputs and GT-vs-prediction comparison plots for the rollback demos are available under `reference_outputs/`.
|
| 25 |
-
|
| 26 |
-
Use `../quick_start/run_example.py` to run the released VLAC model on any of these examples.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
examples/reference_outputs/README.md
DELETED
|
@@ -1,13 +0,0 @@
|
|
| 1 |
-
# Reference Outputs
|
| 2 |
-
|
| 3 |
-
This directory contains reference inference outputs for the bundled rollback demos.
|
| 4 |
-
|
| 5 |
-
Included files:
|
| 6 |
-
|
| 7 |
-
- `example_01.jsonl`: reference output for `example_01`
|
| 8 |
-
- `example_02.jsonl`: reference output for `example_02`
|
| 9 |
-
- `example_01_gt_vs_pred.png`: GT-vs-pred comparison for `example_01`
|
| 10 |
-
- `example_02_gt_vs_pred.png`: GT-vs-pred comparison for `example_02`
|
| 11 |
-
- `summary_gt_vs_pred.png`: combined overview for the included reference examples
|
| 12 |
-
|
| 13 |
-
These outputs were generated with the released VLAC model using the bundled `quick_start/run_example.py` behavior: `chunk_all` prompting and `2 Hz` video sampling.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
examples/reference_outputs/example_01.jsonl
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"example_id": "example_01", "bucket": "test_nonexpert_seen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251130-162433-bad-purple cube-shaped building block/videos/chunk-000/observation.images.front/episode_000007", "task_instruction": "抓取紫色方块使其从方形洞口落入积木桶中", "task_description": "抓取紫色方块使其从方形洞口落入积木桶中：\n爪夹开始移动：0%\n爪夹接近紫色方块：20%\n爪夹抓紧紫色方块：40%\n爪夹抓紧紫色方块接近方形洞口：60%\n爪夹将紫色方块对准方形洞口：80%\n紫色方块从方形洞口落入积木桶中，爪夹移开：100%", "video_path": "examples/example_01/episode.mp4", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 255], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5], "input_frame_count_2hz": 18, "decoded_video_stats": {"decoded_frame_count": 258.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 抓取紫色方块使其从方形洞口落入积木桶中\n爪夹开始移动：0%\n爪夹接近紫色方块：20%\n爪夹抓紧紫色方块：40%\n爪夹抓紧紫色方块接近方形洞口：60%\n爪夹将紫色方块对准方形洞口：80%\n紫色方块从方形洞口落入积木桶中，爪夹移开：100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 1.3s, 进度: 0%\n时间: 2.7s, 进度: 10%\n时间: 3.7s, 进度: 15%\n时间: 5.5s, 进度: 10%\n时间: 6.3s, 进度: -15%\n时间: 7.1s, 进度: -25%\n时间: 8.0s, 进度: -15%", "benchmark_progress_type": "dense_kinematic_progress", "benchmark_progress_source": {"bucket": "test_nonexpert_seen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251130-162433-bad-purple cube-shaped building block/videos/chunk-000/observation.images.front/episode_000007"}, "benchmark_semantic_anchors": [{"frame": 49, "progress": 0.0}, {"frame": 74, "progress": 5.0}, {"frame": 87, "progress": 10.0}, {"frame": 134, "progress": 15.0}, {"frame": 173, "progress": 13.0}, {"frame": 186, "progress": -15.0}, {"frame": 220, "progress": -30.0}, {"frame": 239, "progress": -20.0}, {"frame": 257, "progress": -10.0}], "gt_dense_progress": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0, 3.2, 3.4, 3.6, 3.8, 4.0, 4.2, 4.4, 4.6, 4.8, 5.0, 5.4, 5.8, 6.2, 6.5, 6.9, 7.3, 7.7, 8.1, 8.5, 8.8, 9.2, 9.6, 10.0, 10.1, 10.2, 10.3, 10.4, 10.5, 10.6, 10.7, 10.9, 11.0, 11.1, 11.2, 11.3, 11.4, 11.5, 11.6, 11.7, 11.8, 11.9, 12.0, 12.1, 12.2, 12.3, 12.4, 12.6, 12.7, 12.8, 12.9, 13.0, 13.1, 13.2, 13.3, 13.4, 13.5, 13.6, 13.7, 13.8, 13.9, 14.0, 14.1, 14.3, 14.4, 14.5, 14.6, 14.7, 14.8, 14.9, 15.0, 14.9, 14.9, 14.8, 14.8, 14.7, 14.7, 14.6, 14.6, 14.5, 14.5, 14.4, 14.4, 14.3, 14.3, 14.2, 14.2, 14.1, 14.1, 14.0, 14.0, 13.9, 13.9, 13.8, 13.8, 13.7, 13.7, 13.6, 13.6, 13.5, 13.5, 13.4, 13.4, 13.3, 13.3, 13.2, 13.2, 13.1, 13.1, 13.0, 10.8, 8.7, 6.5, 4.4, 2.2, 0.1, -2.1, -4.2, -6.4, -8.5, -10.7, -12.8, -15.0, -15.4, -15.9, -16.3, -16.8, -17.2, -17.6, -18.1, -18.5, -19.0, -19.4, -19.9, -20.3, -20.7, -21.2, -21.6, -22.1, -22.5, -22.9, -23.4, -23.8, -24.3, -24.7, -25.1, -25.6, -26.0, -26.5, -26.9, -27.4, -27.8, -28.2, -28.7, -29.1, -29.6, -30.0, -29.5, -28.9, -28.4, -27.9, -27.4, -26.8, -26.3, -25.8, -25.3, -24.7, -24.2, -23.7, -23.2, -22.6, -22.1, -21.6, -21.1, -20.5, -20.0, -19.4, -18.9, -18.3, -17.8, -17.2, -16.7, -16.1, -15.6, -15.0, -14.4, -13.9, -13.3, -12.8, -12.2, -11.7, -11.1, -10.6, -10.0], "gt_dense_timestamps_sec": [0.0, 0.033333, 0.066667, 0.1, 0.133333, 0.166667, 0.2, 0.233333, 0.266667, 0.3, 0.333333, 0.366667, 0.4, 0.433333, 0.466667, 0.5, 0.533333, 0.566667, 0.6, 0.633333, 0.666667, 0.7, 0.733333, 0.766667, 0.8, 0.833333, 0.866667, 0.9, 0.933333, 0.966667, 1.0, 1.033333, 1.066667, 1.1, 1.133333, 1.166667, 1.2, 1.233333, 1.266667, 1.3, 1.333333, 1.366667, 1.4, 1.433333, 1.466667, 1.5, 1.533333, 1.566667, 1.6, 1.633333, 1.666667, 1.7, 1.733333, 1.766667, 1.8, 1.833333, 1.866667, 1.9, 1.933333, 1.966667, 2.0, 2.033333, 2.066667, 2.1, 2.133333, 2.166667, 2.2, 2.233333, 2.266667, 2.3, 2.333333, 2.366667, 2.4, 2.433333, 2.466667, 2.5, 2.533333, 2.566667, 2.6, 2.633333, 2.666667, 2.7, 2.733333, 2.766667, 2.8, 2.833333, 2.866667, 2.9, 2.933333, 2.966667, 3.0, 3.033333, 3.066667, 3.1, 3.133333, 3.166667, 3.2, 3.233333, 3.266667, 3.3, 3.333333, 3.366667, 3.4, 3.433333, 3.466667, 3.5, 3.533333, 3.566667, 3.6, 3.633333, 3.666667, 3.7, 3.733333, 3.766667, 3.8, 3.833333, 3.866667, 3.9, 3.933333, 3.966667, 4.0, 4.033333, 4.066667, 4.1, 4.133333, 4.166667, 4.2, 4.233333, 4.266667, 4.3, 4.333333, 4.366667, 4.4, 4.433333, 4.466667, 4.5, 4.533333, 4.566667, 4.6, 4.633333, 4.666667, 4.7, 4.733333, 4.766667, 4.8, 4.833333, 4.866667, 4.9, 4.933333, 4.966667, 5.0, 5.033333, 5.066667, 5.1, 5.133333, 5.166667, 5.2, 5.233333, 5.266667, 5.3, 5.333333, 5.366667, 5.4, 5.433333, 5.466667, 5.5, 5.533333, 5.566667, 5.6, 5.633333, 5.666667, 5.7, 5.733333, 5.766667, 5.8, 5.833333, 5.866667, 5.9, 5.933333, 5.966667, 6.0, 6.033333, 6.066667, 6.1, 6.133333, 6.166667, 6.2, 6.233333, 6.266667, 6.3, 6.333333, 6.366667, 6.4, 6.433333, 6.466667, 6.5, 6.533333, 6.566667, 6.6, 6.633333, 6.666667, 6.7, 6.733333, 6.766667, 6.8, 6.833333, 6.866667, 6.9, 6.933333, 6.966667, 7.0, 7.033333, 7.066667, 7.1, 7.133333, 7.166667, 7.2, 7.233333, 7.266667, 7.3, 7.333333, 7.366667, 7.4, 7.433333, 7.466667, 7.5, 7.533333, 7.566667, 7.6, 7.633333, 7.666667, 7.7, 7.733333, 7.766667, 7.8, 7.833333, 7.866667, 7.9, 7.933333, 7.966667, 8.0, 8.033333, 8.066667, 8.1, 8.133333, 8.166667, 8.2, 8.233333, 8.266667, 8.3, 8.333333, 8.366667, 8.4, 8.433333, 8.466667, 8.5, 8.533333, 8.566667], "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [1.3, 2.7, 3.7, 5.5, 6.3, 7.1, 8.0], "pred_curve_point_progress": [0.0, 10.0, 15.0, 10.0, -15.0, -25.0, -15.0], "pred_dense_progress_aligned_to_gt": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.23809285714285752, 0.47619285714285736, 0.7142857142857133, 0.9523785714285709, 1.1904785714285706, 1.4285714285714282, 1.6666642857142857, 1.9047642857142855, 2.1428571428571432, 2.380949999999999, 2.6190499999999988, 2.8571428571428563, 3.095235714285714, 3.3333357142857136, 3.571428571428571, 3.8095214285714287, 4.0476214285714285, 4.285714285714285, 4.523807142857142, 4.761907142857142, 4.999999999999999, 5.238092857142855, 5.476192857142855, 5.714285714285714, 5.95237857142857, 6.19047857142857, 6.428571428571429, 6.666664285714285, 6.904764285714285, 7.142857142857141, 7.3809499999999995, 7.61905, 7.857142857142856, 8.095235714285714, 8.333335714285715, 8.571428571428571, 8.809521428571426, 9.047621428571427, 9.285714285714285, 9.523807142857141, 9.761907142857142, 10.0, 10.166664999999998, 10.333334999999998, 10.499999999999998, 10.666665, 10.833335, 10.999999999999998, 11.166665, 11.333335, 11.5, 11.666664999999998, 11.833334999999998, 12.0, 12.166664999999998, 12.333334999999998, 12.5, 12.666664999999998, 12.833334999999998, 12.999999999999998, 13.166665, 13.333335, 13.499999999999998, 13.666665, 13.833335, 14.0, 14.166664999999998, 14.333334999999998, 14.5, 14.666664999999998, 14.833334999999998, 15.0, 14.907408333333334, 14.814813888888889, 14.722222222222223, 14.629630555555556, 14.537036111111112, 14.444444444444445, 14.351852777777777, 14.259258333333333, 14.166666666666668, 14.074075, 13.981480555555557, 13.88888888888889, 13.796297222222222, 13.703702777777778, 13.61111111111111, 13.518519444444445, 13.425925, 13.333333333333334, 13.240741666666668, 13.148147222222224, 13.055555555555555, 12.962963888888888, 12.870369444444444, 12.777777777777779, 12.685186111111111, 12.592591666666667, 12.500000000000002, 12.407408333333333, 12.314813888888889, 12.222222222222221, 12.129630555555556, 12.037036111111112, 11.944444444444445, 11.85185277777778, 11.759258333333335, 11.666666666666666, 11.574075, 11.481480555555555, 11.38888888888889, 11.296297222222222, 11.203702777777778, 11.111111111111112, 11.018519444444443, 10.925925, 10.833333333333332, 10.740741666666667, 10.648147222222223, 10.555555555555557, 10.46296388888889, 10.370369444444446, 10.277777777777777, 10.185186111111111, 10.092591666666667, 10.0, 8.958343750000004, 7.916656250000005, 6.875000000000011, 5.833343749999988, 4.791656249999988, 3.749999999999993, 2.7083437499999983, 1.666656249999999, 0.6250000000000036, -0.41665624999999196, -1.4583437499999903, -2.500000000000014, -3.5416562500000097, -4.583343750000008, -5.6250000000000036, -6.666656249999999, -7.708343749999997, -8.749999999999993, -9.791656250000017, -10.833343750000015, -11.87500000000001, -12.916656250000006, -13.958343750000004, -15.0, -15.416662499999997, -15.833337499999997, -16.250000000000007, -16.666662500000005, -17.083337500000006, -17.500000000000004, -17.9166625, -18.3333375, -18.75, -19.166662500000008, -19.583337500000006, -20.000000000000007, -20.416662500000005, -20.833337500000003, -21.25, -21.6666625, -22.0833375, -22.500000000000007, -22.916662500000008, -23.333337500000006, -23.750000000000007, -24.1666625, -24.583337500000003, -25.0, -24.629633333333324, -24.259255555555548, -23.888888888888882, -23.51852222222222, -23.14814444444444, -22.77777777777778, -22.407411111111113, -22.037033333333333, -21.66666666666666, -21.296299999999995, -20.92592222222222, -20.555555555555554, -20.185188888888888, -19.814811111111112, -19.444444444444446, -19.074077777777774, -18.703699999999994, -18.33333333333333, -17.962966666666667, -17.592588888888887, -17.22222222222222, -16.85185555555556, -16.481477777777783, -16.111111111111107, -15.740744444444442, -15.370366666666664, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0, -15.0], "curve_compare_metrics": {"point_count": 258, "mae": 2.4426571920757962, "rmse": 3.2500268666664343, "pearson": 0.977103530223041, "spearman": 0.9351348813030206}, "curve_compare_plot": "examples/reference_outputs/example_01_gt_vs_pred.png"}
|
|
|
|
| 1 |
+
{"example_id": "example_01", "bucket": "test_nonexpert_seen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251130-162433-bad-purple cube-shaped building block/videos/chunk-000/observation.images.front/episode_000007", "task_instruction": "抓取紫色方块使其从方形洞口落入积木桶中", "task_description": "抓取紫色方块使其从方形洞口落入积木桶中：\n爪夹开始移动：0%\n爪夹接近紫色方块：20%\n爪夹抓紧紫色方块：40%\n爪夹抓紧紫色方块接近方形洞口：60%\n爪夹将紫色方块对准方形洞口：80%\n紫色方块从方形洞口落入积木桶中，爪夹移开：100%", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 255], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5], "input_frame_count_2hz": 18, "decoded_video_stats": {"decoded_frame_count": 258.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 抓取紫色方块使其从方形洞口落入积木桶中\n爪夹开始移动：0%\n爪夹接近紫色方块：20%\n爪夹抓紧紫色方块：40%\n爪夹抓紧紫色方块接近方形洞口：60%\n爪夹将紫色方块对准方形洞口：80%\n紫色方块从方形洞口落入积木桶中，爪夹移开：100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 1.3s, 进度: 0%\n时间: 2.7s, 进度: 10%\n时间: 3.7s, 进度: 15%\n时间: 5.5s, 进度: 10%\n时间: 6.3s, 进度: -15%\n时间: 7.1s, 进度: -25%\n时间: 8.0s, 进度: -15%", "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [1.3, 2.7, 3.7, 5.5, 6.3, 7.1, 8.0], "pred_curve_point_progress": [0.0, 10.0, 15.0, 10.0, -15.0, -25.0, -15.0], "video_path": "examples/example_01/episode.mp4", "preview_video_path": "examples/reference_outputs/example_01_pred_progress.mp4"}
|
examples/reference_outputs/{summary_gt_vs_pred.png → example_01_pred_progress.mp4}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:95c5e7bb99377d2844d15d4772313e749e8cd5a176ab03ca19ebc54a5ab30a7a
|
| 3 |
+
size 2088505
|
examples/reference_outputs/example_02.jsonl
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"example_id": "example_02", "bucket": "test_nonexpert_unseen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251128-183101-bad-onion-express-box/videos/chunk-000/observation.images.front/episode_000000", "task_instruction": "把洋葱放进快递箱里。", "task_description": "把洋葱放进快递箱里。\n开始移动:0%\n爪夹靠近洋葱:20%\n爪夹抓取洋葱:40%\n爪夹抓住洋葱接近快递箱:60%\n爪夹将洋葱放入快递箱:80%\n爪夹松开,洋葱落入快递箱:100%", "video_path": "examples/example_02/episode.mp4", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 255, 270, 285], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5], "input_frame_count_2hz": 20, "decoded_video_stats": {"decoded_frame_count": 289.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 把洋葱放进快递箱里。\n开始移动:0%\n爪夹靠近洋葱:20%\n爪夹抓取洋葱:40%\n爪夹抓住洋葱接近快递箱:60%\n爪夹将洋葱放入快递箱:80%\n爪夹松开,洋葱落入快递箱:100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 0.0s, 进度: 0%\n时间: 2.3s, 进度: 5%\n时间: 3.3s, 进度: 8%\n时间: 4.3s, 进度: 10%\n时间: 5.1s, 进度: -5%\n时间: 6.1s, 进度: -8%\n时间: 7.1s, 进度: -2%\n时间: 8.3s, 进度: 2%\n时间: 9.5s, 进度: 4%", "benchmark_progress_type": "dense_kinematic_progress", "benchmark_progress_source": {"bucket": "test_nonexpert_unseen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251128-183101-bad-onion-express-box/videos/chunk-000/observation.images.front/episode_000000"}, "benchmark_semantic_anchors": [{"frame": 0, "progress": 0.0}, {"frame": 70, "progress": 10.0}, {"frame": 130, "progress": 18.0}, {"frame": 162, "progress": 5.0}, {"frame": 240, "progress": -5.0}, {"frame": 272, "progress": -10.0}], "gt_dense_progress": [0.0, 0.1, 0.3, 0.4, 0.6, 0.7, 0.9, 1.0, 1.1, 1.3, 1.4, 1.6, 1.7, 1.9, 2.0, 2.1, 2.3, 2.4, 2.6, 2.7, 2.9, 3.0, 3.1, 3.3, 3.4, 3.6, 3.7, 3.9, 4.0, 4.1, 4.3, 4.4, 4.6, 4.7, 4.9, 5.0, 5.1, 5.3, 5.4, 5.6, 5.7, 5.9, 6.0, 6.1, 6.3, 6.4, 6.6, 6.7, 6.9, 7.0, 7.1, 7.3, 7.4, 7.6, 7.7, 7.9, 8.0, 8.1, 8.3, 8.4, 8.6, 8.7, 8.9, 9.0, 9.1, 9.3, 9.4, 9.6, 9.7, 9.9, 10.0, 10.1, 10.3, 10.4, 10.5, 10.7, 10.8, 10.9, 11.1, 11.2, 11.3, 11.5, 11.6, 11.7, 11.9, 12.0, 12.1, 12.3, 12.4, 12.5, 12.7, 12.8, 12.9, 13.1, 13.2, 13.3, 13.5, 13.6, 13.7, 13.9, 14.0, 14.1, 14.3, 14.4, 14.5, 14.7, 14.8, 14.9, 15.1, 15.2, 15.3, 15.5, 15.6, 15.7, 15.9, 16.0, 16.1, 16.3, 16.4, 16.5, 16.7, 16.8, 16.9, 17.1, 17.2, 17.3, 17.5, 17.6, 17.7, 17.9, 18.0, 17.6, 17.2, 16.8, 16.4, 16.0, 15.6, 15.2, 14.8, 14.3, 13.9, 13.5, 13.1, 12.7, 12.3, 11.9, 11.5, 11.1, 10.7, 10.3, 9.9, 9.5, 9.1, 8.7, 8.2, 7.8, 7.4, 7.0, 6.6, 6.2, 5.8, 5.4, 5.0, 4.9, 4.7, 4.6, 4.5, 4.4, 4.2, 4.1, 4.0, 3.8, 3.7, 3.6, 3.5, 3.3, 3.2, 3.1, 2.9, 2.8, 2.7, 2.6, 2.4, 2.3, 2.2, 2.1, 1.9, 1.8, 1.7, 1.5, 1.4, 1.3, 1.2, 1.0, 0.9, 0.8, 0.6, 0.5, 0.4, 0.3, 0.1, 0.0, -0.1, -0.3, -0.4, -0.5, -0.6, -0.8, -0.9, -1.0, -1.2, -1.3, -1.4, -1.5, -1.7, -1.8, -1.9, -2.1, -2.2, -2.3, -2.4, -2.6, -2.7, -2.8, -2.9, -3.1, -3.2, -3.3, -3.5, -3.6, -3.7, -3.8, -4.0, -4.1, -4.2, -4.4, -4.5, -4.6, -4.7, -4.9, -5.0, -5.2, -5.3, -5.5, -5.6, -5.8, -5.9, -6.1, -6.2, -6.4, -6.6, -6.7, -6.9, -7.0, -7.2, -7.3, -7.5, -7.7, -7.8, -8.0, -8.1, -8.3, -8.4, -8.6, -8.8, -8.9, -9.1, -9.2, -9.4, -9.5, -9.7, -9.8, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0, -10.0], "gt_dense_timestamps_sec": [0.0, 0.033333, 0.066667, 0.1, 0.133333, 0.166667, 0.2, 0.233333, 0.266667, 0.3, 0.333333, 0.366667, 0.4, 0.433333, 0.466667, 0.5, 0.533333, 0.566667, 0.6, 0.633333, 0.666667, 0.7, 0.733333, 0.766667, 0.8, 0.833333, 0.866667, 0.9, 0.933333, 0.966667, 1.0, 1.033333, 1.066667, 1.1, 1.133333, 1.166667, 1.2, 1.233333, 1.266667, 1.3, 1.333333, 1.366667, 1.4, 1.433333, 1.466667, 1.5, 1.533333, 1.566667, 1.6, 1.633333, 1.666667, 1.7, 1.733333, 1.766667, 1.8, 1.833333, 1.866667, 1.9, 1.933333, 1.966667, 2.0, 2.033333, 2.066667, 2.1, 2.133333, 2.166667, 2.2, 2.233333, 2.266667, 2.3, 2.333333, 2.366667, 2.4, 2.433333, 2.466667, 2.5, 2.533333, 2.566667, 2.6, 2.633333, 2.666667, 2.7, 2.733333, 2.766667, 2.8, 2.833333, 2.866667, 2.9, 2.933333, 2.966667, 3.0, 3.033333, 3.066667, 3.1, 3.133333, 3.166667, 3.2, 3.233333, 3.266667, 3.3, 3.333333, 3.366667, 3.4, 3.433333, 3.466667, 3.5, 3.533333, 3.566667, 3.6, 3.633333, 3.666667, 3.7, 3.733333, 3.766667, 3.8, 3.833333, 3.866667, 3.9, 3.933333, 3.966667, 4.0, 4.033333, 4.066667, 4.1, 4.133333, 4.166667, 4.2, 4.233333, 4.266667, 4.3, 4.333333, 4.366667, 4.4, 4.433333, 4.466667, 4.5, 4.533333, 4.566667, 4.6, 4.633333, 4.666667, 4.7, 4.733333, 4.766667, 4.8, 4.833333, 4.866667, 4.9, 4.933333, 4.966667, 5.0, 5.033333, 5.066667, 5.1, 5.133333, 5.166667, 5.2, 5.233333, 5.266667, 5.3, 5.333333, 5.366667, 5.4, 5.433333, 5.466667, 5.5, 5.533333, 5.566667, 5.6, 5.633333, 5.666667, 5.7, 5.733333, 5.766667, 5.8, 5.833333, 5.866667, 5.9, 5.933333, 5.966667, 6.0, 6.033333, 6.066667, 6.1, 6.133333, 6.166667, 6.2, 6.233333, 6.266667, 6.3, 6.333333, 6.366667, 6.4, 6.433333, 6.466667, 6.5, 6.533333, 6.566667, 6.6, 6.633333, 6.666667, 6.7, 6.733333, 6.766667, 6.8, 6.833333, 6.866667, 6.9, 6.933333, 6.966667, 7.0, 7.033333, 7.066667, 7.1, 7.133333, 7.166667, 7.2, 7.233333, 7.266667, 7.3, 7.333333, 7.366667, 7.4, 7.433333, 7.466667, 7.5, 7.533333, 7.566667, 7.6, 7.633333, 7.666667, 7.7, 7.733333, 7.766667, 7.8, 7.833333, 7.866667, 7.9, 7.933333, 7.966667, 8.0, 8.033333, 8.066667, 8.1, 8.133333, 8.166667, 8.2, 8.233333, 8.266667, 8.3, 8.333333, 8.366667, 8.4, 8.433333, 8.466667, 8.5, 8.533333, 8.566667, 8.6, 8.633333, 8.666667, 8.7, 8.733333, 8.766667, 8.8, 8.833333, 8.866667, 8.9, 8.933333, 8.966667, 9.0, 9.033333, 9.066667, 9.1, 9.133333, 9.166667, 9.2, 9.233333, 9.266667, 9.3, 9.333333, 9.366667, 9.4, 9.433333, 9.466667, 9.5, 9.533333, 9.566667, 9.6], "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [0.0, 2.3, 3.3, 4.3, 5.1, 6.1, 7.1, 8.3, 9.5], "pred_curve_point_progress": [0.0, 5.0, 8.0, 10.0, -5.0, -8.0, -2.0, 2.0, 4.0], "pred_dense_progress_aligned_to_gt": [0.0, 0.07246304347826088, 0.14492826086956526, 0.21739130434782614, 0.289854347826087, 0.3623195652173914, 0.4347826086956523, 0.5072456521739132, 0.5797108695652174, 0.6521739130434784, 0.7246369565217392, 0.7971021739130436, 0.8695652173913045, 0.9420282608695654, 1.0144934782608697, 1.0869565217391306, 1.1594195652173913, 1.2318847826086958, 1.3043478260869568, 1.3768108695652177, 1.449276086956522, 1.5217391304347827, 1.5942021739130436, 1.6666673913043482, 1.739130434782609, 1.8115934782608698, 1.884058695652174, 1.956521739130435, 2.028984782608696, 2.1014500000000003, 2.173913043478261, 2.246376086956522, 2.3188413043478264, 2.3913043478260874, 2.463767391304348, 2.536232608695652, 2.6086956521739135, 2.6811586956521745, 2.7536239130434788, 2.8260869565217397, 2.8985500000000006, 2.971015217391305, 3.0434782608695654, 3.1159413043478263, 3.1884065217391306, 3.2608695652173916, 3.333332608695653, 3.4057978260869572, 3.478260869565218, 3.5507239130434787, 3.623189130434783, 3.695652173913044, 3.768115217391305, 3.840580434782609, 3.91304347826087, 3.9855065217391314, 4.057971739130435, 4.130434782608696, 4.202897826086957, 4.2753630434782615, 4.347826086956522, 4.420289130434783, 4.492754347826088, 4.565217391304349, 4.6376804347826095, 4.710145652173914, 4.782608695652175, 4.855071739130436, 4.92753695652174, 5.0, 5.099999, 5.200001, 5.300000000000001, 5.399999000000001, 5.500001000000001, 5.6000000000000005, 5.699999, 5.800001, 5.9, 5.999999000000001, 6.100001000000001, 6.200000000000001, 6.299999000000001, 6.4000010000000005, 6.5, 6.599999, 6.700001, 6.800000000000001, 6.899999000000001, 7.000001000000001, 7.1000000000000005, 7.199999, 7.300001, 7.4, 7.499999000000001, 7.600001000000001, 7.700000000000001, 7.799999000000001, 7.9000010000000005, 8.0, 8.066666000000001, 8.133334000000001, 8.2, 8.266666, 8.333334, 8.4, 8.466666, 8.533334, 8.600000000000001, 8.666666, 8.733334, 8.8, 8.866666, 8.933334, 9.0, 9.066666000000001, 9.133334000000001, 9.2, 9.266666, 9.333334, 9.4, 9.466666, 9.533334, 9.6, 9.666666000000001, 9.733334000000001, 9.8, 9.866666, 9.933334, 10.0, 9.375006250000004, 8.749993750000003, 8.12499999999999, 7.500006249999993, 6.874993749999993, 6.2499999999999964, 5.625006249999999, 4.999993749999999, 4.375000000000003, 3.7500062499999887, 3.124993749999989, 2.499999999999992, 1.8750062499999949, 1.2499937499999945, 0.6249999999999982, 6.250000001983835e-06, -0.6250062499999984, -1.2500000000000124, -1.8749937500000087, -2.500006250000009, -3.1250000000000053, -3.7499937500000033, -4.375006250000002, -5.0, -5.099999000000002, -5.200001000000002, -5.300000000000002, -5.399999000000001, -5.500001000000001, -5.6000000000000005, -5.699999, -5.800001, -5.900000000000002, -5.999999000000002, -6.100001000000002, -6.200000000000001, -6.299999000000001, -6.4000010000000005, -6.5, -6.599999000000002, -6.700001000000002, -6.800000000000002, -6.899999000000001, -7.000001000000001, -7.1000000000000005, -7.199999, -7.300001, -7.400000000000002, -7.499999000000002, -7.600001000000002, -7.700000000000001, -7.799999000000001, -7.9000010000000005, -8.0, -7.800001999999996, -7.599997999999996, -7.399999999999997, -7.200001999999998, -6.999997999999998, -6.799999999999999, -6.600002, -6.399998, -6.199999999999996, -6.000001999999997, -5.799997999999997, -5.599999999999998, -5.400001999999999, -5.199997999999999, -5.0, -4.800001999999996, -4.599997999999996, -4.399999999999997, -4.200001999999998, -3.999997999999998, -3.799999999999999, -3.600002, -3.399998, -3.1999999999999957, -3.0000019999999967, -2.799997999999997, -2.599999999999998, -2.400001999999999, -2.199997999999999, -2.0, -1.8888899999999977, -1.7777766666666646, -1.6666666666666652, -1.5555566666666658, -1.4444433333333326, -1.3333333333333335, -1.2222233333333339, -1.111110000000001, -0.9999999999999984, -0.8888899999999991, -0.7777766666666661, -0.6666666666666667, -0.5555566666666674, -0.4444433333333342, -0.3333333333333348, -0.22222333333333255, -0.11110999999999938, 0.0, 0.11110999999999915, 0.22222333333333255, 0.3333333333333317, 0.4444433333333313, 0.5555566666666643, 0.6666666666666665, 0.7777766666666661, 0.8888899999999991, 0.9999999999999987, 1.111110000000001, 1.2222233333333339, 1.3333333333333304, 1.4444433333333326, 1.555556666666666, 1.666666666666662, 1.7777766666666643, 1.8888899999999977, 2.0, 2.0555549999999982, 2.1111116666666647, 2.166666666666666, 2.2222216666666643, 2.2777783333333312, 2.333333333333332, 2.3888883333333335, 2.4444450000000004, 2.4999999999999987, 2.555555, 2.6111116666666665, 2.6666666666666647, 2.722221666666666, 2.7777783333333326, 2.833333333333334, 2.888888333333332, 2.9444449999999986, 3.0, 3.0555549999999982, 3.111111666666665, 3.166666666666666, 3.2222216666666674, 3.277778333333334, 3.333333333333332, 3.3888883333333335, 3.444445, 3.4999999999999982, 3.555555, 3.6111116666666665, 3.666666666666668, 3.722221666666666, 3.7777783333333326, 3.833333333333334, 3.888888333333332, 3.9444449999999986, 4.0, 4.0, 4.0, 4.0], "curve_compare_metrics": {"point_count": 289, "mae": 6.6085736332179925, "rmse": 7.729949344246744, "pearson": 0.43327740758574307, "spearman": 0.5164980259299339}, "curve_compare_plot": "examples/reference_outputs/example_02_gt_vs_pred.png"}
|
|
|
|
| 1 |
+
{"example_id": "example_02", "bucket": "test_nonexpert_unseen", "global_episode_id": "ARX-data/human-data-1204-2cameras/20251128-183101-bad-onion-express-box/videos/chunk-000/observation.images.front/episode_000000", "task_instruction": "把洋葱放进快递箱里。", "task_description": "把洋葱放进快递箱里。\n开始移动:0%\n爪夹靠近洋葱:20%\n爪夹抓取洋葱:40%\n爪夹抓住洋葱接近快递箱:60%\n爪夹将洋葱放入快递箱:80%\n爪夹松开,洋葱落入快递箱:100%", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165, 180, 195, 210, 225, 240, 255, 270, 285], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0, 7.5, 8.0, 8.5, 9.0, 9.5], "input_frame_count_2hz": 20, "decoded_video_stats": {"decoded_frame_count": 289.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 把洋葱放进快递箱里。\n开始移动:0%\n爪夹靠近洋葱:20%\n爪夹抓取洋葱:40%\n爪夹抓住洋葱接近快递箱:60%\n爪夹将洋葱放入快递箱:80%\n爪夹松开,洋葱落入快递箱:100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 0.0s, 进度: 0%\n时间: 2.3s, 进度: 5%\n时间: 3.3s, 进度: 8%\n时间: 4.3s, 进度: 10%\n时间: 5.1s, 进度: -5%\n时间: 6.1s, 进度: -8%\n时间: 7.1s, 进度: -2%\n时间: 8.3s, 进度: 2%\n时间: 9.5s, 进度: 4%", "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [0.0, 2.3, 3.3, 4.3, 5.1, 6.1, 7.1, 8.3, 9.5], "pred_curve_point_progress": [0.0, 5.0, 8.0, 10.0, -5.0, -8.0, -2.0, 2.0, 4.0], "video_path": "examples/example_02/episode.mp4", "preview_video_path": "examples/reference_outputs/example_02_pred_progress.mp4"}
|
examples/reference_outputs/{example_01_gt_vs_pred.png → example_02_pred_progress.mp4}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:013ffd92c2eeecebbbf748f2bc543b5ad62d9b52635b52e186c6d2d5bf48811e
|
| 3 |
+
size 2603165
|
examples/reference_outputs/example_03.jsonl
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"example_id": "example_03", "bucket": "test_expert_seen", "global_episode_id": "ARX-data/human-data-0114/20260112-yuanshuaijun/20260111-204118-put-triangular-beaker-onto-tripod-good2/videos/chunk-000/observation.images.front/episode_000002", "task_instruction": "将三角烧杯放在三脚架上。", "task_description": "将三角烧杯放在三脚架上。\n爪夹准备移动：0%\n爪夹开始移动：10%\n爪夹靠近三角烧杯：20%\n爪夹抓取三角烧杯：40%\n爪夹合拢并靠近三脚架：60%\n爪夹移动到三脚架的正上方：80%\n爪夹松开，三角烧杯落在三脚架上：100%", "prompt_variant": "chunk_all", "input_sample_hz": 2.0, "input_frame_indices_2hz": [0, 15, 30, 45, 60, 75, 90, 105, 120, 135, 150, 165], "input_timestamps_sec_2hz": [0.0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5], "input_frame_count_2hz": 12, "decoded_video_stats": {"decoded_frame_count": 172.0, "decoded_avg_fps": 30.0}, "prompt": "任务描述和具体规划: 将三角烧杯放在三脚架上。\n爪夹准备移动：0%\n爪夹开始移动：10%\n爪夹靠近三角烧杯：20%\n爪夹抓取三角烧杯：40%\n爪夹合拢并靠近三脚架：60%\n爪夹移动到三脚架的正上方：80%\n爪夹松开，三角烧杯落在三脚架上：100%\n\n请根据任务描述和具体规划，找到并逐点生成视频中的关键动作点和相应的进度标注。输出格式要求：每个关键点一行，格式为：\n时间: X.Xs, 进度: Y%\n\n请严格按照上述格式输出，不要输出额外说明。", "response": "时间: 0.1s, 进度: 0%\n时间: 0.6s, 进度: 10%\n时间: 1.1s, 进度: 20%\n时间: 1.6s, 进度: 30%\n时间: 2.3s, 进度: 40%\n时间: 3.4s, 进度: 60%\n时间: 4.1s, 进度: 80%\n时间: 5.3s, 进度: 100%", "pred_curve_parse_ok": true, "pred_curve_point_times_sec": [0.1, 0.6, 1.1, 1.6, 2.3, 3.4, 4.1, 5.3], "pred_curve_point_progress": [0.0, 10.0, 20.0, 30.0, 40.0, 60.0, 80.0, 100.0], "video_path": "examples/example_03/episode.mp4", "preview_video_path": "examples/reference_outputs/example_03_pred_progress.mp4"}
|
examples/reference_outputs/{example_02_gt_vs_pred.png → example_03_pred_progress.mp4}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:684ef749e15396234157b0d09b536f0020b48b34eefdc26f8feb9b7f81d218de
|
| 3 |
+
size 1119731
|