GnanaPraveen commited on
Commit
7900dfe
·
verified ·
1 Parent(s): a0acfdb

docs: synchronize public release artifacts

Browse files
Files changed (37) hide show
  1. README.md +13 -21
  2. RELEASE.json +1 -1
  3. inference/README.md +4 -4
  4. inference/SHA256SUMS +31 -15
  5. inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md +355 -103
  6. inference/policy/Bagel_Robotics_1/LICENSE +6 -0
  7. inference/policy/Bagel_Robotics_1/README.md +6 -6
  8. inference/policy/Bagel_Robotics_1/REEVALUATION.md +12 -10
  9. inference/policy/Bagel_Robotics_1/THIRD_PARTY_NOTICES.md +0 -13
  10. inference/policy/Bagel_Robotics_1/VALIDATION.md +13 -3
  11. inference/policy/Bagel_Robotics_1/artifact-manifest.json +1 -1
  12. inference/policy/Bagel_Robotics_1/download_checkpoint.py +1 -1
  13. inference/policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md +102 -0
  14. inference/policy/Bagel_Robotics_1/evaluation/seed-1/summary.md +101 -0
  15. inference/policy/Bagel_Robotics_1/evaluation/seed-1/verification.json +21 -0
  16. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes +3 -0
  17. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md +5 -0
  18. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/SHA256SUMS +14 -0
  19. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv +0 -0
  20. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py +106 -0
  21. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md +5 -0
  22. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json +0 -0
  23. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv +30 -0
  24. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md +5 -0
  25. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md +5 -0
  26. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv +180 -0
  27. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md +101 -0
  28. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md +101 -0
  29. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv +32 -0
  30. inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json +213 -0
  31. inference/policy/Bagel_Robotics_1/evidence/pr-audit.json +6 -3
  32. inference/policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json +187 -0
  33. inference/policy/Bagel_Robotics_1/runtime/LICENSE +6 -0
  34. inference/policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md +0 -8
  35. inference/policy/Bagel_Robotics_1/runtime/pyproject.toml +1 -1
  36. inference/policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py +1 -1
  37. inference/policy/Bagel_Robotics_1/source-provenance.json +3 -2
README.md CHANGED
@@ -2,14 +2,12 @@
2
  language: en
3
  tags:
4
  - robotics
5
- - vpp2
6
  - structured-pruning
7
  - robodojo
8
- ---
9
 
10
  # Bagel-Robotics-1 — checkpoint 1000
11
 
12
- Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is the validation base; reused source components are credited in the notices below.
13
 
14
  Inference release of the **exact checkpoint 1000** used in the Bagel-Robotics-1 evaluation campaign. **The repository is public and ungated.** Anyone can download the checkpoint and inference files without a Hugging Face account or access token.
15
 
@@ -19,15 +17,13 @@ The video expert has **12,468,409,664 parameters**, 23.95% fewer than its record
19
 
20
  - Original checkpoint revision: `2651631c5d08819088475e8778676f5cf6d9afcf`.
21
  - Identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
22
- - [artifact-manifest.json](artifact-manifest.json) pins all six bundle files and the separate public encoder/tokenizer assets.
23
- - [SHA256SUMS](SHA256SUMS) records the unchanged SHA-256 hashes of all six bundle files.
24
 
25
- The release adds documentation and inference packaging without changing any checkpoint file. The original revision is preserved, so historical provenance remains accessible in Git history. Current release files focus on inference. `COMPLETE.json` is the original export receipt, including export-time paths and `simulation_completed: false`; it is retained unchanged for integrity, not used as a current evaluation-status claim. Those historical paths are not required for deployment.
26
 
27
  ## Download the exact checkpoint
28
 
29
- The checkpoint is publicly downloadable at the pinned revision below. No login or access token is required.
30
-
31
  ```bash
32
  hf download bageldotcom/Bagel-Robotics-1 \
33
  --revision 2651631c5d08819088475e8778676f5cf6d9afcf \
@@ -35,35 +31,31 @@ hf download bageldotcom/Bagel-Robotics-1 \
35
  --local-dir /absolute/path/to/checkpoints/cp1000
36
  ```
37
 
38
- There are approximately 29.0 GB of checkpoint tensors. Keep all six files together. The inference downloader checks their sizes and SHA-256 hashes and downloads approximately 16.7 GB of separately pinned VAE/conditioning/tokenizer assets. Allow extra disk space for caches and installation.
39
 
40
  ## Inference adapter and reevaluation
41
 
42
- The standalone XPolicyLab adapter is provided under [`inference/`](inference/), at release tag `bagel-robotics-1-cp1000-inference-v2`. No private research repository is required. Historical tagged documentation may still describe private access; all revisions are now public and require no access token. Use the current linked instructions below.
43
 
44
  ```bash
45
  hf download bageldotcom/Bagel-Robotics-1 \
46
- --revision bagel-robotics-1-cp1000-inference-v2 \
47
  --include 'inference/**' \
48
  --local-dir /absolute/path/to/Bagel-Robotics-1-release
49
 
50
- git clone https://github.com/XPolicyLab/XPolicyLab.git /absolute/path/to/workspace/XPolicyLab
51
- git -C /absolute/path/to/workspace/XPolicyLab checkout 6baccd57ff958570d0db1588370289ee36c8bb70
52
  python /absolute/path/to/Bagel-Robotics-1-release/inference/install_adapter.py /absolute/path/to/workspace/XPolicyLab
53
  ```
54
 
55
- Follow the current [policy README](inference/policy/Bagel_Robotics_1/README.md) to install the Python 3.10 environment, download/verify assets and run both debug checks. Follow [REEVALUATION.md](inference/policy/Bagel_Robotics_1/REEVALUATION.md) for task, suite and three-seed commands. Install RoboDojo and simulator assets separately using its [official guide](https://robodojo-benchmark.com/doc/usage/install-and-download/).
56
-
57
- The adapter uses `policy_name=Bagel_Robotics_1`, `ckpt_name=cp1000`, ARX X5 end-effector actions, and one environment per policy process. Fixed inference settings: BF16; predict 32 actions and execute 16; ten Euler steps; shift 1; 90% new / 10% previous temporal blending. Qualification used a 96 GB RTX PRO 6000 Blackwell, Python 3.10.21 and PyTorch 2.11.0+cu130; a lower minimum GPU capacity has not been qualified.
58
 
59
- Use the Bagel-Robotics-1 compact checkpoint loader supplied in the inference package. The implementation and current reviewer instructions are available in [XPolicyLab PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234).
60
 
61
  ## Validation and release scope
62
 
63
- The adapter's existing qualification includes 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes, and eight exact action/history/reset/backend checks on the tested recordings. Full multi-seed official leaderboard validation remains separate. Historical research scores and scene exceptions are documented in the adapter's EVALUATION_RESULTS.md; they are not official leaderboard results.
64
 
65
- The technical report and training/pruning/export code are planned for **November 20, 2026**. This release contains inference code and weights; optimizer, scheduler, distributed RNG/sampler state, training datasets and simulator assets are not included. It does not support exact training continuation from weights alone.
66
 
67
- ## Attribution and terms
68
 
69
- VPP2 is the base model used to validate the Bagel-Robotics-1 pruning, recovery-training and data-selection approach. Reused source components come from [VPP2](https://github.com/roboterax/video-prediction-policy-2)/FastWAM and Wan2.1/DiffSynth. See [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and the adapter's retained MIT/Apache-2.0 notices. Model and auxiliary-asset upstream terms continue to apply; an inference-code license is not a new blanket license for all weights or datasets.
 
2
  language: en
3
  tags:
4
  - robotics
 
5
  - structured-pruning
6
  - robodojo
 
7
 
8
  # Bagel-Robotics-1 — checkpoint 1000
9
 
10
+ Bagel-Robotics-1 combines task-critical data selection, adaptive structured pruning and recovery training to produce a compact world-action model. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
11
 
12
  Inference release of the **exact checkpoint 1000** used in the Bagel-Robotics-1 evaluation campaign. **The repository is public and ungated.** Anyone can download the checkpoint and inference files without a Hugging Face account or access token.
13
 
 
17
 
18
  - Original checkpoint revision: `2651631c5d08819088475e8778676f5cf6d9afcf`.
19
  - Identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
20
+ - [`artifact-manifest.json`](artifact-manifest.json) pins all six bundle files and the separate public encoder/tokenizer assets.
21
+ - [`SHA256SUMS`](SHA256SUMS) records the unchanged SHA-256 hashes of all six bundle files.
22
 
23
+ The release adds documentation and inference packaging without changing any checkpoint file. The original revision is preserved, so historical provenance remains accessible in Git history. `COMPLETE.json` is the original export receipt, retained unchanged for integrity and not used as a current evaluation-status claim.
24
 
25
  ## Download the exact checkpoint
26
 
 
 
27
  ```bash
28
  hf download bageldotcom/Bagel-Robotics-1 \
29
  --revision 2651631c5d08819088475e8778676f5cf6d9afcf \
 
31
  --local-dir /absolute/path/to/checkpoints/cp1000
32
  ```
33
 
34
+ There are approximately 29.0 GB of checkpoint tensors. Keep all six files together. The inference downloader checks their sizes and SHA-256 hashes and downloads approximately 16.7 GB of separately pinned VAE/conditioning/tokenizer assets.
35
 
36
  ## Inference adapter and reevaluation
37
 
38
+ The standalone XPolicyLab adapter is provided under [`inference/`](inference/). The current public adapter follows the instructions in [`inference/README.md`](inference/README.md); no private research repository or access token is required.
39
 
40
  ```bash
41
  hf download bageldotcom/Bagel-Robotics-1 \
 
42
  --include 'inference/**' \
43
  --local-dir /absolute/path/to/Bagel-Robotics-1-release
44
 
45
+ # Then install the adapter into an existing XPolicyLab checkout.
 
46
  python /absolute/path/to/Bagel-Robotics-1-release/inference/install_adapter.py /absolute/path/to/workspace/XPolicyLab
47
  ```
48
 
49
+ Follow [`inference/policy/Bagel_Robotics_1/README.md`](inference/policy/Bagel_Robotics_1/README.md) to install the Python 3.10 environment, download and verify assets, and run both debug checks. Follow [`REEVALUATION.md`](inference/policy/Bagel_Robotics_1/REEVALUATION.md) for task, suite and three-seed commands. Install RoboDojo and simulator assets separately using its [official guide](https://robodojo-benchmark.com/doc/usage/install-and-download/).
 
 
50
 
51
+ The adapter uses `policy_name=Bagel_Robotics_1`, `ckpt_name=cp1000`, ARX X5 end-effector actions and one environment per policy process. Fixed inference settings: BF16; predict 32 actions and execute 16; ten Euler steps; shift 1; 90% new / 10% previous temporal blending.
52
 
53
  ## Validation and release scope
54
 
55
+ The adapter qualification includes 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes and eight exact action/history/reset/backend checks on tested recordings. Full multi-seed official leaderboard validation remains separate. Historical research scores and scene exceptions are documented in [`EVALUATION_RESULTS.md`](inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md); they are not official leaderboard results.
56
 
57
+ The training, pruning and export implementation is publicly available in the [Bagel-Robotics-1 GitHub repository](https://github.com/bageldotcom/Bagel-Robotics-1). The technical report remains planned for **November 20, 2026**. This Hugging Face repository contains the inference release and weights; it does not claim that optimizer state or exact training continuation can be reconstructed from weights alone.
58
 
59
+ ## Attribution
60
 
61
+ VPP2 is acknowledged as the base model used to validate the Bagel-Robotics-1 pruning, recovery-training and data-selection approach. Applicable terms are retained in the repository license files.
RELEASE.json CHANGED
@@ -30,7 +30,7 @@
30
  "size": 24937214042
31
  }
32
  },
33
- "scope": "Public, ungated inference checkpoint and supporting adapter; no login or access token required; training/pruning/export source planned November 20, 2026",
34
  "historical_provenance": "Original checkpoint revision and checksums preserved. Older commits retain original provenance and export records; the current release view contains inference materials.",
35
  "official_leaderboard_result": false
36
  }
 
30
  "size": 24937214042
31
  }
32
  },
33
+ "scope": "Public, ungated inference checkpoint and supporting adapter; no login or access token required; training, pruning and export source are published separately in GitHub; technical report planned for November 20, 2026",
34
  "historical_provenance": "Original checkpoint revision and checksums preserved. Older commits retain original provenance and export records; the current release view contains inference materials.",
35
  "official_leaderboard_result": false
36
  }
inference/README.md CHANGED
@@ -1,8 +1,8 @@
1
  # Bagel-Robotics-1 inference adapter
2
 
3
- Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is the validation base; reused source components are credited in the policy's source notices.
4
 
5
- This folder is the standalone XPolicyLab adapter for Bagel-Robotics-1 checkpoint 1000. It contains inference code and evaluation instructions, with no training repository dependency. Model tensors download separately from `bageldotcom/Bagel-Robotics-1`, pinned to `2651631c5d08819088475e8778676f5cf6d9afcf` and verified against `policy/Bagel_Robotics_1/artifact-manifest.json`.
6
 
7
  Install this adapter into an existing XPolicyLab checkout (the installer refuses to overwrite an existing policy):
8
 
@@ -10,8 +10,8 @@ Install this adapter into an existing XPolicyLab checkout (the installer refuses
10
  python install_adapter.py /absolute/path/to/XPolicyLab
11
  ```
12
 
13
- Then follow [Installation](policy/Bagel_Robotics_1/README.md#installation) for the Python 3.10 environment, exact checkpoint and encoder download, and debug checks. [REEVALUATION.md](policy/Bagel_Robotics_1/REEVALUATION.md) gives native simulator task/suite/three-seed commands. RoboDojo and its assets are installed separately.
14
 
15
- The adapter is prepared against XPolicyLab `6baccd57ff958570d0db1588370289ee36c8bb70`; source and dependency pins are preserved in the policy folder. The adapter is under review in [XPolicyLab PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234). Training/pruning/export source and the technical report are planned for November 20, 2026.
16
 
17
  The repository is public and ungated. The pinned checkpoint and inference files can be downloaded without a Hugging Face account or access token.
 
1
  # Bagel-Robotics-1 inference adapter
2
 
3
+ Bagel-Robotics-1 combines task-critical data selection, adaptive structured pruning and recovery training to produce a compact world-action model. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
4
 
5
+ This folder is the standalone XPolicyLab adapter for checkpoint 1000. It contains inference code, configuration and evaluation instructions. Model tensors download separately from [`bageldotcom/Bagel-Robotics-1`](https://huggingface.co/bageldotcom/Bagel-Robotics-1), pinned to `2651631c5d08819088475e8778676f5cf6d9afcf` and verified against `policy/Bagel_Robotics_1/artifact-manifest.json`.
6
 
7
  Install this adapter into an existing XPolicyLab checkout (the installer refuses to overwrite an existing policy):
8
 
 
10
  python install_adapter.py /absolute/path/to/XPolicyLab
11
  ```
12
 
13
+ Then follow [`policy/Bagel_Robotics_1/README.md`](policy/Bagel_Robotics_1/README.md) for the Python 3.10 environment, exact checkpoint and encoder download, and debug checks. [`REEVALUATION.md`](policy/Bagel_Robotics_1/REEVALUATION.md) gives native simulator task, suite and three-seed commands. RoboDojo and its assets are installed separately.
14
 
15
+ The adapter is prepared against XPolicyLab `6baccd57ff958570d0db1588370289ee36c8bb70`; source and dependency pins are preserved in the policy folder. The training, pruning and export implementation is publicly available in the [Bagel-Robotics-1 repository](https://github.com/bageldotcom/Bagel-Robotics-1). The technical report remains planned for November 20, 2026.
16
 
17
  The repository is public and ungated. The pinned checkpoint and inference files can be downloaded without a Hugging Face account or access token.
inference/SHA256SUMS CHANGED
@@ -1,36 +1,52 @@
1
- 781418e0cf3334382c393e25358c3ac7d478bcd3709d57cb0fb15e72163260c6 README.md
2
  db9db816c8dd0f432068df166bbe3d9aa36ee4d47c5239b4862f73afa3843b06 install_adapter.py
3
  6c9437916a0dc5b6ed5fd56b361a4a1894be910e6965c9570e8683cc74a40813 policy/Bagel_Robotics_1/.gitignore
4
- 2095c3b590cfbf230698d30a491a49a24cfff856608d578ab2fdb9aa7749b674 policy/Bagel_Robotics_1/EVALUATION_RESULTS.md
5
- 8379dddfbbb1154ad9cc57f008f1f8d4a8d26c3de99bd6b36aa4d8067888aa92 policy/Bagel_Robotics_1/LICENSE
6
- c2160921c02d85d155347a6fec08442ebcb7b033c609e029a7fa31ae7901c5c5 policy/Bagel_Robotics_1/README.md
7
- 7e9cac33a9c6cf9c4854f44afcad473059f2962759b7c7e3b42ae44ed9cf22bc policy/Bagel_Robotics_1/REEVALUATION.md
8
- bf0d0498ea83fbc82593de12234892d4458099f36a9ff6d79c920add58640ab8 policy/Bagel_Robotics_1/THIRD_PARTY_NOTICES.md
9
- 4a4d71a75e893be9ddf802b4e3c853f2010f8ccc277e7b7a5ce2ae1e17614116 policy/Bagel_Robotics_1/VALIDATION.md
10
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/__init__.py
11
- e7667cec0939b219dc1396dcf34c8fadcd95f509e6606df9757b6c909967b5f2 policy/Bagel_Robotics_1/artifact-manifest.json
12
  76fec0d61c9344d8d2c5f61853a93269c8b19ae16d08de673f2ced5bdfe08b79 policy/Bagel_Robotics_1/assets.py
13
  2a16b03892ede7aa0a93cfd65e01a239498aa992bc4f1affb9cdde54253dae08 policy/Bagel_Robotics_1/backend.py
14
  68f337b21b1e517e1df27550e64fbc542b644528adc8f3f47fb8e44849185af6 policy/Bagel_Robotics_1/deploy.py
15
  6f25e548de5e0e10ed797ba65f59c7b64c986d340d7917d46d9aa92091c29b5f policy/Bagel_Robotics_1/deploy.yml
16
- e8ce9e58f22e4330b2f1cca1f45e304d9cd466941ec795ebcf46782d1299fe93 policy/Bagel_Robotics_1/download_checkpoint.py
17
  467e70c368f1574f6a39afdb3c84d6b10e8430bdbf638d9742a58585c7e60bc3 policy/Bagel_Robotics_1/download_checkpoint.sh
18
  daae036dc45b13f90f1b5b5c9e813262d62585aba467534e2402d4933e8dd97c policy/Bagel_Robotics_1/eval.sh
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  6110ad701ed4c7dd9768b61add6df64ace15c7109ccafdace4ea8ab8934b23cb policy/Bagel_Robotics_1/evidence/action-comparison.json
20
  2c8b3a55f75117533bf7ec0cfa8cfa7c42cd9b51d2517161fc022e8e15c5b9d3 policy/Bagel_Robotics_1/evidence/five-suite-results.json
21
  3187c8c75bb85c393b65f23838e8163fd201081b82b492601b2a6c7bb552d43a policy/Bagel_Robotics_1/evidence/gpu-validation.json
22
  f73d49a6b9c96824eddb76ae146b720a1d2feafba69c2ab7e9222c9167f3985f policy/Bagel_Robotics_1/evidence/local-results.json
23
- 86e4485d4f0ca34325151de212fed8974b5b09b2cf94d6c9041cda7d0476f957 policy/Bagel_Robotics_1/evidence/pr-audit.json
 
24
  d842f9639df62df7fddc84f095771b438f02f955605e221e4f7b160893592454 policy/Bagel_Robotics_1/install.sh
25
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/Apache-2.0.txt
26
  7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe policy/Bagel_Robotics_1/licenses/RoboDojo-MIT.txt
27
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/XPolicyLab.txt
28
  56418464b2a88e7aea31d436ee42851dfababad97db9178abdce318b5a7e621b policy/Bagel_Robotics_1/model.py
29
  2d1698608d28de6395f8c8794460b292f0233d2a4a5ef76a5d83152631ffe389 policy/Bagel_Robotics_1/requirements-transport.txt
30
- 8379dddfbbb1154ad9cc57f008f1f8d4a8d26c3de99bd6b36aa4d8067888aa92 policy/Bagel_Robotics_1/runtime/LICENSE
31
- 43973f510e70c55382293f19573eb81c42e965f5286f52650ca920d35ad8df77 policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md
32
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/runtime/licenses/Apache-2.0.txt
33
- 2258964574a8c567f3f6f871e54ef3664218bed4c40ee73eee4ed8df304fab0b policy/Bagel_Robotics_1/runtime/pyproject.toml
34
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/__init__.py
35
  70e67d196cb46263677ebe873c5837657d6b8f90ca40cb6845f4782f5377ba2d policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/checkpoint_compat.py
36
  8729d8e1386c74a0e3bd1c800a7c883725c9d00441f19bcbe7e40c2f8ff7af84 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/data.py
@@ -55,7 +71,7 @@ ce632db536b479c96a7160968eb314deb4e2dd1ab1b986e68f5dd8dea0ed6468 policy/Bagel_R
55
  5f42547a8571a2758ea6d4285834d5eaafc67285c6db42d44ec3c317e8402cc2 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/mot.py
56
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/__init__.py
57
  15de04574c1d8ccb794ca6560b1e501e5c83cc86104bb80f1eb3cc3b0324ffe0 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/scheduler_continuous.py
58
- 41e8632ad7898bf3a63cf7c874179adae609583ceb1b4dadf2b4c22669865323 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py
59
  d203d5ec0f6c913b052e22202eff0fe238e087040ae0de9843b7a22a5533b5f3 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan21.py
60
  f65e44c9d1b7811d2996b3fa22e6fe4c2ac5e6edf1fc2c14251cf582bf8d1f05 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_clip.py
61
  516b8dd09d576898c8760968ada4b617f5fe6a7c3fd945787898ae326df8e9a1 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_dit.py
@@ -68,7 +84,7 @@ dd3afea396e0ba3e619a1a66121f3c41dd0ca4174e1cf1f9f459b200f23e2690 policy/Bagel_R
68
  9b0ba19de9a64458b12a0fb24f0811f49d9e9fca5d8b815f5f5ecd3b7886fd63 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/utils/pytorch_utils.py
69
  113091a0c7fa38ecda3aae470bb6f151d0d034cfc187d4d752350f8cb9d71191 policy/Bagel_Robotics_1/setup_eval_env_client.sh
70
  428ada84e7b5d2c131118e7c8c47a65fb60bc34ce49db94b2e33608c354b93b9 policy/Bagel_Robotics_1/setup_eval_policy_server.sh
71
- 6597accf2e5ebc6ac46cdc9cfefc9a1622f24fecc9059516dab4fc3abd4530c6 policy/Bagel_Robotics_1/source-provenance.json
72
  42be4b09c56377f4686b2e722de760fa3bafd1c6b92605ada269d94e19efc95e policy/Bagel_Robotics_1/temporal_ensemble.py
73
  d5a7f0219ee4fc94c32add51121b83bf2138950a39de67cd42d9a76dddf391eb policy/Bagel_Robotics_1/tests/fixtures/checkpoint_processor.json
74
  add0937c0762cba1395426f8c46fc3b5b9bc2d6da63a51202b7b781a53732db9 policy/Bagel_Robotics_1/tests/test_backend.py
 
1
+ b54511ea4ae7a120abde0d9edf3acc4a67ab35d96c276dfb857f09410beac4fd README.md
2
  db9db816c8dd0f432068df166bbe3d9aa36ee4d47c5239b4862f73afa3843b06 install_adapter.py
3
  6c9437916a0dc5b6ed5fd56b361a4a1894be910e6965c9570e8683cc74a40813 policy/Bagel_Robotics_1/.gitignore
4
+ 0ccef2aa651ff17cc56c91985f81c235d92122bcda70fe9ab864091bf63ac59b policy/Bagel_Robotics_1/EVALUATION_RESULTS.md
5
+ ae9584319bcb20cfb02cf4784c34b098ed7ac4bba7c1d8ad8c9a22eda572a21d policy/Bagel_Robotics_1/LICENSE
6
+ 6724c8e65d6eb246ada70f28959a9da869ebb66af3e7b0418f6f44687e786ab9 policy/Bagel_Robotics_1/README.md
7
+ dbfe8357fa1ee02f1a4ca9922c6f0c845172ead932a1d306063103659f5cf1cd policy/Bagel_Robotics_1/REEVALUATION.md
8
+ 81452c82f6cf9648197d4fe66134f6e1c9e7c6524ae7f209602d7d39b25836dc policy/Bagel_Robotics_1/VALIDATION.md
 
9
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/__init__.py
10
+ 674af7f9f237559d7c73100f5dd0c6836a65aea4a2ee9c6defae3a6efa0ca94e policy/Bagel_Robotics_1/artifact-manifest.json
11
  76fec0d61c9344d8d2c5f61853a93269c8b19ae16d08de673f2ced5bdfe08b79 policy/Bagel_Robotics_1/assets.py
12
  2a16b03892ede7aa0a93cfd65e01a239498aa992bc4f1affb9cdde54253dae08 policy/Bagel_Robotics_1/backend.py
13
  68f337b21b1e517e1df27550e64fbc542b644528adc8f3f47fb8e44849185af6 policy/Bagel_Robotics_1/deploy.py
14
  6f25e548de5e0e10ed797ba65f59c7b64c986d340d7917d46d9aa92091c29b5f policy/Bagel_Robotics_1/deploy.yml
15
+ 410fea45c8c15349cbcafefd0dae3c9be6947b67b6eb495e9436ec79217bab85 policy/Bagel_Robotics_1/download_checkpoint.py
16
  467e70c368f1574f6a39afdb3c84d6b10e8430bdbf638d9742a58585c7e60bc3 policy/Bagel_Robotics_1/download_checkpoint.sh
17
  daae036dc45b13f90f1b5b5c9e813262d62585aba467534e2402d4933e8dd97c policy/Bagel_Robotics_1/eval.sh
18
+ 08fa69475293a505de2d74173827621fe27886be5723e38f1e0118e1529604c4 policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md
19
+ a2946a8c62a7cf3d1e64d8e8655d462533e2a3937b49756bbd458ab81586cad9 policy/Bagel_Robotics_1/evaluation/seed-1/summary.md
20
+ ec35c92be6c3bc2792da1fa2888620c9fb3bea79c37aa5ed7f26927a867d8f8b policy/Bagel_Robotics_1/evaluation/seed-1/verification.json
21
+ a79a06b50af6ef7df8f313c0cb84e954f9a328f4eac1c394216de0681ade251f policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes
22
+ c965c48fbaea7e1dd5852ff556d11c441a4897245a9b132fe55afbbdd1488314 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md
23
+ 6ed678932036e827103cd89de0c4748515c54b178eb7862c201eb2976d28af21 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv
24
+ 0b19a2cc51364baaacb070c5245eef274b5e9b4393b550ca30365fb4d0c603fb policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py
25
+ 7a2ac06a26b8496271a4e5c1d334ed9693d1b96e8058b99834e5e52eaa46d842 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md
26
+ b5284fdaf2d0eb3dd7dfc36e91d2842183d4695b2317c1338f145ed78694db57 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json
27
+ 5e710be09f95a3200c1c33755f667b9f95797b4ef5d696d3ee006d3dc6e09f92 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv
28
+ 2a7fb1a4ba1d8374b4f0317644f75ba360db17f43a0f22a3c5da50cf02da3544 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md
29
+ c0fe627bdbde293db26714658eac7695d2f6cb5d6e66842a3d29f93edcc70c66 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md
30
+ 147306c5805b47d1ecaa7179db7f9a15b1b682e492630e6ab685160b4c23ab63 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv
31
+ c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md
32
+ 141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md
33
+ ab9c45c7766c3760304aeb53c517dbc7ccebc97f970016d4a845ccf2df936141 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv
34
+ 958dbd046ff0d94562a5281f391ba3dfb7274f7f020a40bf191df87fae1d3044 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json
35
  6110ad701ed4c7dd9768b61add6df64ace15c7109ccafdace4ea8ab8934b23cb policy/Bagel_Robotics_1/evidence/action-comparison.json
36
  2c8b3a55f75117533bf7ec0cfa8cfa7c42cd9b51d2517161fc022e8e15c5b9d3 policy/Bagel_Robotics_1/evidence/five-suite-results.json
37
  3187c8c75bb85c393b65f23838e8163fd201081b82b492601b2a6c7bb552d43a policy/Bagel_Robotics_1/evidence/gpu-validation.json
38
  f73d49a6b9c96824eddb76ae146b720a1d2feafba69c2ab7e9222c9167f3985f policy/Bagel_Robotics_1/evidence/local-results.json
39
+ 7ad05fcefc49a0878e86969757f70c649201e0155ee07cd125eee228e69892f3 policy/Bagel_Robotics_1/evidence/pr-audit.json
40
+ e97227862bc82a11d9163bf6a928885949d5888c155c4a516d33c5115239e747 policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json
41
  d842f9639df62df7fddc84f095771b438f02f955605e221e4f7b160893592454 policy/Bagel_Robotics_1/install.sh
42
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/Apache-2.0.txt
43
  7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe policy/Bagel_Robotics_1/licenses/RoboDojo-MIT.txt
44
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/XPolicyLab.txt
45
  56418464b2a88e7aea31d436ee42851dfababad97db9178abdce318b5a7e621b policy/Bagel_Robotics_1/model.py
46
  2d1698608d28de6395f8c8794460b292f0233d2a4a5ef76a5d83152631ffe389 policy/Bagel_Robotics_1/requirements-transport.txt
47
+ 3c289cd7e240d09910bfdfa562a63647f09289430b55eb16efbf20ea5456a6c0 policy/Bagel_Robotics_1/runtime/LICENSE
 
48
  c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/runtime/licenses/Apache-2.0.txt
49
+ eb166587ec972e0df7f2385b759abb405d678359c1f97817998f189c902ca98a policy/Bagel_Robotics_1/runtime/pyproject.toml
50
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/__init__.py
51
  70e67d196cb46263677ebe873c5837657d6b8f90ca40cb6845f4782f5377ba2d policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/checkpoint_compat.py
52
  8729d8e1386c74a0e3bd1c800a7c883725c9d00441f19bcbe7e40c2f8ff7af84 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/data.py
 
71
  5f42547a8571a2758ea6d4285834d5eaafc67285c6db42d44ec3c317e8402cc2 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/mot.py
72
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/__init__.py
73
  15de04574c1d8ccb794ca6560b1e501e5c83cc86104bb80f1eb3cc3b0324ffe0 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/scheduler_continuous.py
74
+ a1df6528f7f9536b2ff4ad3913fa718d40dfb1061dc35e9fa0e28e2e1fa3bf96 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py
75
  d203d5ec0f6c913b052e22202eff0fe238e087040ae0de9843b7a22a5533b5f3 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan21.py
76
  f65e44c9d1b7811d2996b3fa22e6fe4c2ac5e6edf1fc2c14251cf582bf8d1f05 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_clip.py
77
  516b8dd09d576898c8760968ada4b617f5fe6a7c3fd945787898ae326df8e9a1 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_dit.py
 
84
  9b0ba19de9a64458b12a0fb24f0811f49d9e9fca5d8b815f5f5ecd3b7886fd63 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/utils/pytorch_utils.py
85
  113091a0c7fa38ecda3aae470bb6f151d0d034cfc187d4d752350f8cb9d71191 policy/Bagel_Robotics_1/setup_eval_env_client.sh
86
  428ada84e7b5d2c131118e7c8c47a65fb60bc34ce49db94b2e33608c354b93b9 policy/Bagel_Robotics_1/setup_eval_policy_server.sh
87
+ 0be927dbf173f09fd2177f60c02100fc280757d5edc8c5d7fd52dc3e21217487 policy/Bagel_Robotics_1/source-provenance.json
88
  42be4b09c56377f4686b2e722de760fa3bafd1c6b92605ada269d94e19efc95e policy/Bagel_Robotics_1/temporal_ensemble.py
89
  d5a7f0219ee4fc94c32add51121b83bf2138950a39de67cd42d9a76dddf391eb policy/Bagel_Robotics_1/tests/fixtures/checkpoint_processor.json
90
  add0937c0762cba1395426f8c46fc3b5b9bc2d6da63a51202b7b781a53732db9 policy/Bagel_Robotics_1/tests/test_backend.py
inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md CHANGED
@@ -1,103 +1,355 @@
1
- # Bagel-Robotics-1 results
2
-
3
- Bagel-Robotics-1 / AdaPrune Final, checkpoint 1000. Success means full task completion; partial scores average credited progress over all reported outcomes, including failures. Partial values here are out of 100. Generalization combines standard and randomized episodes; other suites use their recorded standard scene selections.
4
-
5
- | Suite | Successful / reported outcomes | Success rate | Mean partial /100 |
6
- |---|---:|---:|---:|
7
- | Precision | 126/400 | 31.50% | 40.43 |
8
- | Memory† | 173/300 | 57.67% | 58.00 |
9
- | Long Horizon‡ | 137/400 | 34.25% | 53.00 |
10
- | Generalization | 183/600 | 30.50% | 36.82 |
11
- | Open§ | 16/400 | 4.00% | 4.48 |
12
- | **Equal-weight five-suite average** | **635/2,100 total; not the averaging formula** | **31.58%** | **38.55** |
13
-
14
- ## Original scenes, repairs and replacements
15
-
16
- † Memory has 299 valid original results: 173 successes, **57.86% success and 58.19/100 partial**. Original sorting-sequence episode 47 remains unscored. Its separately repaired variant is a valid failure with score zero; the headline includes that variant, giving 300 reported outcomes. The repaired variant is not an unchanged-original result.
17
-
18
- ‡ Long Horizon includes 398 original results plus egg-holder replacements 50 and 51 for invalid original layouts 15 and 20. Both replacements failed the task, with partial scores 0 and 0.25. Original-only totals: 137/398 successes, 53.2035/100 partial.
19
-
20
- § Open includes 399 original results plus pickup replacement 50 for invalid original layout 5. That replacement failed with partial zero. Original-only totals: 16/399 successes, 4.4862/100 partial.
21
-
22
- Generalization combines standard and randomized episodes over all 12 tasks (600 results). The nine user-supplied completed tasks contribute 131/450 successes; the three node-2 tasks contribute 52/150. The original stack-block layout 6 and laptop/headphones layout 22 succeeded on retry; these are original scenes, not replacements. Cancellation of duplicate queue entries does not discard the separately supplied completed results.
23
-
24
- These totals contain **2,096 original results, three replacements and one repaired variant**. The overall score gives each suite equal weight, not each episode. Organizer acceptance of local substitutions or repairs is not assumed.
25
-
26
- ## Sources and scope
27
-
28
- Precision, Open and Long Horizon retain the original receipt hashes and replacement evidence in `evidence/local-results.json`. The completed five-suite calculation is in `evidence/five-suite-results.json`.
29
-
30
- Memory and nine Generalization tasks come from the user's supplied completed task summaries. Their raw episode logs were not independently re-audited during this packaging update. Stack blocks, store laptop/headphones and sort nesting dolls use node-2 receipts verified in the preceding results review, including successful original-layout retries. No new multi-seed estimate is implied.
31
-
32
- These results belong to the frozen research runner. Packaged-adapter debug, smoke and matched-action validation are reported separately in `VALIDATION.md`; research scores do not substitute for those tests. This is not an official leaderboard entry.
33
-
34
- ## Task-level breakdown
35
-
36
- ### Precision
37
-
38
- | Task | Successful / reported outcomes | Success rate | Mean partial /100 |
39
- |---|---:|---:|---:|
40
- | Build tower | 32/50 | 64.00% | 73.60 |
41
- | Deposit coin | 11/50 | 22.00% | 26.80 |
42
- | Fasten screws | 18/50 | 36.00% | 66.80 |
43
- | Insert key | 0/50 | 0.00% | 15.00 |
44
- | Insert tubes | 32/50 | 64.00% | 75.20 |
45
- | Play xylophone | 0/50 | 0.00% | 0.00 |
46
- | Plug in charger | 2/50 | 4.00% | 4.00 |
47
- | Pour balls into vase | 31/50 | 62.00% | 62.00 |
48
-
49
- ### Memory
50
-
51
- | Task | Successful / reported outcomes | Success rate | Mean partial /100 |
52
- |---|---:|---:|---:|
53
- | Cover blocks | 50/50 | 100.00% | 100.00 |
54
- | Imitate sorting sequence | 0/50 | 0.00% | 2.00 |
55
- | Match and pick from conveyor | 46/50 | 92.00% | 92.00 |
56
- | Press by number | 50/50 | 100.00% | 100.00 |
57
- | Swap T | 11/50 | 22.00% | 22.00 |
58
- | Swap blocks | 16/50 | 32.00% | 32.00 |
59
-
60
- ### Long Horizon
61
-
62
- | Task | Successful / reported outcomes | Success rate | Mean partial /100 |
63
- |---|---:|---:|---:|
64
- | Classify objects | 16/50 | 32.00% | 44.50 |
65
- | Fill egg holder | 2/50 | 4.00% | 13.80 |
66
- | Fill pen holder | 11/50 | 22.00% | 43.90 |
67
- | Make kong | 43/50 | 86.00% | 86.00 |
68
- | Organize table | 9/50 | 18.00% | 64.50 |
69
- | Play stacking toy | 6/50 | 12.00% | 32.00 |
70
- | Play tic tac toe | 1/50 | 2.00% | 40.50 |
71
- | Put bottles into dustbin | 49/50 | 98.00% | 98.80 |
72
-
73
- ### Generalization
74
-
75
- | Task | Successful / reported outcomes | Success rate | Mean partial /100 |
76
- |---|---:|---:|---:|
77
- | Arrange largest number | 2/50 | 4.00% | 11.00 |
78
- | Fold clothes | 36/50 | 72.00% | 74.40 |
79
- | Hang mugs | 14/50 | 28.00% | 38.50 |
80
- | Make toast | 22/50 | 44.00% | 52.50 |
81
- | Pack objects into box | 6/50 | 12.00% | 35.40 |
82
- | Pour liquid into cup | 22/50 | 44.00% | 44.00 |
83
- | Push T | 1/50 | 2.00% | 2.00 |
84
- | Sort nesting dolls by size | 10/50 | 20.00% | 20.00 |
85
- | Stack blocks | 22/50 | 44.00% | 50.00 |
86
- | Stack bowls | 26/50 | 52.00% | 58.90 |
87
- | Store laptop and headphones | 20/50 | 40.00% | 51.20 |
88
- | Sweep blocks | 2/50 | 4.00% | 4.00 |
89
-
90
- ### Open
91
-
92
- | Task | Successful / reported outcomes | Success rate | Mean partial /100 |
93
- |---|---:|---:|---:|
94
- | Align blocks | 0/50 | 0.00% | 0.00 |
95
- | General pickup | 12/50 | 24.00% | 24.00 |
96
- | Stack blocks by language | 4/50 | 8.00% | 9.20 |
97
- | Solve equation | 0/50 | 0.00% | 0.00 |
98
- | Classify objects by language | 0/50 | 0.00% | 2.60 |
99
- | Pick from conveyor by image | 0/50 | 0.00% | 0.00 |
100
- | Store tools in toolbox | 0/50 | 0.00% | 0.00 |
101
- | Pour by language | 0/50 | 0.00% | 0.00 |
102
-
103
- Checkpoint identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bagel-Robotics-1: three-seed local results
2
+
3
+ ## Overview
4
+
5
+ Checkpoint 1000 (AdaPrune Final), seeds **0, 1 and 2**: five suites, 42 tasks and **6,300 reported outcomes**. The selection includes 28 later replacement layouts, one repaired Memory episode and four earlier Seed-1 Precision substitutions; the complete appendix is at the end of this report.
6
+
7
+ | Suite | Successes / reported outcomes (3 seeds) | Success rate (%) | Partial score /100 |
8
+ | --- | ---: | ---: | ---: |
9
+ | Precision | 409/1200 | 34.08 ± 1.90 | 42.43 ± 1.42 |
10
+ | Memory | 509/900 | 56.56 ± 2.06 | 56.89 ± 2.17 |
11
+ | Generalization | 510/1800 | 28.33 ± 1.71 | 35.20 ± 1.35 |
12
+ | Long Horizon | 424/1200 | 35.33 ± 0.82 | 53.03 ± 0.28 |
13
+ | Open | 44/1200 | 3.67 ± 0.31 | 4.25 ± 0.25 |
14
+ | **Five-suite average** | **6,300 outcomes** | **31.59 ± 0.33** | **38.36 ± 0.46** |
15
+
16
+ **Reading the scores:** success means full task completion. Partial scores are out of 100 and include credited progress in both successful and failed episodes. All ± values in this report are population standard deviations across the three seeds. Each seed's overall score weights the five suites equally; it is not the total success count divided by 6,300.
17
+
18
+ **Checkpoint and inference:** [bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1/tree/2651631c5d08819088475e8778676f5cf6d9afcf), pinned revision `2651631c5d08819088475e8778676f5cf6d9afcf`; BF16, ten Euler steps, shift 1, 32 predicted / 16 executed actions and 90/10 temporal blending. These are retained local evaluation results; packaged-adapter checks are recorded in [VALIDATION.md](VALIDATION.md).
19
+
20
+ ## Individual seeds
21
+
22
+ | Seed | Suite | Scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
23
+ | --- | --- | ---: | ---: | ---: | ---: | ---: |
24
+ | 0 | Precision | 400/400 | 144 | 256 | 36.00 | 43.50 |
25
+ | 0 | Memory | 300/300 | 161 | 139 | 53.67 | 53.87 |
26
+ | 0 | Generalization | 600/600 | 158 | 442 | 26.33 | 33.52 |
27
+ | 0 | Long Horizon | 400/400 | 145 | 255 | 36.25 | 53.39 |
28
+ | 0 | Open | 400/400 | 15 | 385 | 3.75 | 4.36 |
29
+ | **0** | **Equal-weight five-suite average** | **2,100/2,100** | **623** | **1477** | **31.20** | **37.73** |
30
+ | 1 | Precision | 400/400 | 126 | 274 | 31.50 | 40.42 |
31
+ | 1 | Memory | 300/300 | 173 | 127 | 57.67 | 58.00 |
32
+ | 1 | Generalization | 600/600 | 183 | 417 | 30.50 | 36.83 |
33
+ | 1 | Long Horizon | 400/400 | 137 | 263 | 34.25 | 53.00 |
34
+ | 1 | Open | 400/400 | 16 | 384 | 4.00 | 4.47 |
35
+ | **1** | **Equal-weight five-suite average** | **2,100/2,100** | **635** | **1465** | **31.58** | **38.55** |
36
+ | 2 | Precision | 400/400 | 139 | 261 | 34.75 | 43.38 |
37
+ | 2 | Memory | 300/300 | 175 | 125 | 58.33 | 58.82 |
38
+ | 2 | Generalization | 600/600 | 169 | 431 | 28.17 | 35.27 |
39
+ | 2 | Long Horizon | 400/400 | 142 | 258 | 35.50 | 52.70 |
40
+ | 2 | Open | 400/400 | 13 | 387 | 3.25 | 3.90 |
41
+ | **2** | **Equal-weight five-suite average** | **2,100/2,100** | **638** | **1462** | **32.00** | **38.81** |
42
+
43
+ ## Task results
44
+
45
+ Each task has 50 reported outcomes per seed. Generalization combines 25 standard and 25 randomized episodes. Success columns show successes/evaluated episodes and percentage; failures equal evaluated episodes minus successes. Expand a suite to see its tasks.
46
+
47
+ <details>
48
+ <summary>Precision</summary>
49
+
50
+ | Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
51
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
52
+ | Build tower | 40/50 (80.00%) | 85.60 | 32/50 (64.00%) | 73.60 | 34/50 (68.00%) | 75.40 | 70.67 ± 6.80 | 78.20 ± 5.28 |
53
+ | Deposit coin | 14/50 (28.00%) | 32.40 | 11/50 (22.00%) | 26.80 | 12/50 (24.00%) | 28.40 | 24.67 ± 2.49 | 29.20 ± 2.36 |
54
+ | Fasten screws | 24/50 (48.00%) | 72.20 | 18/50 (36.00%) | 66.80 | 20/50 (40.00%) | 68.20 | 41.33 ± 4.99 | 69.07 ± 2.29 |
55
+ | Insert key | 0/50 (0.00%) | 15.00 | 0/50 (0.00%) | 15.00 | 0/50 (0.00%) | 15.00 | 0.00 ± 0.00 | 15.00 ± 0.00 |
56
+ | Insert tubes | 36/50 (72.00%) | 82.80 | 32/50 (64.00%) | 75.20 | 30/50 (60.00%) | 74.00 | 65.33 ± 4.99 | 77.33 ± 3.90 |
57
+ | Play xylophone | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
58
+ | Plug in charger | 6/50 (12.00%) | 12.00 | 2/50 (4.00%) | 4.00 | 4/50 (8.00%) | 8.00 | 8.00 ± 3.27 | 8.00 ± 3.27 |
59
+ | Pour balls into vase | 24/50 (48.00%) | 48.00 | 31/50 (62.00%) | 62.00 | 39/50 (78.00%) | 78.00 | 62.67 ± 12.26 | 62.67 ± 12.26 |
60
+
61
+ </details>
62
+
63
+
64
+ <details>
65
+ <summary>Memory</summary>
66
+
67
+ | Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
68
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
69
+ | Cover blocks | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 100.00 ± 0.00 | 100.00 ± 0.00 |
70
+ | Imitate sorting sequence | 1/50 (2.00%) | 3.20 | 0/50 (0.00%) | 2.00 | 0/50 (0.00%) | 2.90 | 0.67 ± 0.94 | 2.70 ± 0.51 |
71
+ | Match and pick from conveyor | 41/50 (82.00%) | 82.00 | 46/50 (92.00%) | 92.00 | 43/50 (86.00%) | 86.00 | 86.67 ± 4.11 | 86.67 ± 4.11 |
72
+ | Press by number | 49/50 (98.00%) | 98.00 | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 99.33 ± 0.94 | 99.33 ± 0.94 |
73
+ | Swap t | 11/50 (22.00%) | 22.00 | 11/50 (22.00%) | 22.00 | 16/50 (32.00%) | 32.00 | 25.33 ± 4.71 | 25.33 ± 4.71 |
74
+ | Swap blocks | 9/50 (18.00%) | 18.00 | 16/50 (32.00%) | 32.00 | 16/50 (32.00%) | 32.00 | 27.33 ± 6.60 | 27.33 ± 6.60 |
75
+
76
+ </details>
77
+
78
+
79
+ <details>
80
+ <summary>Generalization</summary>
81
+
82
+ | Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
83
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
84
+ | Arrange largest number | 3/50 (6.00%) | 10.60 | 2/50 (4.00%) | 11.00 | 0/50 (0.00%) | 7.60 | 3.33 ± 2.49 | 9.73 ± 1.52 |
85
+ | Fold clothes | 33/50 (66.00%) | 68.00 | 36/50 (72.00%) | 74.40 | 37/50 (74.00%) | 76.80 | 70.67 ± 3.40 | 73.07 ± 3.71 |
86
+ | Hang mugs | 15/50 (30.00%) | 41.40 | 14/50 (28.00%) | 38.50 | 14/50 (28.00%) | 40.60 | 28.67 ± 0.94 | 40.17 ± 1.22 |
87
+ | Make toast | 16/50 (32.00%) | 47.50 | 22/50 (44.00%) | 52.50 | 17/50 (34.00%) | 45.00 | 36.67 ± 5.25 | 48.33 ± 3.12 |
88
+ | Pack objects into box | 1/50 (2.00%) | 26.50 | 6/50 (12.00%) | 35.40 | 3/50 (6.00%) | 29.90 | 6.67 ± 4.11 | 30.60 ± 3.67 |
89
+ | Pour liquid into cup | 15/50 (30.00%) | 30.00 | 22/50 (44.00%) | 44.00 | 23/50 (46.00%) | 46.00 | 40.00 ± 7.12 | 40.00 ± 7.12 |
90
+ | Push t | 0/50 (0.00%) | 0.00 | 1/50 (2.00%) | 2.00 | 1/50 (2.00%) | 2.00 | 1.33 ± 0.94 | 1.33 ± 0.94 |
91
+ | Sort nesting dolls by size | 10/50 (20.00%) | 20.00 | 10/50 (20.00%) | 20.00 | 7/50 (14.00%) | 14.00 | 18.00 ± 2.83 | 18.00 ± 2.83 |
92
+ | Stack blocks | 18/50 (36.00%) | 43.50 | 22/50 (44.00%) | 50.00 | 23/50 (46.00%) | 52.00 | 42.00 ± 4.32 | 48.50 ± 3.63 |
93
+ | Stack bowls | 24/50 (48.00%) | 55.50 | 26/50 (52.00%) | 58.90 | 23/50 (46.00%) | 54.10 | 48.67 ± 2.49 | 56.17 ± 2.02 |
94
+ | Store laptop and headphones | 18/50 (36.00%) | 49.20 | 20/50 (40.00%) | 51.20 | 17/50 (34.00%) | 47.20 | 36.67 ± 2.49 | 49.20 ± 1.63 |
95
+ | Sweep blocks | 5/50 (10.00%) | 10.00 | 2/50 (4.00%) | 4.00 | 4/50 (8.00%) | 8.00 | 7.33 ± 2.49 | 7.33 ± 2.49 |
96
+
97
+ </details>
98
+
99
+
100
+ <details>
101
+ <summary>Long Horizon</summary>
102
+
103
+ | Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
104
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
105
+ | Classify objects | 23/50 (46.00%) | 56.70 | 16/50 (32.00%) | 44.50 | 21/50 (42.00%) | 51.10 | 40.00 ± 5.89 | 50.77 ± 4.99 |
106
+ | Fill egg holder | 1/50 (2.00%) | 11.00 | 2/50 (4.00%) | 13.80 | 0/50 (0.00%) | 12.20 | 2.00 ± 1.63 | 12.33 ± 1.15 |
107
+ | Fill pen holder | 18/50 (36.00%) | 53.90 | 11/50 (22.00%) | 43.90 | 15/50 (30.00%) | 52.40 | 29.33 ± 5.73 | 50.07 ± 4.40 |
108
+ | Make kong | 42/50 (84.00%) | 84.00 | 43/50 (86.00%) | 86.00 | 44/50 (88.00%) | 88.00 | 86.00 ± 1.63 | 86.00 ± 1.63 |
109
+ | Organize table | 7/50 (14.00%) | 57.50 | 9/50 (18.00%) | 64.50 | 6/50 (12.00%) | 58.00 | 14.67 ± 2.49 | 60.00 ± 3.19 |
110
+ | Play stacking toy | 7/50 (14.00%) | 31.80 | 6/50 (12.00%) | 32.00 | 4/50 (8.00%) | 27.20 | 11.33 ± 2.49 | 30.33 ± 2.22 |
111
+ | Play tic tac toe | 0/50 (0.00%) | 35.80 | 1/50 (2.00%) | 40.50 | 3/50 (6.00%) | 33.90 | 2.67 ± 2.49 | 36.73 ± 2.77 |
112
+ | Put bottles into dustbin | 47/50 (94.00%) | 96.40 | 49/50 (98.00%) | 98.80 | 49/50 (98.00%) | 98.80 | 96.67 ± 1.89 | 98.00 ± 1.13 |
113
+
114
+ </details>
115
+
116
+
117
+ <details>
118
+ <summary>Open</summary>
119
+
120
+ | Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
121
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
122
+ | Align blocks | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
123
+ | Classify objects by language | 0/50 (0.00%) | 3.60 | 0/50 (0.00%) | 2.60 | 3/50 (6.00%) | 9.20 | 2.00 ± 2.83 | 5.13 ± 2.90 |
124
+ | General pickup | 13/50 (26.00%) | 26.00 | 12/50 (24.00%) | 24.00 | 8/50 (16.00%) | 16.00 | 22.00 ± 4.32 | 22.00 ± 4.32 |
125
+ | Pick from conveyor by image | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
126
+ | Pour by language | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
127
+ | Solve equation | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
128
+ | Stack blocks by language | 2/50 (4.00%) | 4.80 | 4/50 (8.00%) | 9.20 | 2/50 (4.00%) | 6.00 | 5.33 ± 1.89 | 6.67 ± 1.86 |
129
+ | Store tools in toolbox | 0/50 (0.00%) | 0.50 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.17 ± 0.24 |
130
+
131
+ </details>
132
+
133
+ ## Generalization split
134
+
135
+ <details>
136
+ <summary>Standard versus randomized results</summary>
137
+
138
+ Each split has 300 episodes per seed. The following rows preserve the 25-episode task halves.
139
+
140
+ | Seed | Split | Successes / scored | Success rate (%) | Partial /100 |
141
+ | --- | --- | ---: | ---: | ---: |
142
+ | 0 | Standard | 134/300 | 44.67 | 51.13 |
143
+ | 0 | Randomized | 24/300 | 8.00 | 15.90 |
144
+ | 1 | Standard | 148/300 | 49.33 | 54.88 |
145
+ | 1 | Randomized | 35/300 | 11.67 | 18.77 |
146
+ | 2 | Standard | 129/300 | 43.00 | 49.30 |
147
+ | 2 | Randomized | 40/300 | 13.33 | 21.23 |
148
+
149
+ | Task / split | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 |
150
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
151
+ | Arrange largest number / standard | 3/25 (12.00%) | 16.60 | 1/25 (4.00%) | 13.00 | 0/25 (0.00%) | 8.60 |
152
+ | Arrange largest number / randomized | 0/25 (0.00%) | 4.60 | 1/25 (4.00%) | 9.00 | 0/25 (0.00%) | 6.60 |
153
+ | Fold clothes / standard | 17/25 (68.00%) | 70.40 | 20/25 (80.00%) | 80.80 | 16/25 (64.00%) | 68.00 |
154
+ | Fold clothes / randomized | 16/25 (64.00%) | 65.60 | 16/25 (64.00%) | 68.00 | 21/25 (84.00%) | 85.60 |
155
+ | Hang mugs / standard | 15/25 (60.00%) | 71.00 | 14/25 (56.00%) | 67.40 | 14/25 (56.00%) | 69.00 |
156
+ | Hang mugs / randomized | 0/25 (0.00%) | 11.80 | 0/25 (0.00%) | 9.60 | 0/25 (0.00%) | 12.20 |
157
+ | Make toast / standard | 14/25 (56.00%) | 71.00 | 18/25 (72.00%) | 79.00 | 15/25 (60.00%) | 67.00 |
158
+ | Make toast / randomized | 2/25 (8.00%) | 24.00 | 4/25 (16.00%) | 26.00 | 2/25 (8.00%) | 23.00 |
159
+ | Pack objects into box / standard | 0/25 (0.00%) | 29.40 | 4/25 (16.00%) | 42.60 | 2/25 (8.00%) | 34.60 |
160
+ | Pack objects into box / randomized | 1/25 (4.00%) | 23.60 | 2/25 (8.00%) | 28.20 | 1/25 (4.00%) | 25.20 |
161
+ | Pour liquid into cup / standard | 13/25 (52.00%) | 52.00 | 17/25 (68.00%) | 68.00 | 16/25 (64.00%) | 64.00 |
162
+ | Pour liquid into cup / randomized | 2/25 (8.00%) | 8.00 | 5/25 (20.00%) | 20.00 | 7/25 (28.00%) | 28.00 |
163
+ | Push t / standard | 0/25 (0.00%) | 0.00 | 1/25 (4.00%) | 4.00 | 1/25 (4.00%) | 4.00 |
164
+ | Push t / randomized | 0/25 (0.00%) | 0.00 | 0/25 (0.00%) | 0.00 | 0/25 (0.00%) | 0.00 |
165
+ | Sort nesting dolls by size / standard | 9/25 (36.00%) | 36.00 | 9/25 (36.00%) | 36.00 | 7/25 (28.00%) | 28.00 |
166
+ | Sort nesting dolls by size / randomized | 1/25 (4.00%) | 4.00 | 1/25 (4.00%) | 4.00 | 0/25 (0.00%) | 0.00 |
167
+ | Stack blocks / standard | 18/25 (72.00%) | 76.20 | 22/25 (88.00%) | 89.80 | 23/25 (92.00%) | 93.20 |
168
+ | Stack blocks / randomized | 0/25 (0.00%) | 10.80 | 0/25 (0.00%) | 10.20 | 0/25 (0.00%) | 10.80 |
169
+ | Stack bowls / standard | 24/25 (96.00%) | 96.60 | 23/25 (92.00%) | 93.20 | 21/25 (84.00%) | 86.40 |
170
+ | Stack bowls / randomized | 0/25 (0.00%) | 14.40 | 3/25 (12.00%) | 24.60 | 2/25 (8.00%) | 21.80 |
171
+ | Store laptop and headphones / standard | 17/25 (68.00%) | 78.40 | 17/25 (68.00%) | 76.80 | 12/25 (48.00%) | 60.80 |
172
+ | Store laptop and headphones / randomized | 1/25 (4.00%) | 20.00 | 3/25 (12.00%) | 25.60 | 5/25 (20.00%) | 33.60 |
173
+ | Sweep blocks / standard | 4/25 (16.00%) | 16.00 | 2/25 (8.00%) | 8.00 | 2/25 (8.00%) | 8.00 |
174
+ | Sweep blocks / randomized | 1/25 (4.00%) | 4.00 | 0/25 (0.00%) | 0.00 | 2/25 (8.00%) | 8.00 |
175
+
176
+
177
+ </details>
178
+
179
+ ## Reproduction files
180
+
181
+ <details>
182
+ <summary>Official summary, episode data and regeneration instructions</summary>
183
+
184
+ - [Download the unchanged RoboDojo summary](evaluation/seeds-0-1-2/summary.md).
185
+ - [Episode data and verification files](evaluation/seeds-0-1-2/): all 6,300 episode scores, replacement and retry ledgers, unscored originals, protocol records, checksums and the original-selection summary.
186
+
187
+ The native summary's overview uses the same standard deviation as this report; its task-table ± values instead use half the highest-to-lowest range. Its final task-weighted row is a different average: use the five-suite Overview for the headline. The earlier Notion report used sample standard deviation; its means are identical, but its ± values are larger. Neither describes a confidence interval. The native policy label identifies replacements and repair and represents this same checkpoint.
188
+
189
+ ### Evaluation and summary versions
190
+
191
+ - Evaluation lineage: frozen RoboDojo snapshot based on `08b7ee46034c3d0c8a389b2b7bfc138f4d55ee4d`, with local runtime/reproducibility patches. All 34 recorded protocol manifests match the retained baseline's 55 task-source hashes; 23 also record and match all 55 configuration hashes. Eleven cloud protocol manifests do not include those configuration hashes.
192
+ - Summary script revision: `f57e1633ffed59761bc8e568cfe7530c4b33381c`, verified byte-for-byte. Generating the summary at this revision does not change the benchmark version that produced the episodes.
193
+ - This compilation joins the retained cloud archive, earlier verified seed-1 package and completed node-2 records. The 750 overlapping seed-1 outcomes agree and are included once. Seed 2 Open comes from the completed retained cloud campaign.
194
+
195
+ Evaluation/model seeds 0/1/2 are recorded in each protocol. The native simulator reset uses the selected layout identity.
196
+
197
+ ### Regenerate the summary
198
+
199
+ Use a separate RoboDojo checkout at the pinned summary revision; do not change an active evaluation checkout. No GPU, model weights or simulator installation is needed to summarize these scores.
200
+
201
+ ```bash
202
+ git clone https://github.com/RoboDojo-Benchmark/RoboDojo.git /tmp/robodojo-summary-source
203
+ git -C /tmp/robodojo-summary-source checkout f57e1633ffed59761bc8e568cfe7530c4b33381c
204
+
205
+ # From policy/Bagel_Robotics_1:
206
+ cd evaluation/seeds-0-1-2
207
+ sha256sum -c SHA256SUMS
208
+ python3 generate_summary.py --robodojo-root /tmp/robodojo-summary-source \
209
+ --output-dir /tmp/bagel-robotics-1-reported-summary
210
+ cmp summary.md /tmp/bagel-robotics-1-reported-summary/summary.md
211
+
212
+ python3 generate_summary.py --robodojo-root /tmp/robodojo-summary-source \
213
+ --view original --output-dir /tmp/bagel-robotics-1-original-summary
214
+ cmp summary-original-selection.md /tmp/bagel-robotics-1-original-summary/summary.md
215
+ ```
216
+
217
+ The output directories must be new or empty. `generate_summary.py` verifies the two pinned script hashes, groups the published score fields into 162 native task/seed JSON files and invokes **`bash scripts/robodojo.sh summarize` unchanged**. Actual layout IDs and outcomes are preserved. These grouped JSONs are a score projection of the retained native records, not new simulator runs or byte-identical copies of the original one-episode files.
218
+
219
+ This score-only package omits rollout videos. The unmodified summarizer consequently reports missing videos in `summarize.log`; no placeholder videos are created. For model re-evaluation, use [REEVALUATION.md](REEVALUATION.md).
220
+
221
+ </details>
222
+
223
+ ## Replacement and repair appendix
224
+
225
+ All reported outcomes retain their native success Boolean and partial score. Replacement scenes use supplied unused layouts; each is counted once in place of its unscored original. Failed startup attempts are not task failures and are not included in scored denominators.
226
+
227
+ ### Outcome accounting
228
+
229
+ | Seed | Scored original selection | Later replacement layouts | Repaired variant | Reported outcomes |
230
+ | --- | ---: | ---: | ---: | ---: |
231
+ | 0 | 2084 | 16 | 0 | 2100 |
232
+ | 1 | 2096 | 3 | 1 | 2100 |
233
+ | 2 | 2091 | 9 | 0 | 2100 |
234
+ | **Total** | **6,271** | **28** | **1** | **6,300** |
235
+
236
+ ### Complete replacement and repair ledger
237
+
238
+ Partial scores below are out of 100. All listed originals remain unscored in the unchanged-original view.
239
+
240
+ | Seed | Task | Original layout | Reported layout | Type | Full success | Partial /100 |
241
+ | --- | --- | ---: | ---: | --- | --- | ---: |
242
+ | 0 | Fill egg holder | 28 | 50 | Supplied replacement | No | 40.00 |
243
+ | 0 | Fill egg holder | 35 | 51 | Supplied replacement | No | 0.00 |
244
+ | 0 | Fill egg holder | 37 | 52 | Supplied replacement | No | 0.00 |
245
+ | 0 | Imitate sorting sequence | 4 | 50 | Supplied replacement | No | 0.00 |
246
+ | 0 | Imitate sorting sequence | 8 | 51 | Supplied replacement | No | 5.00 |
247
+ | 0 | Imitate sorting sequence | 17 | 52 | Supplied replacement | No | 0.00 |
248
+ | 0 | Imitate sorting sequence | 29 | 53 | Supplied replacement | No | 0.00 |
249
+ | 0 | Imitate sorting sequence | 36 | 54 | Supplied replacement | No | 0.00 |
250
+ | 0 | Imitate sorting sequence | 48 | 55 | Supplied replacement | No | 0.00 |
251
+ | 0 | Imitate sorting sequence | 49 | 56 | Supplied replacement | No | 0.00 |
252
+ | 0 | General pickup | 29 | 50 | Supplied replacement | Yes | 100.00 |
253
+ | 0 | Deposit coin | 36 | 51 | Supplied replacement | No | 0.00 |
254
+ | 0 | Deposit coin | 37 | 52 | Supplied replacement | Yes | 100.00 |
255
+ | 0 | Deposit coin | 41 | 53 | Supplied replacement | No | 0.00 |
256
+ | 0 | Deposit coin | 43 | 54 | Supplied replacement | No | 0.00 |
257
+ | 0 | Deposit coin | 31 | 55 | Supplied replacement | No | 0.00 |
258
+ | 1 | Fill egg holder | 15 | 50 | Supplied replacement | No | 0.00 |
259
+ | 1 | Fill egg holder | 20 | 51 | Supplied replacement | No | 25.00 |
260
+ | 1 | Imitate sorting sequence | 47 | 47 | Repaired same layout | No | 0.00 |
261
+ | 1 | General pickup | 5 | 50 | Supplied replacement | No | 0.00 |
262
+ | 2 | Fill egg holder | 13 | 50 | Supplied replacement | No | 25.00 |
263
+ | 2 | Fill egg holder | 18 | 52 | Supplied replacement | No | 0.00 |
264
+ | 2 | Fill egg holder | 40 | 53 | Supplied replacement | No | 10.00 |
265
+ | 2 | Imitate sorting sequence | 16 | 50 | Supplied replacement | No | 0.00 |
266
+ | 2 | Imitate sorting sequence | 24 | 51 | Supplied replacement | No | 15.00 |
267
+ | 2 | Imitate sorting sequence | 32 | 52 | Supplied replacement | No | 0.00 |
268
+ | 2 | Imitate sorting sequence | 43 | 53 | Supplied replacement | No | 0.00 |
269
+ | 2 | Deposit coin | 24 | 50 | Supplied replacement | No | 0.00 |
270
+ | 2 | Deposit coin | 33 | 51 | Supplied replacement | No | 0.00 |
271
+
272
+
273
+ ### Earlier Seed 1 Precision selection
274
+
275
+ The frozen Deposit Coin selection already contained supplied layouts **50, 51, 52 and 53**, with original IDs **4, 8, 39 and 42** omitted. These are four pre-existing selected substitutions, separate from the 28 later recovery replacements above. No pairwise mapping between these two sets is inferred. “Original selection” means the frozen manifest used by that run, not necessarily layouts 0–49. Their scored outcomes are:
276
+
277
+ | Selected layout | Full success | Partial /100 |
278
+ | --- | --- | ---: |
279
+ | 50 | Yes | 100.00 |
280
+ | 51 | No | 0.00 |
281
+ | 52 | No | 20.00 |
282
+ | 53 | No | 0.00 |
283
+
284
+ ### Why replacements were used
285
+
286
+ - Deposit Coin, General Pickup and Fill Egg Holder originals failed native startup/stability validation before policy actions. Sorting Sequence originals failed the benchmark demonstration-validity check. Original attempts are retained in the source records.
287
+ - Seed 0 Deposit Coin reserve 50 failed all three startup attempts; reserve 55 was used for original layout 31. Reserve 50 contributes no score.
288
+ - Seed 2 Fill Egg Holder reserve 51 also failed startup and contributes no score. The scored replacements are 13→50, 18→52 and 40→53.
289
+
290
+ ### Memory Seed 1: repaired Sorting Sequence layout 47
291
+
292
+ The original supplied demonstration was invalid. A separately recorded repair changes the demonstration timing, adding **1,431 physics frames at 0.004 seconds/frame (5.724 seconds)**. The policy step budget remains 1,600. This repaired variant completed as a **valid failure, partial score 0**, after 629 policy steps. It is not an unchanged-original result.
293
+
294
+ - Original demonstration SHA-256: `170c63d412e6168d6d71dd58d5870f39d3caf85836ca0bb79d441742e7f6083e`.
295
+ - Repaired demonstration SHA-256: `526c9c572d1e7a635114da6e8cf436fd4ad12451927e25e8940da0cd143a1182`.
296
+ - Repair protocol: `8c8f9894f60ac3a905644871b9675d9b6c87ba4e3f52e7228951b49ac1623d75`.
297
+
298
+ ### Completed scores recovered after shutdown errors
299
+
300
+ | Seed | Task / layout | Retained native outcome | Recovery |
301
+ | --- | --- | --- | --- |
302
+ | 1 | Match and Pick from Conveyor / 30 | Success; partial 100/100; 388 actions | Native evaluation completed before simulator teardown exit −6. Completion was reconciled from retained evidence; no rerun or score change. |
303
+ | 2 | Insert Tubes / 25 | Failure; partial 40/100; 500 actions | Native evaluation completed before shutdown timeout. Completion was reconciled; the original error receipt and native score remain unchanged. |
304
+
305
+ ### Retries of original scenes
306
+
307
+ The `retried-originals.csv` ledger in the episode-data directory lists scored originals with more than one recorded attempt or a later attempt index: **80 for seed 0, 66 for seed 1 and 33 for seed 2**. They are counted once, with their original scene identity. Retained campaigns expose attempt history differently; this field is not a uniform count of extra evaluated episodes. Seed 2 Swap T layout 3, for example, completed on its second original attempt with failure and partial zero.
308
+
309
+ The original-selection-only tables below show the scores before the 28 later replacements and repaired variant were included. This appendix records the local selection; it does not imply committee approval of substitutions.
310
+
311
+ ### Original-selection-only comparisons
312
+
313
+ <details>
314
+ <summary>Suite scores and affected tasks before later replacements and repair</summary>
315
+
316
+ This view excludes the 28 later replacement layouts and the repaired Memory variant. It contains **6,271 scored original-selection outcomes**. Invalid original episodes remain unscored, not zero-valued failures. Seed 1 Precision retains its historical frozen selection, including the four earlier substitutions listed above.
317
+
318
+ These tables divide by the number of scored outcomes shown. The unchanged native `summary-original-selection.md` instead leaves incomplete task/seed cells blank and excludes them from its aggregates, so its coverage and averages differ.
319
+
320
+ | Seed | Suite | Scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
321
+ | --- | --- | ---: | ---: | ---: | ---: | ---: |
322
+ | 0 | Precision | 395/400 | 143 | 252 | 36.20 | 43.80 |
323
+ | 0 | Memory | 293/300 | 161 | 132 | 54.95 | 55.14 |
324
+ | 0 | Generalization | 600/600 | 158 | 442 | 26.33 | 33.52 |
325
+ | 0 | Long Horizon | 397/400 | 145 | 252 | 36.52 | 53.69 |
326
+ | 0 | Open | 399/400 | 14 | 385 | 3.51 | 4.12 |
327
+ | 1 | Precision | 400/400 | 126 | 274 | 31.50 | 40.42 |
328
+ | 1 | Memory | 299/300 | 173 | 126 | 57.86 | 58.19 |
329
+ | 1 | Generalization | 600/600 | 183 | 417 | 30.50 | 36.83 |
330
+ | 1 | Long Horizon | 398/400 | 137 | 261 | 34.42 | 53.20 |
331
+ | 1 | Open | 399/400 | 16 | 383 | 4.01 | 4.49 |
332
+ | 2 | Precision | 398/400 | 139 | 259 | 34.92 | 43.59 |
333
+ | 2 | Memory | 296/300 | 175 | 121 | 59.12 | 59.56 |
334
+ | 2 | Generalization | 600/600 | 169 | 431 | 28.17 | 35.27 |
335
+ | 2 | Long Horizon | 397/400 | 142 | 255 | 35.77 | 53.01 |
336
+ | 2 | Open | 400/400 | 13 | 387 | 3.25 | 3.90 |
337
+
338
+ #### Tasks affected by later replacements or repair
339
+
340
+ | Seed | Task | Original scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
341
+ | --- | --- | ---: | ---: | ---: | ---: | ---: |
342
+ | 0 | Deposit coin | 45/50 | 13 | 32 | 28.89 | 33.78 |
343
+ | 0 | Fill egg holder | 47/50 | 1 | 46 | 2.13 | 10.85 |
344
+ | 0 | General pickup | 49/50 | 12 | 37 | 24.49 | 24.49 |
345
+ | 0 | Imitate sorting sequence | 43/50 | 1 | 42 | 2.33 | 3.60 |
346
+ | 1 | Fill egg holder | 48/50 | 2 | 46 | 4.17 | 13.85 |
347
+ | 1 | General pickup | 49/50 | 12 | 37 | 24.49 | 24.49 |
348
+ | 1 | Imitate sorting sequence | 49/50 | 0 | 49 | 0.00 | 2.04 |
349
+ | 2 | Deposit coin | 48/50 | 12 | 36 | 25.00 | 29.58 |
350
+ | 2 | Fill egg holder | 47/50 | 0 | 47 | 0.00 | 12.23 |
351
+ | 2 | Imitate sorting sequence | 46/50 | 0 | 46 | 0.00 | 2.83 |
352
+
353
+ All other task rows match the reported task tables above. The `episode-scores.csv` ledger can be filtered with `record_type=original` to reproduce this view.
354
+
355
+ </details>
inference/policy/Bagel_Robotics_1/LICENSE CHANGED
@@ -22,3 +22,9 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24
  SOFTWARE.
 
 
 
 
 
 
 
22
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24
  SOFTWARE.
25
+
26
+ Component attribution
27
+ VPP2/FastWAM source retains the MIT copyright and terms above.
28
+ Wan2.1/DiffSynth-derived components retain Apache-2.0 terms; see licenses/Apache-2.0.txt.
29
+ The XPolicyLab interface retains its license in licenses/XPolicyLab.txt.
30
+ Model weights and datasets retain their respective upstream terms.
inference/policy/Bagel_Robotics_1/README.md CHANGED
@@ -1,8 +1,8 @@
1
  # Bagel-Robotics-1
2
 
3
- **Contributor:** Bagel Labs | **Paper:** Technical report in preparation | **arXiv:** Not yet available | **Original code:** [Bagel-Robotics-1 implementation](.)
4
 
5
- Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is the validation base; reused components in `runtime/` are credited in the [source notices](THIRD_PARTY_NOTICES.md).
6
 
7
  Shared conventions — argument meanings, checkpoint naming, split-machine deployment, `EVAL_ENV_TYPE` — are documented in the [XPolicyLab README](https://github.com/XPolicyLab/XPolicyLab/blob/main/README.md). Official results: [RoboDojo LeaderBoard](https://robodojo-benchmark.com/LeaderBoard).
8
 
@@ -26,7 +26,7 @@ PYTHON_BIN=python3.10 bash install.sh "$BAGEL_POLICY_ENV"
26
 
27
  Keep these variables in the same shell. The installer sets up the standalone runtime, XPolicyLab and PyTorch 2.11 / torchvision 0.26 CUDA 13.0 wheels; `TORCH_INDEX_URL` overrides the wheel index. Tested setup: Python 3.10.21, PyTorch 2.11.0+cu130, RTX PRO 6000 Blackwell (96 GB), 128 GiB host RAM and eight CPUs.
28
 
29
- The [checkpoint repository](https://huggingface.co/bageldotcom/Bagel-Robotics-1) is public and ungated. No login or access token is required. Download and verify all assets:
30
 
31
  ```bash
32
  PYTHON_BIN="$BAGEL_POLICY_ENV/bin/python" bash download_checkpoint.sh
@@ -36,11 +36,11 @@ Allow approximately **45.7 GB** for checkpoint and encoder assets, plus cache an
36
 
37
  ## Data Processing
38
 
39
- Inference receives decoded RGB observations from XPolicyLab. No conversion is required, so `process_data.sh` is omitted for this eval-only submission. Recovery training uses selected important task segments from the RoboDojo EE16 dataset rather than the complete dataset; the selection recipe will accompany the technical report.
40
 
41
  ## Training
42
 
43
- This submission is **eval-only**; `train.sh` is omitted. The technical report and training, pruning and export code are planned for **November 20, 2026**. Maintainer agreement on the eval-only submission is requested.
44
 
45
  ## Evaluation
46
 
@@ -110,4 +110,4 @@ The server launcher applies the recorded deterministic backend settings. Direct
110
 
111
  ## Notes
112
 
113
- Reported five-suite research results: **31.58% success / 38.55 mean partial out of 100**, equally weighted across suites. [EVALUATION_RESULTS.md](EVALUATION_RESULTS.md) contains suite/task tables, source details and the three replacement scenes plus one repaired Memory variant. Official three-seed evaluation is pending.
 
1
  # Bagel-Robotics-1
2
 
3
+ **Contributor:** Bagel Labs | **Paper:** Technical report in preparation | **arXiv:** Not yet available | **Original code:** [Bagel-Robotics-1 implementation](https://github.com/bageldotcom/Bagel-Robotics-1)
4
 
5
+ Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
6
 
7
  Shared conventions — argument meanings, checkpoint naming, split-machine deployment, `EVAL_ENV_TYPE` — are documented in the [XPolicyLab README](https://github.com/XPolicyLab/XPolicyLab/blob/main/README.md). Official results: [RoboDojo LeaderBoard](https://robodojo-benchmark.com/LeaderBoard).
8
 
 
26
 
27
  Keep these variables in the same shell. The installer sets up the standalone runtime, XPolicyLab and PyTorch 2.11 / torchvision 0.26 CUDA 13.0 wheels; `TORCH_INDEX_URL` overrides the wheel index. Tested setup: Python 3.10.21, PyTorch 2.11.0+cu130, RTX PRO 6000 Blackwell (96 GB), 128 GiB host RAM and eight CPUs.
28
 
29
+ The [checkpoint repository](https://huggingface.co/bageldotcom/Bagel-Robotics-1) is public and ungated. No Hugging Face account or access token is required. Download and verify all assets:
30
 
31
  ```bash
32
  PYTHON_BIN="$BAGEL_POLICY_ENV/bin/python" bash download_checkpoint.sh
 
36
 
37
  ## Data Processing
38
 
39
+ Inference receives decoded RGB observations from XPolicyLab. No conversion is required, so `process_data.sh` is omitted from this eval-only adapter. Training uses selected segments from the RoboDojo EE16 LeRobot v3.0 dataset. The public [data preparation guide](https://github.com/bageldotcom/Bagel-Robotics-1/blob/main/docs/DATA.md) provides the pinned dataset, conversion instructions and exact selection manifests.
40
 
41
  ## Training
42
 
43
+ This XPolicyLab adapter remains **eval-only**; `train.sh` is omitted. Dataset selection, mask learning, physical pruning and recovery code are publicly available in [Bagel-Robotics-1](https://github.com/bageldotcom/Bagel-Robotics-1). Follow the [training guide](https://github.com/bageldotcom/Bagel-Robotics-1/blob/main/docs/TRAINING.md) for configurations, required inputs and commands. The technical report remains planned for **November 20, 2026**. Maintainer agreement on the eval-only adapter is requested.
44
 
45
  ## Evaluation
46
 
 
110
 
111
  ## Notes
112
 
113
+ Three-seed local results (seeds **0, 1 and 2**): **31.59% success / 38.36 mean partial out of 100**, equally weighted across five suites. The report covers 42 tasks and 6,300 outcomes, including labeled replacements and one repaired Memory variant. [Detailed results](EVALUATION_RESULTS.md) contains all suite/task scores, the official summary and the replacement appendix.
inference/policy/Bagel_Robotics_1/REEVALUATION.md CHANGED
@@ -2,17 +2,19 @@
2
 
3
  Use the [README quickstart](README.md#installation) to install Python 3.10 policy
4
  dependencies, download and verify the pinned files, and pass both
5
- debug checks. The standalone inference runtime is included; the private research
6
- training repository is not needed. Install RoboDojo and its assets separately.
 
7
 
8
  The only policy checkpoint is `cp1000`, from
9
  [bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1),
10
  revision `2651631c5d08819088475e8778676f5cf6d9afcf`. The downloader and loader reject
11
  files that do not match `artifact-manifest.json`. The VAE, image/text encoders and
12
  tokenizer are downloaded automatically at their separately pinned public revision.
13
- The checkpoint is public and ungated. Download it without a Hugging Face account
14
- or access token using the README commands. The required encoder assets are also
15
- public.
 
16
 
17
  ## What a reviewer can reproduce
18
 
@@ -23,8 +25,8 @@ two recorded observation sequences. The full five-suite research score table has
23
  not been rerun end to end through this packaged adapter.
24
 
25
  The commands below start a fresh official evaluation using supplied original
26
- layouts. They do not recreate the research run's three replacement scenes or one
27
- repaired Memory variant. Compare episode selection and seed coverage before
28
  comparing score totals; retain startup failures for committee review. The public
29
  comparison receipts describe the tested scope, but the original research runner
30
  and raw comparison fixtures are not included in this PR. They are not required to
@@ -129,8 +131,8 @@ episodes or combined with the two-episode smoke results.
129
  The default reevaluation uses original supplied layouts. If a scene fails before
130
  any robot action, preserve its startup error and retry history; do not silently
131
  replace the scene or score an infrastructure failure as a policy failure. Seek
132
- the committee's handling decision. The historical research run used three
133
- labeled replacements and one repaired Memory variant, described in
134
  [EVALUATION_RESULTS.md](EVALUATION_RESULTS.md); those exceptions are not activated
135
  by these commands. Official verification and acceptance remain with RoboDojo.
136
 
@@ -138,7 +140,7 @@ by these commands. Official verification and acceptance remain with RoboDojo.
138
 
139
  | Symptom | Check |
140
  |---|---|
141
- | HTTP 401/403 or repository not found | The checkpoint is public and ungated. Check the canonical repository and pinned revision. No access token is required; if a stale credential is being sent, retry anonymously. |
142
  | Hash/size mismatch | Check the pinned revision and local file completeness; do not disable verification. |
143
  | Missing encoder or tokenizer | Run the full downloader; `--metadata-only` does not fetch tensors or encoders. |
144
  | Missing `env_cfg` or simulator | Complete the official workspace setup; `env_cfg/` belongs beside XPolicyLab. |
 
2
 
3
  Use the [README quickstart](README.md#installation) to install Python 3.10 policy
4
  dependencies, download and verify the pinned files, and pass both
5
+ debug checks. The standalone inference runtime is included. The public
6
+ [training repository](https://github.com/bageldotcom/Bagel-Robotics-1) is optional for
7
+ model reevaluation. Install RoboDojo and its assets separately.
8
 
9
  The only policy checkpoint is `cp1000`, from
10
  [bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1),
11
  revision `2651631c5d08819088475e8778676f5cf6d9afcf`. The downloader and loader reject
12
  files that do not match `artifact-manifest.json`. The VAE, image/text encoders and
13
  tokenizer are downloaded automatically at their separately pinned public revision.
14
+ The checkpoint and required encoder assets are public and ungated. No Hugging
15
+ Face account, access token or evaluator access approval is required. Run the
16
+ downloader directly as shown in the README; the pinned revisions and checksum
17
+ checks are unchanged.
18
 
19
  ## What a reviewer can reproduce
20
 
 
25
  not been rerun end to end through this packaged adapter.
26
 
27
  The commands below start a fresh official evaluation using supplied original
28
+ layouts. They do not recreate the research run's historical scene substitutions or repaired
29
+ Memory variant. Compare episode selection and seed coverage before
30
  comparing score totals; retain startup failures for committee review. The public
31
  comparison receipts describe the tested scope, but the original research runner
32
  and raw comparison fixtures are not included in this PR. They are not required to
 
131
  The default reevaluation uses original supplied layouts. If a scene fails before
132
  any robot action, preserve its startup error and retry history; do not silently
133
  replace the scene or score an infrastructure failure as a policy failure. Seek
134
+ the committee's handling decision. The historical research run includes 28 later replacement layouts, one repaired
135
+ Memory variant and four earlier Seed-1 Precision substitutions, described in
136
  [EVALUATION_RESULTS.md](EVALUATION_RESULTS.md); those exceptions are not activated
137
  by these commands. Official verification and acceptance remain with RoboDojo.
138
 
 
140
 
141
  | Symptom | Check |
142
  |---|---|
143
+ | HTTP 401/403 or repository not found | Confirm the canonical repository and pinned revision above. Both repositories are public; no token is required. Retry with `HF_HUB_DISABLE_IMPLICIT_TOKEN=1` to exclude stale cached credentials, and check network/proxy access if the error persists. |
144
  | Hash/size mismatch | Check the pinned revision and local file completeness; do not disable verification. |
145
  | Missing encoder or tokenizer | Run the full downloader; `--metadata-only` does not fetch tensors or encoders. |
146
  | Missing `env_cfg` or simulator | Complete the official workspace setup; `env_cfg/` belongs beside XPolicyLab. |
inference/policy/Bagel_Robotics_1/THIRD_PARTY_NOTICES.md DELETED
@@ -1,13 +0,0 @@
1
- # Source and artifact notices
2
-
3
- - Reused source components from [VPP2](https://github.com/roboterax/video-prediction-policy-2)/FastWAM retain the MIT notice in
4
- `LICENSE`, copyright 2026 The FastWAM Authors. See `source-provenance.json`
5
- for the source identities and packaging changes.
6
- - Wan2.1/DiffSynth-derived model components retain Apache-2.0 terms; see
7
- `licenses/Apache-2.0.txt` and upstream Wan2.1/DiffSynth source notices.
8
- - The XPolicyLab interface follows its upstream license, included in
9
- `licenses/XPolicyLab.txt`. RoboDojo and simulator assets are not bundled.
10
- - Encoder weights, tokenizer files and model weights have their own upstream
11
- terms. The code license does not assign a new license to those artifacts.
12
- - The existing preview PR by Soroush Mehraban was consulted as a reference; its
13
- alternate checkpoint, code and release promises are not this submission.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
inference/policy/Bagel_Robotics_1/VALIDATION.md CHANGED
@@ -1,6 +1,6 @@
1
  # Adapter validation
2
 
3
- Status: **Technical qualification passed: 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes, and all eight exact replay/backend checks. [PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234) is open. The dedicated private read credential is verified; delivery to an assigned evaluator remains a separate handoff. The technical report and training/pruning/export code are planned for November 20, 2026. Full official benchmark verification remains pending.**
4
 
5
  | Check | Evidence and scope |
6
  |---|---|
@@ -37,11 +37,21 @@ audit. CPU checks used Python 3.12.7 / PyTorch 2.11.0+cpu with existing test
37
  dependencies; they do not replace the earlier fresh Python 3.10.21 / CUDA 13.0
38
  installation and GPU qualification. Inference code and settings are unchanged.
39
 
40
- The reviewer guide now explicitly fetches the PR source and explains private
41
- authentication. A complete fresh-machine simulator/assets installation, different
42
  GPU types, and a full five-suite packaged-adapter run have not been qualified by
43
  this audit.
44
 
 
 
 
 
 
 
 
 
 
 
45
  ## Reproduce the transport checks
46
 
47
  Use the policy README's debug command with the actual checkpoint. Repeat it with `DEBUG_OBS_ENCODED=1`. Both runs must finish normally and print the evaluation-finished marker.
 
1
  # Adapter validation
2
 
3
+ Status: **Technical qualification passed: 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes, and all eight exact replay/backend checks. [PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234) is open. The checkpoint is public and ungated; no evaluator credential handoff is required. The technical report remains planned for November 20, 2026. Full official benchmark verification remains pending.**
4
 
5
  | Check | Evidence and scope |
6
  |---|---|
 
37
  dependencies; they do not replace the earlier fresh Python 3.10.21 / CUDA 13.0
38
  installation and GPU qualification. Inference code and settings are unchanged.
39
 
40
+ That audit used authenticated access; the checkpoint is now public, as verified
41
+ below. A complete fresh-machine simulator/assets installation, different
42
  GPU types, and a full five-suite packaged-adapter run have not been qualified by
43
  this audit.
44
 
45
+ ## Public checkpoint access (November 7, 2026)
46
+
47
+ The checkpoint and all required encoder/tokenizer assets are publicly accessible
48
+ without authentication. The published metadata downloader passed in an isolated
49
+ empty cache with the submission-pinned `huggingface-hub==0.36.0` and no token.
50
+ All four checkpoint JSON files matched their SHA-256 checksums; all 15 asset URLs
51
+ passed anonymous ranged downloads and matched pinned remote identities and sizes.
52
+ The checkpoint revision, hashes and inference settings are unchanged. Evidence:
53
+ [public-checkpoint-access.json](evidence/public-checkpoint-access.json).
54
+
55
  ## Reproduce the transport checks
56
 
57
  Use the policy README's debug command with the actual checkpoint. Repeat it with `DEBUG_OBS_ENCODED=1`. Both runs must finish normally and print the evaluation-finished marker.
inference/policy/Bagel_Robotics_1/artifact-manifest.json CHANGED
@@ -6,7 +6,7 @@
6
  "video_parameters": 12468409664,
7
  "repo_id": "bageldotcom/Bagel-Robotics-1",
8
  "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
9
- "private": true,
10
  "files": {
11
  "COMPLETE.json": {
12
  "sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
 
6
  "video_parameters": 12468409664,
7
  "repo_id": "bageldotcom/Bagel-Robotics-1",
8
  "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
9
+ "private": false,
10
  "files": {
11
  "COMPLETE.json": {
12
  "sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
inference/policy/Bagel_Robotics_1/download_checkpoint.py CHANGED
@@ -1,4 +1,4 @@
1
- """Download the pinned existing private CP1000 artifact and public encoder assets."""
2
  from pathlib import Path
3
  import argparse
4
  import json
 
1
+ """Download the pinned public CP1000 artifact and encoder assets; no token required."""
2
  from pathlib import Path
3
  import argparse
4
  import json
inference/policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Bagel-Robotics-1 — complete reported results
2
+
3
+ Checkpoint 1000, **evaluation seed 1**. This companion report contains the previously reported five-suite totals and all 42 task results. The tables below are a readable companion to the unchanged [native summary](summary.md); all 42 tasks have been verified against retained per-episode native records. Success means full task completion; partial scores measure credited progress, including in failed episodes.
4
+
5
+ All 2,100 reported outcomes were checked against retained native result files and completion records: 1,350 from Node 2, plus 450 Generalization outcomes and 300 reported Memory outcomes from the retained Nebius archive. The Memory archive contains 299 valid originals and a separately labeled repaired variant.
6
+
7
+ | Suite | Successes / reported outcomes | Success rate | Mean partial /100 |
8
+ |---|---:|---:|---:|
9
+ | Precision | 126/400 | 31.50% | 40.43 |
10
+ | Memory | 173/300 | 57.67% | 58.00 |
11
+ | Long Horizon | 137/400 | 34.25% | 53.00 |
12
+ | Generalization | 183/600 | 30.50% | 36.82 |
13
+ | Open | 16/400 | 4.00% | 4.48 |
14
+ | **Equal-weight five-suite average** | **635/2,100 total** | **31.58%** | **38.55** |
15
+
16
+ The overall score is the mean of the five suite scores, not successes divided by all episodes.
17
+
18
+ ## Scene selection and sources
19
+
20
+ - Memory: 299 original valid results plus the separately repaired sorting-sequence layout 47 (failure, partial 0). Original-only totals: 173/299 successes (57.86%), partial 58.19/100. The original layout remains unscored.
21
+ - Long Horizon: 398 originals plus supplied egg-holder layouts 50 and 51, replacing unscored originals 15 and 20. The replacements scored 0 and 0.25, both failures.
22
+ - Open: 399 originals plus supplied pickup layout 50, replacing unscored original 5. The replacement failed with partial 0.
23
+ - Generalization: 25 standard plus 25 randomized episodes per task. All 12 tasks were checked against native records: three from Node 2 and nine from the retained Nebius archive. The combined standard/random total is 600 outcomes.
24
+
25
+ “Original” refers to the frozen task selections. Deposit Coin uses supplied layouts 50–53 instead of 4, 8, 39 and 42; Imitate Sorting Sequence uses supplied layouts 50–53 instead of 18, 33, 34 and 44. These selections were fixed before the saved runs.
26
+
27
+ ## Task-level results
28
+
29
+ ### Precision
30
+
31
+ | Task | Successes / reported outcomes | Success rate | Mean partial /100 |
32
+ |---|---:|---:|---:|
33
+ | Build tower | 32/50 | 64.00% | 73.60 |
34
+ | Deposit coin | 11/50 | 22.00% | 26.80 |
35
+ | Fasten screws | 18/50 | 36.00% | 66.80 |
36
+ | Insert key | 0/50 | 0.00% | 15.00 |
37
+ | Insert tubes | 32/50 | 64.00% | 75.20 |
38
+ | Play xylophone | 0/50 | 0.00% | 0.00 |
39
+ | Plug in charger | 2/50 | 4.00% | 4.00 |
40
+ | Pour balls into vase | 31/50 | 62.00% | 62.00 |
41
+
42
+ ### Memory
43
+
44
+ | Task | Successes / reported outcomes | Success rate | Mean partial /100 |
45
+ |---|---:|---:|---:|
46
+ | Cover blocks | 50/50 | 100.00% | 100.00 |
47
+ | Imitate sorting sequence | 0/50 | 0.00% | 2.00 |
48
+ | Match and pick from conveyor | 46/50 | 92.00% | 92.00 |
49
+ | Press by number | 50/50 | 100.00% | 100.00 |
50
+ | Swap T | 11/50 | 22.00% | 22.00 |
51
+ | Swap blocks | 16/50 | 32.00% | 32.00 |
52
+
53
+ ### Long Horizon
54
+
55
+ | Task | Successes / reported outcomes | Success rate | Mean partial /100 |
56
+ |---|---:|---:|---:|
57
+ | Classify objects | 16/50 | 32.00% | 44.50 |
58
+ | Fill egg holder | 2/50 | 4.00% | 13.80 |
59
+ | Fill pen holder | 11/50 | 22.00% | 43.90 |
60
+ | Make kong | 43/50 | 86.00% | 86.00 |
61
+ | Organize table | 9/50 | 18.00% | 64.50 |
62
+ | Play stacking toy | 6/50 | 12.00% | 32.00 |
63
+ | Play tic tac toe | 1/50 | 2.00% | 40.50 |
64
+ | Put bottles into dustbin | 49/50 | 98.00% | 98.80 |
65
+
66
+ ### Generalization
67
+
68
+ | Task | Successes / reported outcomes | Success rate | Mean partial /100 |
69
+ |---|---:|---:|---:|
70
+ | Arrange largest number | 2/50 | 4.00% | 11.00 |
71
+ | Fold clothes | 36/50 | 72.00% | 74.40 |
72
+ | Hang mugs | 14/50 | 28.00% | 38.50 |
73
+ | Make toast | 22/50 | 44.00% | 52.50 |
74
+ | Pack objects into box | 6/50 | 12.00% | 35.40 |
75
+ | Pour liquid into cup | 22/50 | 44.00% | 44.00 |
76
+ | Push T | 1/50 | 2.00% | 2.00 |
77
+ | Sort nesting dolls by size | 10/50 | 20.00% | 20.00 |
78
+ | Stack blocks | 22/50 | 44.00% | 50.00 |
79
+ | Stack bowls | 26/50 | 52.00% | 58.90 |
80
+ | Store laptop and headphones | 20/50 | 40.00% | 51.20 |
81
+ | Sweep blocks | 2/50 | 4.00% | 4.00 |
82
+
83
+ ### Open
84
+
85
+ | Task | Successes / reported outcomes | Success rate | Mean partial /100 |
86
+ |---|---:|---:|---:|
87
+ | Align blocks | 0/50 | 0.00% | 0.00 |
88
+ | Classify objects by language | 0/50 | 0.00% | 2.60 |
89
+ | General pickup | 12/50 | 24.00% | 24.00 |
90
+ | Pick from conveyor by image | 0/50 | 0.00% | 0.00 |
91
+ | Pour by language | 0/50 | 0.00% | 0.00 |
92
+ | Solve equation | 0/50 | 0.00% | 0.00 |
93
+ | Stack blocks by language | 4/50 | 8.00% | 9.20 |
94
+ | Store tools in toolbox | 0/50 | 0.00% | 0.00 |
95
+
96
+ Checkpoint identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
97
+
98
+ ## Reading the native summary
99
+
100
+ [summary.md](summary.md) is the unchanged output of `bash scripts/robodojo.sh summarize` at [RoboDojo revision f57e163](https://github.com/RoboDojo-Benchmark/RoboDojo/tree/f57e1633ffed59761bc8e568cfe7530c4b33381c), generated from the retained native records. [verification.json](verification.json) records its SHA-256 and the completed checks.
101
+
102
+ All 42 tasks are reported for seed 1. The native summary shows 42/126 (33.3%) because its denominator includes seeds 0, 1 and 2; the other seed columns are blank. Its `±0.00` reflects one available seed, not measured variation across seeds. The native script's rounding can differ by 0.01 from the readable tables above; the underlying episode scores agree. The summary's `with_labeled_replacements_and_repair` suffix identifies the reported scene selection; the model name is Bagel-Robotics-1.
inference/policy/Bagel_Robotics_1/evaluation/seed-1/summary.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RoboDojo Evaluation Summary
2
+
3
+ ## Overview (by capability dimension)
4
+
5
+ Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
6
+
7
+ | Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
8
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
9
+ | Bagel-Robotics-1_with_labeled_replacements_and_repair | 33.3% † (42/126) | 30.50 ±0.00 | 36.82 ±0.00 | 31.50 ±0.00 | 40.42 ±0.00 | 34.25 ±0.00 | 53.00 ±0.00 | 57.67 ±0.00 | 58.00 ±0.00 | 4.00 ±0.00 | 4.47 ±0.00 | 31.58 ±0.00 | 38.55 ±0.00 |
10
+
11
+ ## Generalization: Standard vs Random (25 episodes each)
12
+
13
+ Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
14
+
15
+ | Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
16
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
17
+ | Bagel-Robotics-1_with_labeled_replacements_and_repair | 49.33 ±0.00 | 54.88 ±0.00 | 11.67 ±0.00 | 18.77 ±0.00 | -37.67 | -36.12 |
18
+
19
+ ### Per-task Standard vs Random
20
+
21
+ Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
22
+
23
+ #### Bagel-Robotics-1_with_labeled_replacements_and_repair
24
+
25
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
26
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
27
+ | <nobr>stack_bowls</nobr> | | | 92 | 93.20 | | | | |
28
+ | <nobr>stack_bowls_random</nobr> | | | 12 | 24.60 | | | | |
29
+ | <nobr>push_T</nobr> | | | 4 | 4.00 | | | | |
30
+ | <nobr>push_T_random</nobr> | | | 0 | 0.00 | | | | |
31
+ | <nobr>pack_objects_into_box</nobr> | | | 16 | 42.60 | | | | |
32
+ | <nobr>pack_objects_into_box_random</nobr> | | | 8 | 28.20 | | | | |
33
+ | <nobr>fold_clothes</nobr> | | | 80 | 80.80 | | | | |
34
+ | <nobr>fold_clothes_random</nobr> | | | 64 | 68.00 | | | | |
35
+ | <nobr>hang_mugs</nobr> | | | 56 | 67.40 | | | | |
36
+ | <nobr>hang_mugs_random</nobr> | | | 0 | 9.60 | | | | |
37
+ | <nobr>sweep_blocks</nobr> | | | 8 | 8.00 | | | | |
38
+ | <nobr>sweep_blocks_random</nobr> | | | 0 | 0.00 | | | | |
39
+ | <nobr>pour_liquid_into_cup</nobr> | | | 68 | 68.00 | | | | |
40
+ | <nobr>pour_liquid_into_cup_random</nobr> | | | 20 | 20.00 | | | | |
41
+ | <nobr>make_toast</nobr> | | | 72 | 79.00 | | | | |
42
+ | <nobr>make_toast_random</nobr> | | | 16 | 26.00 | | | | |
43
+ | <nobr>arrange_largest_number</nobr> | | | 4 | 13.00 | | | | |
44
+ | <nobr>arrange_largest_number_random</nobr> | | | 4 | 9.00 | | | | |
45
+ | <nobr>sort_nesting_dolls_by_size</nobr> | | | 36 | 36.00 | | | | |
46
+ | <nobr>sort_nesting_dolls_by_size_random</nobr> | | | 4 | 4.00 | | | | |
47
+ | <nobr>store_laptop_and_headphones</nobr> | | | 68 | 76.80 | | | | |
48
+ | <nobr>store_laptop_and_headphones_random</nobr> | | | 12 | 25.60 | | | | |
49
+ | <nobr>stack_blocks</nobr> | | | 88 | 89.80 | | | | |
50
+ | <nobr>stack_blocks_random</nobr> | | | 0 | 10.20 | | | | |
51
+ | **Avg Standard** | | | 49.3 | 54.88 | | | 49 ±0 | 54.88 ±0.00 |
52
+ | **Avg Random** | | | 11.7 | 18.77 | | | 12 ±0 | 18.77 ±0.00 |
53
+
54
+ ## Bagel-Robotics-1_with_labeled_replacements_and_repair
55
+
56
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
57
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
58
+ | <nobr>align_blocks</nobr> | | | 0 | 0.00 | | | | |
59
+ | <nobr>arrange_largest_number</nobr> | | | 4 | 11.00 | | | | |
60
+ | <nobr>build_tower</nobr> | | | 64 | 73.60 | | | | |
61
+ | <nobr>classify_objects</nobr> | | | 32 | 44.50 | | | | |
62
+ | <nobr>classify_objects_by_language</nobr> | | | 0 | 2.60 | | | | |
63
+ | <nobr>cover_blocks</nobr> | | | 100 | 100.00 | | | | |
64
+ | <nobr>deposit_coin</nobr> | | | 22 | 26.80 | | | | |
65
+ | <nobr>fasten_screws</nobr> | | | 36 | 66.80 | | | | |
66
+ | <nobr>fill_egg_holder</nobr> | | | 4 | 13.80 | | | | |
67
+ | <nobr>fill_pen_holder</nobr> | | | 22 | 43.90 | | | | |
68
+ | <nobr>fold_clothes</nobr> | | | 72 | 74.40 | | | | |
69
+ | <nobr>general_pickup</nobr> | | | 24 | 24.00 | | | | |
70
+ | <nobr>hang_mugs</nobr> | | | 28 | 38.50 | | | | |
71
+ | <nobr>imitate_sorting_sequence</nobr> | | | 0 | 2.00 | | | | |
72
+ | <nobr>insert_key</nobr> | | | 0 | 15.00 | | | | |
73
+ | <nobr>insert_tubes</nobr> | | | 64 | 75.20 | | | | |
74
+ | <nobr>make_kong</nobr> | | | 86 | 86.00 | | | | |
75
+ | <nobr>make_toast</nobr> | | | 44 | 52.50 | | | | |
76
+ | <nobr>match_and_pick_from_conveyor</nobr> | | | 92 | 92.00 | | | | |
77
+ | <nobr>organize_table</nobr> | | | 18 | 64.50 | | | | |
78
+ | <nobr>pack_objects_into_box</nobr> | | | 12 | 35.40 | | | | |
79
+ | <nobr>pick_from_conveyor_by_image</nobr> | | | 0 | 0.00 | | | | |
80
+ | <nobr>play_Xylophone</nobr> | | | 0 | 0.00 | | | | |
81
+ | <nobr>play_stacking_toy</nobr> | | | 12 | 32.00 | | | | |
82
+ | <nobr>play_tic_tac_toe</nobr> | | | 2 | 40.50 | | | | |
83
+ | <nobr>plug_in_charger</nobr> | | | 4 | 4.00 | | | | |
84
+ | <nobr>pour_balls_into_vase</nobr> | | | 62 | 62.00 | | | | |
85
+ | <nobr>pour_by_language</nobr> | | | 0 | 0.00 | | | | |
86
+ | <nobr>pour_liquid_into_cup</nobr> | | | 44 | 44.00 | | | | |
87
+ | <nobr>press_by_number</nobr> | | | 100 | 100.00 | | | | |
88
+ | <nobr>push_T</nobr> | | | 2 | 2.00 | | | | |
89
+ | <nobr>put_bottles_into_dustbin</nobr> | | | 98 | 98.80 | | | | |
90
+ | <nobr>solve_equation</nobr> | | | 0 | 0.00 | | | | |
91
+ | <nobr>sort_nesting_dolls_by_size</nobr> | | | 20 | 20.00 | | | | |
92
+ | <nobr>stack_blocks</nobr> | | | 44 | 50.00 | | | | |
93
+ | <nobr>stack_blocks_by_language</nobr> | | | 8 | 9.20 | | | | |
94
+ | <nobr>stack_bowls</nobr> | | | 52 | 58.90 | | | | |
95
+ | <nobr>store_laptop_and_headphones</nobr> | | | 40 | 51.20 | | | | |
96
+ | <nobr>store_tools_in_toolbox</nobr> | | | 0 | 0.00 | | | | |
97
+ | <nobr>swap_T</nobr> | | | 22 | 22.00 | | | | |
98
+ | <nobr>swap_blocks</nobr> | | | 32 | 32.00 | | | | |
99
+ | <nobr>sweep_blocks</nobr> | | | 4 | 4.00 | | | | |
100
+ | **Avg (per seed)** | | | 30.2 | 37.45 | | | 30 ±0 | 37.45 ±0.00 |
101
+
inference/policy/Bagel_Robotics_1/evaluation/seed-1/verification.json ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "revision": "f57e1633ffed59761bc8e568cfe7530c4b33381c",
3
+ "files": {
4
+ "scripts/robodojo.sh": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3",
5
+ "scripts/internal/summarize_result.py": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2"
6
+ },
7
+ "unmodified_official_sources_verified": true,
8
+ "source": "https://github.com/RoboDojo-Benchmark/RoboDojo",
9
+ "generated_records": 2100,
10
+ "matching_reported_tasks": 42,
11
+ "sha256_of_native_summary": "a2946a8c62a7cf3d1e64d8e8655d462533e2a3937b49756bbd458ab81586cad9",
12
+ "native_markdown_task_cells_verified": 42,
13
+ "native_summary_copy_identical": true,
14
+ "original_selected_records": 2096,
15
+ "labeled_replacements": 3,
16
+ "separate_repaired_variants": 1,
17
+ "memory_original_layout47_unscored": true,
18
+ "nebius_retention_verified": true,
19
+ "clean_offline_regeneration_identical": true,
20
+ "all_scores_match_previously_reported": true
21
+ }
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ # Preserve the official summarizer's trailing blank line byte-for-byte.
2
+ summary.md whitespace=-blank-at-eof
3
+ summary-original-selection.md whitespace=-blank-at-eof
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ # Three-seed evaluation data
2
+
3
+ This directory retains the unchanged official summaries, episode data, verification records and regeneration script. Results and reproduction instructions are now in one report.
4
+
5
+ [Open the consolidated results report](../../EVALUATION_RESULTS.md#reproduction-files).
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/SHA256SUMS ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ a79a06b50af6ef7df8f313c0cb84e954f9a328f4eac1c394216de0681ade251f .gitattributes
2
+ c965c48fbaea7e1dd5852ff556d11c441a4897245a9b132fe55afbbdd1488314 README.md
3
+ 6ed678932036e827103cd89de0c4748515c54b178eb7862c201eb2976d28af21 episode-scores.csv
4
+ 0b19a2cc51364baaacb070c5245eef274b5e9b4393b550ca30365fb4d0c603fb generate_summary.py
5
+ 7a2ac06a26b8496271a4e5c1d334ed9693d1b96e8058b99834e5e52eaa46d842 original-only-breakdown.md
6
+ b5284fdaf2d0eb3dd7dfc36e91d2842183d4695b2317c1338f145ed78694db57 protocols.json
7
+ 5e710be09f95a3200c1c33755f667b9f95797b4ef5d696d3ee006d3dc6e09f92 replacements-and-repairs.csv
8
+ 2a7fb1a4ba1d8374b4f0317644f75ba360db17f43a0f22a3c5da50cf02da3544 replacements-and-repairs.md
9
+ c0fe627bdbde293db26714658eac7695d2f6cb5d6e66842a3d29f93edcc70c66 results-breakdown.md
10
+ 147306c5805b47d1ecaa7179db7f9a15b1b682e492630e6ab685160b4c23ab63 retried-originals.csv
11
+ c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5 summary-original-selection.md
12
+ 141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f summary.md
13
+ ab9c45c7766c3760304aeb53c517dbc7ccebc97f970016d4a845ccf2df936141 unscored-episodes.csv
14
+ 958dbd046ff0d94562a5281f391ba3dfb7274f7f020a40bf191df87fae1d3044 verification.json
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv ADDED
The diff for this file is too large to render. See raw diff
 
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Regenerate the unchanged RoboDojo summary from the published score ledger.
3
+
4
+ This groups already-scored outcomes; it does not run or rescore any episode.
5
+ Python standard library only. See README.md for the pinned RoboDojo checkout.
6
+ """
7
+ import argparse
8
+ import collections
9
+ import csv
10
+ import hashlib
11
+ import json
12
+ import math
13
+ import os
14
+ from pathlib import Path
15
+ import shutil
16
+ import subprocess
17
+
18
+ SCRIPTS = {
19
+ "scripts/robodojo.sh": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3",
20
+ "scripts/internal/summarize_result.py": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2",
21
+ }
22
+ POLICY = "Bagel-Robotics-1_with_labeled_replacements_and_repair"
23
+ SUITES = {"precision": 400, "memory": 300, "generalization": 600,
24
+ "long_horizon": 400, "open": 400}
25
+
26
+
27
+ def main():
28
+ parser = argparse.ArgumentParser(description=__doc__)
29
+ parser.add_argument("--robodojo-root", type=Path, required=True)
30
+ parser.add_argument("--output-dir", type=Path, required=True,
31
+ help="New or empty output directory; never use a live eval_result directory")
32
+ parser.add_argument("--view", choices=["reported", "original"], default="reported")
33
+ args = parser.parse_args()
34
+ root = args.robodojo_root.resolve()
35
+ for name, expected in SCRIPTS.items():
36
+ if hashlib.sha256((root / name).read_bytes()).hexdigest() != expected:
37
+ parser.error(f"{name} must match RoboDojo f57e1633ffed59761bc8e568cfe7530c4b33381c")
38
+ ledger = Path(__file__).with_name("episode-scores.csv")
39
+ with ledger.open(newline="") as stream:
40
+ rows = list(csv.DictReader(stream))
41
+ if len(rows) != 6300:
42
+ parser.error("Expected 6,300 reported outcomes")
43
+ seen = set()
44
+ for row in rows:
45
+ row["seed"] = int(row["seed"])
46
+ row["layout_id"] = int(row["layout_id"])
47
+ if row["success"] not in ("true", "false"):
48
+ parser.error("Invalid success field")
49
+ row["success"] = row["success"] == "true"
50
+ row["partial_score"] = float(row["partial_score"])
51
+ if not math.isfinite(row["partial_score"]) or not 0 <= row["partial_score"] <= 1:
52
+ parser.error("Invalid native partial score")
53
+ key = row["seed"], row["task"], row["layout_id"]
54
+ if key in seen:
55
+ parser.error(f"Duplicate scored identity: {key}")
56
+ seen.add(key)
57
+ counts = collections.Counter((r["seed"], r["suite"]) for r in rows)
58
+ expected = {(s, suite): count for s in range(3) for suite, count in SUITES.items()}
59
+ if counts != expected:
60
+ parser.error("Seed/suite coverage differs from the published selection")
61
+ selected = rows if args.view == "reported" else [r for r in rows if r["record_type"] == "original"]
62
+ if len(selected) != (6300 if args.view == "reported" else 6271):
63
+ parser.error("Unexpected selection count")
64
+ output = args.output_dir.resolve()
65
+ if output.exists() and any(output.iterdir()):
66
+ parser.error("Output directory must be empty; existing results will not be overwritten")
67
+ output.mkdir(parents=True, exist_ok=True)
68
+ eval_root = output / "eval_result" / "RoboDojo"
69
+ grouped = collections.defaultdict(list)
70
+ for row in selected:
71
+ grouped[row["seed"], row["task"]].append(row)
72
+ policy = POLICY if args.view == "reported" else "Bagel-Robotics-1_original_selection"
73
+ for (seed, task), episodes in sorted(grouped.items()):
74
+ episodes.sort(key=lambda r: r["layout_id"])
75
+ n = len(episodes)
76
+ target = (eval_root / task / policy / "arx_x5" /
77
+ f"{seed}_ckpt_name=cp1000,action_type=ee" / "2026-11-09_00-00-00" / "_result.json")
78
+ target.parent.mkdir(parents=True, exist_ok=True)
79
+ data = {
80
+ "eval_time": n,
81
+ "success_rate": sum(r["success"] for r in episodes) / n,
82
+ "score": 100 * math.fsum(r["partial_score"] for r in episodes) / n,
83
+ "details": {str(r["layout_id"]): {
84
+ "layout_id": r["layout_id"], "success": r["success"], "score": r["partial_score"],
85
+ "record_type": r["record_type"], "replaces_episode": r["replaces_episode"],
86
+ } for r in episodes},
87
+ "_aggregation": {
88
+ "source": "episode-scores.csv", "view": args.view, "checkpoint": 1000,
89
+ "note": "Score-field projection of retained results; outcomes and actual layout IDs unchanged."
90
+ },
91
+ }
92
+ target.write_text(json.dumps(data, indent=2) + "\n")
93
+ env = dict(os.environ, ROBODOJO_EVAL_ROOT=str(eval_root))
94
+ result = subprocess.run(["bash", str(root / "scripts/robodojo.sh"), "summarize"],
95
+ cwd=root, env=env, text=True, stdout=subprocess.PIPE,
96
+ stderr=subprocess.STDOUT, check=False)
97
+ (output / "summarize.log").write_text(result.stdout)
98
+ if result.returncode:
99
+ raise SystemExit(f"Summarizer failed ({result.returncode}); see {output / 'summarize.log'}")
100
+ shutil.copyfile(eval_root / "_summary.md", output / "summary.md")
101
+ print(f"Generated {output / 'summary.md'} from {len(selected)} retained outcomes.")
102
+ print("Score-only export: rollout videos are not copied; native video-check messages remain in summarize.log.")
103
+
104
+
105
+ if __name__ == "__main__":
106
+ main()
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ # Original-selection-only comparisons
2
+
3
+ The original-selection-only suite and affected-task tables are now in the consolidated report appendix.
4
+
5
+ [Open the consolidated results report](../../EVALUATION_RESULTS.md#original-selection-only-comparisons).
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json ADDED
The diff for this file is too large to render. See raw diff
 
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ seed,suite,task,report_task,task_variant,episode,layout_id,record_type,success,partial_score,replaces_episode,attempt_count,protocol,completion_record_sha256
2
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.4,fill_egg_holder--0028,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,1d93916dbdad621225ec6ffb2db26ee6ed268282df2f14dce9f2ba95b2ae52fc
3
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0051,51,reserve_replacement,false,0.0,fill_egg_holder--0035,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,4f3ddddc86a5d4f3eafb0be166557bbb19c1117e7c2df55b0608e33aacab0401
4
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0052,52,reserve_replacement,false,0.0,fill_egg_holder--0037,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,69c5e4ff890baf37074de001e6c06b49907a02fa7946c22939fc34546119a428
5
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0050,50,reserve_replacement,false,0.0,imitate_sorting_sequence--0004,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,dd38bacf246a87c684c2e68b7d4e4c7f185ac0547366e5de3a6d22b73f946213
6
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0051,51,reserve_replacement,false,0.05,imitate_sorting_sequence--0008,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,0abc5fc0c7da5feb900757c0e2fc356c90ce49d40488d538371ba6ffc4274d5a
7
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0052,52,reserve_replacement,false,0.0,imitate_sorting_sequence--0017,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,9ba41682374f093d94b0a74bf9ea59a064c9cad635deea9c9932b08eead99a83
8
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0053,53,reserve_replacement,false,0.0,imitate_sorting_sequence--0029,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,02215431e7f053ea82ab92f34938973462210e562b1f5758e7a7ae701d29c300
9
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0054,54,reserve_replacement,false,0.0,imitate_sorting_sequence--0036,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,9cefe96e30626cab2cae45ff9d22247af7f50ea87c8c62d2b00c802fb8b66a31
10
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0055,55,reserve_replacement,false,0.0,imitate_sorting_sequence--0048,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,3d13434232bee26e09e6ce1ce09a111ebedbb5dd7d4c9010f2d181f5f0c1d5d4
11
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0056,56,reserve_replacement,false,0.0,imitate_sorting_sequence--0049,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,42937ed09990643cf2a5ca623319af8a60b98e2a2fca996b4c07565341169816
12
+ 0,open,general_pickup,general_pickup,standard,general_pickup--0050,50,reserve_replacement,true,1.0,general_pickup--0029,1,723d1a0a15d21a3ad95777bf80414a63919dea60ad5656ad7954c55bbd245c22,6f44cc5eff3b39a922cd842e660a8ce3f858a014840a8ec8213c9831f787321c
13
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0051,51,reserve_replacement,false,0.0,deposit_coin--0036,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,32453dc5b75f7525cc6b442bfe5281fc46e073773be6c26c1de68801e5b819b9
14
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0052,52,reserve_replacement,true,1.0,deposit_coin--0037,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,a3e89dc4ccd3fab31bedc56c0d321e5198eeff6c8ad7ba20deaa1bb0a9679180
15
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0053,53,reserve_replacement,false,0.0,deposit_coin--0041,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,0bdcc61a1cdddbb7cb1c693afbf3246a756776bc288981605a8690991ebf01a8
16
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0054,54,reserve_replacement,false,0.0,deposit_coin--0043,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,51191edb880199516b11b343561a3bacb9d2d47c491429584b6d89165ca51c91
17
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0055,55,reserve_replacement,false,0.0,deposit_coin--0031,1,85a5123e2ac96326ad2c877c1f171f5e5b14d819d62951a594cd7c81f5bad14d,cc85bdc9344a35ad4f37414b5ead9cdf6ff3c9bed61c5847c567f00d2fe81bd2
18
+ 1,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.0,fill_egg_holder--0015,1,a876503909848ac16a62806bed0177445f8ee0681970f82f1d49150a1ab5b5b6,e997fd981a5855be673538943400865788b20f4850d0da684d9752ff4778bafa
19
+ 1,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0051,51,reserve_replacement,false,0.25,fill_egg_holder--0020,1,a876503909848ac16a62806bed0177445f8ee0681970f82f1d49150a1ab5b5b6,8a9be90e270a00edd75615061418570fe83af25e28eb742ce2548d10b792713f
20
+ 1,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0047,47,repaired_variant,false,0.0,imitate_sorting_sequence--0047,3,8c8f9894f60ac3a905644871b9675d9b6c87ba4e3f52e7228951b49ac1623d75,9756370436d26011a577e74aaa1f22ed3a88300cd1c5d583677346a05f171274
21
+ 1,open,general_pickup,general_pickup,standard,general_pickup--0050,50,reserve_replacement,false,0.0,general_pickup--0005,1,e32aa0e95d1bdfe638c65520e8512c1e0da1cab16744ac560ba2c8ad59dcbc5d,4f2a6cebf43dc12ea30ee63981e720e7afcec93b03ad3f6a4ba460c59f2d9cf4
22
+ 2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.25,fill_egg_holder--0013,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,e0b2398081fa2dc27515ea5d8515d2dca795224478847f58cc637eabecaf7252
23
+ 2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0052,52,reserve_replacement,false,0.0,fill_egg_holder--0018,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,7d4a1940b244b7abd5211577bf2ef8223f79d78fe1d14ca84829009bf55bbcdc
24
+ 2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0053,53,reserve_replacement,false,0.1,fill_egg_holder--0040,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,c6b9073d83de5c5bb3566e893f019847454af5da6a3762c45d6c6dbbd70be8af
25
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0050,50,reserve_replacement,false,0.0,imitate_sorting_sequence--0016,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,a2e784ebe95a5cd15629ef1713e468a47472ea8d49d62470cfe0e2bf1302352f
26
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0051,51,reserve_replacement,false,0.15,imitate_sorting_sequence--0024,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,bfceb828b8d01210b287caadb9dcacba90002a3d2823caf084e59f98137c7880
27
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0052,52,reserve_replacement,false,0.0,imitate_sorting_sequence--0032,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,8057e37a3180f236bc0f65e6ec62dc79335413985d311999b79317f604873072
28
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0053,53,reserve_replacement,false,0.0,imitate_sorting_sequence--0043,1,d93b3d1ec634a3f609730818701deead33df6bafedb5f7e3afceb81593046ed1,986c4f50cf26c41e31ccb5906d4678117c60de624b1fa140f167d3e8ddc68f0a
29
+ 2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0050,50,reserve_replacement,false,0.0,deposit_coin--0024,1,23fa76d39404c5213a6810d134bd9f012655ddd9faf95176523b826cbb2b5913,0137e168203e7dbdb8f82e5340bc2c1cca2bd033f41ec22569511532fa334938
30
+ 2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0051,51,reserve_replacement,false,0.0,deposit_coin--0033,1,23fa76d39404c5213a6810d134bd9f012655ddd9faf95176523b826cbb2b5913,d27ce8a007a1c1eee5de705f4c034d66e0f15995be335e18cea58fa3d2fe952b
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ # Replacement and repair appendix
2
+
3
+ The complete layout ledger, repair details and completion recoveries are now at the end of the consolidated report.
4
+
5
+ [Open the consolidated results report](../../EVALUATION_RESULTS.md#replacement-and-repair-appendix).
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ # Suite and task results
2
+
3
+ All individual-seed suite scores, task scores and the standard/randomized split are now in the consolidated report.
4
+
5
+ [Open the consolidated results report](../../EVALUATION_RESULTS.md#task-results).
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv ADDED
@@ -0,0 +1,180 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ seed,suite,task,report_task,task_variant,episode,layout_id,record_type,success,partial_score,replaces_episode,attempt_count,protocol,completion_record_sha256
2
+ 0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0018,18,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,9d2bd22ee5480bec9ef3dd8264506066dfaca97806c27aa9429395ed1468fffc
3
+ 0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0019,19,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,7c12a87a4818edd6cec4aec9e186417f630135417c73ac7e943e6e702e9c4640
4
+ 0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0023,23,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,5f3ea13c20b57396c7e61dc8622f63efc604dade39ac762d84c9c18c6e35d170
5
+ 0,generalization,arrange_largest_number_random,arrange_largest_number,random,arrange_largest_number_random--0001,1,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,1ce025b50c5915039d92538be05ae3ae8d9371f9b65ed759317bde56d4237ad2
6
+ 0,generalization,arrange_largest_number_random,arrange_largest_number,random,arrange_largest_number_random--0006,6,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,8a475f841cb5c56dae3f3feb582210d2bf3bb9822fe170743471c17aeecd8d1a
7
+ 0,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0007,7,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ad6c71d820ba07c9f7db95e76721d31d78dd21de59d6bc6946d99474a9a08f67
8
+ 0,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0016,16,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,3e0876a9f314bfef50c1d30a7d87eeec8931a8eb9bba7b8e61b65c270be63c66
9
+ 0,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0002,2,original,true,1.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,9ff73e28ae5c2abe86c82d735586a5ba6b69664b6eeac96cb73308e37966c05f
10
+ 0,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0003,3,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,84ec4e15129abf87f245e8bcbb7a6527655bb152431b48a27f292a8a678c2648
11
+ 0,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0010,10,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,54294e8d19bf2d405dbc84da9c96a7a1182f717103230fc980011d65b11bcb4f
12
+ 0,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0021,21,original,false,0.15,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,d34347a9037adc764fae8323c8df1cbfc245aa48890bfe5119144c2d9591e5f1
13
+ 0,generalization,make_toast,make_toast,standard,make_toast--0011,11,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,38adc2e894856e55c23e26370342325f59df6611b786bbcbd9f4d67a5a2933b9
14
+ 0,generalization,pack_objects_into_box,pack_objects_into_box,standard,pack_objects_into_box--0021,21,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,a1eeb9d7ab1c718d04f25c52cb44fe782a977845afe44ef49dacbf9642b94a21
15
+ 0,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0013,13,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,9d2e824ac546bd947deb3004f0b3c2a9769bd44ec02220ff67ea9001a5a3af0c
16
+ 0,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0017,17,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,cab2e03bfd4c5d3102718f03e90e48dcc3d89d64df5a1c820795eb41f69c8e67
17
+ 0,generalization,pour_liquid_into_cup,pour_liquid_into_cup,standard,pour_liquid_into_cup--0001,1,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,043b9ac45a0b5eeab9d0197d02c308cd8d942781c92fc7db2473e0d57fbff38a
18
+ 0,generalization,pour_liquid_into_cup,pour_liquid_into_cup,standard,pour_liquid_into_cup--0023,23,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,d5f4b028954a195331bb1238ac4541dad07acaa83aa9d3c822d471dc9f587b80
19
+ 0,generalization,push_T,push_T,standard,push_T--0005,5,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,82b3795ca21f7eafc415c2f05cc20960b0a5b34cf3895ecc64882a6fab7e6826
20
+ 0,generalization,push_T,push_T,standard,push_T--0017,17,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,69d77913886d700a8763d1a21a5248b8632ffb03b37a46ed9f0d425d3578188e
21
+ 0,generalization,push_T_random,push_T,random,push_T_random--0007,7,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,524ebe155d7e5f9eddf4b04caa46d3bd3beb459c8a5e17d2d65cbf235b7c7821
22
+ 0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0007,7,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,82e5b52c2d5b5da1e39a110cbb019e48a7d8ca078f640cb865b9ee10a233e781
23
+ 0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0011,11,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,9fd0941210f7252588b9929d56f38d3fe5548d9b319ea7d7bdc07f9b1c2e00b3
24
+ 0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0013,13,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,b2ab34be60d1bab7fe6ef20e27adb15c7f5144ff84372fef1702eac7d349123e
25
+ 0,generalization,stack_blocks,stack_blocks,standard,stack_blocks--0006,6,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,7d511310959bd655bad242eb6a09ecb1231a577242cc3e93e68f233b64d25403
26
+ 0,generalization,stack_bowls_random,stack_bowls,random,stack_bowls_random--0007,7,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,9c87454a6d9559a4965b32a3546770b9ab838a127fd197980be9bf7478c7196f
27
+ 0,generalization,stack_bowls_random,stack_bowls,random,stack_bowls_random--0023,23,original,false,0.15,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,30d4f55e7aa2ffa1cd9da2c94b7527cecd27da85fe69c95e7a2dec37f26ccb70
28
+ 0,generalization,store_laptop_and_headphones,store_laptop_and_headphones,standard,store_laptop_and_headphones--0006,6,original,true,1.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,896663cca4d9f4ad591083afac9d67076cfa70cae5ee87553f9e4c1a59d50183
29
+ 0,generalization,store_laptop_and_headphones_random,store_laptop_and_headphones,random,store_laptop_and_headphones_random--0014,14,original,false,0.2,,3,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,5404aba674e150a13358d2fce30ba2bf79bf795cee0b05d897367034a59d9f84
30
+ 0,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0010,10,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,01c7847f424303ffd38b070a434eea93bdb3b1d57612505b9aa0d676b2bb014c
31
+ 0,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0016,16,original,true,1.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,575f9b0ba6f3c3fd6d15673f31a1ee5a9e234d3f7bee77f5f6e1c0e672cbe851
32
+ 0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0002,2,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,63733e1ee948f3c75ca98eca3446e6475a02c3070d5d6424a0e7bdaf21e608df
33
+ 0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0003,3,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,b659f71b60fad6edab1b155b3b9a39feaf9b551faedbc59a45eabe5b16fd324b
34
+ 0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0004,4,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,59b362ae4bcbaa0196ba18c06a6458d7df604e9fc49ab3c83098abd635c52b0f
35
+ 0,long_horizon,classify_objects,classify_objects,standard,classify_objects--0027,27,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ccf5fe1d901adf28c9f0f6ceed35b8cbec95712b1f17ce00fb3b9b053c3da798
36
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0006,6,original,false,0.25,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,75c977869cdb14ebd90cc2b8a199a51c34b9372f52eeacea7ed39b8467c4f600
37
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0039,39,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,6ad2f9118f46af9f67096a9f36f6ea741a3d78f7491cd8f89108b09d33091d0c
38
+ 0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0048,48,original,false,0.25,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,42af9d17a3d8c9f12e690cea1c98602c98e1cb9f470e66a493cc8e088bb92edb
39
+ 0,long_horizon,fill_pen_holder,fill_pen_holder,standard,fill_pen_holder--0025,25,original,true,1.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,56d5a40edf56ce754d19a1e38cacffef1bba61b0c9676bf7f39712b70a883ac7
40
+ 0,long_horizon,fill_pen_holder,fill_pen_holder,standard,fill_pen_holder--0048,48,original,false,0.4,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,6c9d0c9befd2c74f75e53d847b26037f18d83e636e1297a7ca6ed6bb3c926061
41
+ 0,long_horizon,organize_table,organize_table,standard,organize_table--0000,0,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,2e5c1eae898019bd9fba4eb5af3d8d300c86e947863a48da8267acccf6e3b4e3
42
+ 0,long_horizon,organize_table,organize_table,standard,organize_table--0013,13,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,7bb5875819e0e46d32e050024ed40133999e834b835d548ef490919f6753b2e9
43
+ 0,long_horizon,organize_table,organize_table,standard,organize_table--0024,24,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,3ce909d881288aa5e3a7aef65bfeb4e256ab1aa0b2ed3134782f9bd09b4fe5ce
44
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0001,1,original,false,0.75,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,52c5d76a5158dae269122142a79328ab638c2f5122bbbeeec34b1a0940edabca
45
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0005,5,original,false,0.5,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,415dc8f729394b39a951a3f57d0223d65acfbfcc98d520e15c08f751528f01d4
46
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0009,9,original,false,0.75,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,22f9e233fdf59fd854a95473414fe18ab6dc4972ce632f5968f7e6fa80bfdc7a
47
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0012,12,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,a138013173611e8936a5d79732da540321beaae41448735f49d796872c9ca1fa
48
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0016,16,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,bafe6e83dfc30cf63483783be5e543b5c1830fc9a375fef8202d37566ae2f5ca
49
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0018,18,original,false,0.3,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,56dc83b882811fe048e048a8091c9867913317ea5171c40340d5e9906dea090f
50
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0030,30,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,31e915198b5ff931bc37c3fa445a645ec9874abb798f55b25555f1bdd66d840c
51
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0031,31,original,false,0.1,,3,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,36c173d4d8727c0be548c691bb744182f35b8f298de756679b1021fefd3713bf
52
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0041,41,original,false,0.5,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,3d8f83a609e30e566f05bf59fc330b29f9698d97450bd0fb2102e5c63383cb27
53
+ 0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0042,42,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,626f48a82062746712e69e74d91bcf3499a6b88c6000c7ec5258518161298bdd
54
+ 0,long_horizon,put_bottles_into_dustbin,put_bottles_into_dustbin,standard,put_bottles_into_dustbin--0037,37,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ad972e1019644f623c968861dbf59d95782867f0c06c9e99da63eaf97662441b
55
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0034,34,original,false,0.0,,2,6b32e48eb89420065ceb5ad01bc5ba3b1cc99086cb424e299e5c9f5281297023,3eed7b82f11a1a3dc8cbf4b0c7da5c0b48056da51cbaeb0891abeb898c205b91
56
+ 0,memory,press_by_number,press_by_number,standard,press_by_number--0015,15,original,true,1.0,,2,6b32e48eb89420065ceb5ad01bc5ba3b1cc99086cb424e299e5c9f5281297023,bd7a89fe3452da0f73d1dd2f91d4bcadf8cd5fa3c6b9728290e3144a650e6e0d
57
+ 0,open,solve_equation,solve_equation,standard,solve_equation--0014,14,original,false,0.0,,2,847b0e9ac20a32a47b64b977e656d20c666d8ad62b28b2de5e598ea7fe34fa8b,5907291d39a7fbbf5f6ee1adf57796fb4c136d8962619ae538a7baa44a0a26d9
58
+ 0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0014,14,original,false,0.2,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,24dbbc8616c706068aafb7d00a779d83f3be9967b5f60b2c3f1ab62e207f0741
59
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0000,0,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,246eeacdd1e238b5dc8c4eaa586561ab5b77357dcff794d93f8d9699b671cbf0
60
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0001,1,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6118d38f8f0b13e6e40bba2eb543e48c17e129c9b198ad34936ee1816a09c99e
61
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0002,2,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b7d42329a34c0b97da922529338d9577b2bb4ebbfe0467d4fd0171aa6447b75d
62
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0003,3,original,false,0.4,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,f01e22f7e7ba192b8581804c2b4c50977388ae80b00429cb69b5108845ac5d3e
63
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0004,4,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,9405c6bbbac4661a8b9e9c06018ef6877cbfb4a5bd28defe03cb550f45a105a3
64
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0005,5,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6a2b01f07552ad59cc5cd4239e9ad15f32b52094965865b6deaaaa57674085e7
65
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0006,6,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,bbf45c09db9bdf7f18a9a5ed921f70569290290f105052f2d9c8ba63ae6dad8e
66
+ 0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0007,7,original,false,0.4,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,caf5a5e6e61edcf91a0636d7650ad6310224449c18ba10c4267c1f4855cc1d23
67
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0000,0,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,4cc5ae100707d8575ef496db66401dc495f82e8ada7868c1ce489ac8e2aca5d9
68
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0001,1,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b39a081c41ad873925f783f700c2884ef75b6e2fa8842fa6e96228c2d54e2578
69
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0002,2,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,aa1cf4c181b7290f703f51d73cf76b7e0689734d8f0475c9f894227c1802ae58
70
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0003,3,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,68683ff65457e32796f897d2cce79a7f5382e915629adef29479c450744d3910
71
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0004,4,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,3305f942d10b3724426677e374dafe3acc740264570c41206eee366ed13c8a72
72
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0005,5,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,1c5136ca12ec6406779090abb08effbf00c883f5544920ec1874a3efc3d2b4c4
73
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0006,6,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,8ec87867358523905f77573d220dd95d69a51b49b92154f0b7532cf76ed14a4e
74
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0007,7,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,cb028f0e32fa71e0811df72b8bd27f89a2097fff3c4d5f3df965cd23f559e232
75
+ 0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0008,8,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6971a629863f7cfaf4640a5d23b9bb62362156dc6537e4c19c4636e0e53bb66c
76
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0004,4,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6b4944be17f0fbed035cb566579f4e1079f5acd6f53719e5c3ba1bbda4c71392
77
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0015,15,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,a459e0504046ed10c8d077324065026900b16a972a6a7859c9a713835ce66629
78
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0017,17,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,50691d529242d7394066b5d1fb2fcfbd1c645bb1cea4c6d5fe30135b8ccd644a
79
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0019,19,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,4594a999afc254e283196bd5f03a4cb1bb66985cb2372d928d64c02ae2f52f81
80
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0022,22,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b1ab680a25bd8fd542416db58345853f914a8a2704c78ff3207d223d9e74db39
81
+ 0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0045,45,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,aed9928aede3b15106ba6154abe743117649a77cd27995c28fa617949e0698f1
82
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0000,0,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1787292e76ac9e88a467c3087c8bb30be831d322e390a16ff941cb2d0fe84084
83
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0001,1,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1777bd3c3e290fad72a3396b0b9161845112410ad2ce6fe8650643de0bb4208f
84
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0004,4,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,842eb1343a106d10037b389711249f0fad480537793f89b112bb61b8e9bac644
85
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0006,6,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,5c71f2fe873f06068c9dd9703ae9c690af71e626e7b90c07f4f8a61c75d84601
86
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0007,7,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b4202fa61804ff4e750421f14428e413c81aa3f909d15df4bcdb11d08d7702cc
87
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0013,13,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,ccab300ea1dd05cd7e3c3e03103facd684b453c46e2692ed907c20edf10e1c53
88
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0018,18,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,5999b54b8dafae9842b5cd77d71a45794c21f112e1d91356870d45bf0bcec3b7
89
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0021,21,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1aed902c59236af04b9068e670d5c5cf96a8a1ca836b4a4d46f304d4c22236c9
90
+ 1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0023,23,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,37b59debc232efa1a3622451954097879ff57487a8c3c8a65c4900096a98294e
91
+ 1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0003,3,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b9f15c9ffeb7eabec45ad246cc74a76bab6249e9c37e6958b6570b14c97c6a5b
92
+ 1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0004,4,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,8407bbd937cefeab1e828ee7593cdcb1be88a0c0d9a231253aa15a1442ea4a32
93
+ 1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0006,6,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,728d6bbed636e7e9b8f988d4bf25e845f39cbfd57a6868a22117407e8c2ca454
94
+ 1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0010,10,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7a7e2044f3303973aee1045cdfbe9473295767a24c4df88fad4137c6513228af
95
+ 1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0014,14,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,832c1aa79c2fc686eab658b024e83c268883808519364431799ea6c9953431ec
96
+ 1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0018,18,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,82daf42a3916b578d786241cb337f033a9c937443fcb01f686a7f9c098afb440
97
+ 1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0022,22,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3e6c0bd42d696ed812d68119a5f1d6c7a47dfe1f905703b5d5b93e836e976e9c
98
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0001,1,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,c563a0ff7965f74fb6c44f96f9371ab9297d763696839d96bda20dee2f864785
99
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0006,6,original,false,0.4,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4beaccc987935cd3d3d5d3f51f8dc137c3d48df660e9a42a027611b0b0fdb0dd
100
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0009,9,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4dba4e99cdd67b6bffe630a4252d21a7ef471f6aef2e69c9aa6f2bd1a8d0d181
101
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0010,10,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,9c727ca0f896f9b0aa375f2a4a791c5a6d13cea34de4b2cc09bf83fcb7dc5869
102
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0016,16,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,ea98a5251c7000b2bbe284385cf90c43e59a81fe9b1c0e59c2dfcc5f3425cd80
103
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0018,18,original,false,0.15,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,e1515bd2bc6705e1827c96e6aff7ae42fda82aa0e93b1f77451ed841326d50d4
104
+ 1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0019,19,original,false,0.4,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,32bd292bcd81f3c43738c5c73f51e7b49343dfc91a6227613a51b2138796c71d
105
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0002,2,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,396e3ac0e649db529141627e9f0680a2b2336aa102fd0831f31fd6191a026753
106
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0004,4,original,false,0.1,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b73e57b84cb2a20a59a69cfd8c6f682c03df413e8f3e59afdf90f2798546b9da
107
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0006,6,original,false,0.25,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b8c9a1bca6f4ee7cf27ce39fddb83572d56c07c7f70271d18dcf80f945f6ba57
108
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0009,9,original,false,0.1,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1ba0ee80bfff5f39d09702fb0f1961aab435b4a09bb6218b33e7e131faf892a0
109
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0012,12,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,8ce7c78d63cebf227dc154a786af097d24c8fab7b39d6b86c7223552dabd1b4e
110
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0020,20,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7697c9fd9e9538121b975c5c98c37f64890b7e233f2fbe842aa4128eda2564bd
111
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0022,22,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,144188a615dee96afece66deeb7218b8c74474ce6a34a783c0317e488e890fe8
112
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0023,23,original,false,0.1,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,83a4ef649f1b855aa5de54e3a3c1a2e811151a44cf6455960ad7fdeb2500e4ee
113
+ 1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0024,24,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3bf75c5308d09c22ecb2007ae4ccb6e9eaae037be622fa82235935b8f9972136
114
+ 1,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0021,21,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3fa829be5f8eb370a72cc09d7f635a1d9fbc79f6e3bea28f5398ea893c8c73ed
115
+ 1,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0023,23,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,28a0b6230e15cd428c73f63e4baaf348e0361082073e65005cd82c7dbd30596e
116
+ 1,generalization,push_T_random,push_T,random,push_T_random--0010,10,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,c6bdb024677d49ba2878e0bc3d9d82d067f75bd180ada40879a8d3bd3fe948b8
117
+ 1,generalization,push_T_random,push_T,random,push_T_random--0011,11,original,false,0.0,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,9df9daea9d85328436a92b3b036b5f54d00b3e918e579ca8d992e6b9e1177b23
118
+ 1,generalization,push_T_random,push_T,random,push_T_random--0015,15,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4dee106852ba4035b151450a9ef78edb345a88ba414b6cc350e0ebcb0bd681c2
119
+ 1,generalization,sort_nesting_dolls_by_size_random,sort_nesting_dolls_by_size,random,sort_nesting_dolls_by_size_random--0000,0,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,9341ca1ee851e48b9fb16bb92f1fa25fe0afff95b3a51474b7db4502496d424a
120
+ 1,generalization,sort_nesting_dolls_by_size_random,sort_nesting_dolls_by_size,random,sort_nesting_dolls_by_size_random--0001,1,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,8597697c8e092138b70a13b25babcb3d1f32a7beaa9ec976f09f7c6e58088365
121
+ 1,generalization,stack_blocks,stack_blocks,standard,stack_blocks--0006,6,original,true,1.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,cd78fc6ba624dc8ef29064aa35d1b4d76df68c92a059b238d60dbb941db1b228
122
+ 1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0002,2,original,false,0.15,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,5b0a82132e4acf21052c89b1c886d90e664fb18b00afbc499108b10edce735c1
123
+ 1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0011,11,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,6f17f7dfa1c67d6e4668ac3cbbd3bb335e273dd13ac21c536c546eb52eb94b36
124
+ 1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0014,14,original,false,0.15,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,ca56a1313166a2ad70764d0f8ddb860a1545e24f26cba78f8e8bf2ca3f2599c2
125
+ 1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0002,2,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,ce1a910006f377dce23cae16ad65db3f3076a991edea6ab76fbfe9c2c5792964
126
+ 1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0004,4,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,86e2db0cf9a5bbd7e2cf44c44b32550fbb5d425aacc5c8363370fec0456bb7c7
127
+ 1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0014,14,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,81584c3a793ba3ad4116adcaae944e3b2c331afc4e63d3ae52213e8796843a1d
128
+ 1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0015,15,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,c124fc6274a44db93aea919e3591d292c7df880e4f2af1b82dc21951f3f344b9
129
+ 1,generalization,store_laptop_and_headphones,store_laptop_and_headphones,standard,store_laptop_and_headphones--0022,22,original,true,1.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,73d2d6153c18b86d072d4b79fedebd9ca745613e861fccc94e3e52e49204b9d3
130
+ 1,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0000,0,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7633f7a79b4729864957c784248d3cb9c716ba5ba2dffb3c354d1007138a2bfb
131
+ 1,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0003,3,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,09859ce9fb0d11a21a50d2007ab9d22600d2e79dcdd64d040bc9bdd9641025ad
132
+ 1,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0049,49,original,false,0.75,,2,c2867b1bb38b88d54cbe92103a5cf6d2020c3d44f67c89412182265c4bac8256,aedb082c7c577b11cf7e79cf5a765c14004ca0607872e65a508712f256f8f5b5
133
+ 1,memory,match_and_pick_from_conveyor,match_and_pick_from_conveyor,standard,match_and_pick_from_conveyor--0048,48,original,true,1.0,,2,edcf660c7679d8ced9b72d4d57acc29691c8fd5fc91577e82532bcd46e08c38c,c00bbae79ff276f31caf4db6cca23dff08b7bf1fb3d9c605603d4e4ed7837024
134
+ 1,memory,swap_T,swap_T,standard,swap_T--0022,22,original,true,1.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,d76d3b07df1264e2dfb78f0729bb33b309d0ad44be6c17dec7850f6f7074f3b8
135
+ 1,memory,swap_T,swap_T,standard,swap_T--0025,25,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,fd67e99c97110fe2cfb305d7554871d722bf38fc2c55f411e6f55f878b36dcb1
136
+ 1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0000,0,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,688343532fd457745c18daad9ff701a3ed5953fd35743683185a07dffeb3fdf0
137
+ 1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0005,5,original,true,1.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,8a5cb5653ef6db169d70460b0055767125f2e1a20d03f7c450598b12407a9c2b
138
+ 1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0022,22,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,a4b4274c8c2e2bcc1c9157f69b887908b2f11b1aafcc12483976987aa0402e50
139
+ 1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0029,29,original,false,0.0,,3,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,ca1b94e67fb3278f72cf49e297b538676b236c6583d62e15c39ad778ea566a94
140
+ 1,open,general_pickup,general_pickup,standard,general_pickup--0047,47,original,false,0.0,,2,92558f946ce17044692364bcb3515da3925bcf3c646266c8dc9ce1f38771384e,a82103e77e58e1c65aa304cde88ddf49b46ba7b7eaf29fb7a4dd22c7fa08dc95
141
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0034,34,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,24d4754dd80e3d493dca1d9802d1087cfdf7d9f258bfae73efc1ce3364cbaab9
142
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0037,37,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,0e37dc2edffa0731287627996d448343be9477c30b8fc1fcb66485dd9b5f095b
143
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0038,38,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,e62f48bbed2636c1d3c097d92411f7da359fc950a6835f3d9f03b75a32c1c768
144
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0039,39,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,4a1367c1b8b5b7bd9bf60b02efc74debe96482760463d91ac9934015ca43101f
145
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0040,40,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,80110a17e7abba14ed61a363bb80d5969a7e1833e96ebdc558bf1dbff0a3f006
146
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0041,41,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,7dd88174665764efd6997ca8ab21e94012e22867e57f932fcf65d707f3b0b989
147
+ 1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0042,42,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,37454569da3c33a7ac17da8a70c11032313a21eec68993e41cd898225b154118
148
+ 2,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0001,1,original,false,0.0,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,742a47472c7ebfdb6216f8b6822e8598753786ef0743cc567499c9a72f490161
149
+ 2,long_horizon,make_kong,make_kong,standard,make_kong--0039,39,original,true,1.0,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,d3acf54fdb5b2644c972f175c38f2d8f64ceddb0edd6cfe529249c6bdaa1cdf1
150
+ 2,long_horizon,make_kong,make_kong,standard,make_kong--0043,43,original,true,1.0,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,508b76a2e8b3c50015d0cba005c446bacd091aafeefc7ba723ec4d288dec1b6b
151
+ 2,long_horizon,play_stacking_toy,play_stacking_toy,standard,play_stacking_toy--0030,30,original,false,0.0,,2,e8a1d6c899cf43a21ba6a747be1317d102485e5effa9aa460ffb2258d3643d9f,d4951fe70c07365ec3819b12bc58a070075421b7fbb77a05901b8cd648ab62b2
152
+ 2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0007,7,original,false,0.5,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,bcfa324b282c5695b60188ac32fa58cf0545b683d91119a5181d627fb925ab4e
153
+ 2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0009,9,original,false,0.5,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,db195dd7b9fe78c238b36fe5db9510e6ec6cccb2ac739a56eb827666668fccf4
154
+ 2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0017,17,original,false,0.75,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,e48c3dc3fba1fc165ee836a869e3f275c3af2af5dc7e930bc5669578c9d9c003
155
+ 2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0027,27,original,true,1.0,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,2f7dde4e35e2f494ff6c6404c89159063b9b2136b3e6be33fe6a74a5eae30e2b
156
+ 2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0043,43,original,false,0.3,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,feabf3f67f18bbcbcdf0da8af6c02378aaaa1762382e04f28e79613401297f5d
157
+ 2,memory,cover_blocks,cover_blocks,standard,cover_blocks--0016,16,original,true,1.0,,2,c48429d0a143dea28dd311abd2e2b34f23140ef95abb554b100b851901427b87,3ac9ca3dfec6261ec5a21decd3bf294466ad300bac07566589cff551a7d02df6
158
+ 2,memory,swap_T,swap_T,standard,swap_T--0003,3,original,false,0.0,,2,c48429d0a143dea28dd311abd2e2b34f23140ef95abb554b100b851901427b87,8a53e0ce6050cf1662038fdae6dc0ff7a86ae375439ce7778505d260f149df7d
159
+ 2,open,align_blocks,align_blocks,standard,align_blocks--0001,1,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,9a3983149634c3de0f1957de9236db3ce44c2d60a86761dd0c1549b2722e44a2
160
+ 2,open,align_blocks,align_blocks,standard,align_blocks--0004,4,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f71dc35e0eecb79cc9a46c6a26674533bcda0a8df3f16479f154f1aeb1f8315a
161
+ 2,open,align_blocks,align_blocks,standard,align_blocks--0007,7,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,0673929de298c98cf4c246a694222178499c8b49e1fe0f8a87a2aeef7cdb0d1f
162
+ 2,open,align_blocks,align_blocks,standard,align_blocks--0013,13,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f75e751dbcc28079b1ec163b86fda589c02af44ecc4a832d93168050ef0cb579
163
+ 2,open,align_blocks,align_blocks,standard,align_blocks--0035,35,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,b672829c413aeaf482fcaa1a43a9e3af1a424c1a8401bbcd36212294eb9e57fa
164
+ 2,open,classify_objects_by_language,classify_objects_by_language,standard,classify_objects_by_language--0013,13,original,false,0.0,,2,13fc494c13228764cbed9fae7505e9eafd3e9a578fedd33399b130a583a00cf1,5800d929cd82ae95900db5ce812d2cb2e7dc61b40c13f2a5b2cce4a6354fb745
165
+ 2,open,general_pickup,general_pickup,standard,general_pickup--0008,8,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,89d72079bd22da9e7bebb5ceb80cd608322b9ac0bbc02c6536719b8d5743ea41
166
+ 2,open,solve_equation,solve_equation,standard,solve_equation--0036,36,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f15fb41e7f4e14ddd842811bad6821c8a5baee4ea53a7162d0a6decf3e010316
167
+ 2,open,solve_equation,solve_equation,standard,solve_equation--0044,44,original,false,0.0,,2,13fc494c13228764cbed9fae7505e9eafd3e9a578fedd33399b130a583a00cf1,44260033ab2274bd58b185f5b71e17d58955de7375a7067fa74adcdbe854a220
168
+ 2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0008,8,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,e08259f3647206e7499242d9365f4a3f8685f53fccfaa02630cf7119e44ca7a0
169
+ 2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0014,14,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,a931ad87cd18d27058bfeab723932ca2584680bb0043e02240a9939b4d034df4
170
+ 2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0028,28,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,dc28529548f4fb5438d8fa6527631b176baf40903d50c7c0d30517c5c86c3a18
171
+ 2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0047,47,original,false,0.2,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,8120914346c578e7a45c58777553d7dc226d042085f915eed9193f38f67f54aa
172
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0001,1,original,false,0.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,5ee2fbaa4a2a8e3b32edc82e36bdd9a2d6944952176da5e0a5690c1626170864
173
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0008,8,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,925d8def8e4e5943174327ff0e90a19ac57b2e5977d119dc7d4eaad5a11466bb
174
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0010,10,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,7ec138965910f52f1ffde2e117922ca3267c525c96fa6f92c4d98e28cd42c1fb
175
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0016,16,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,623c156da7194b08c9aec332afdcd408157428204dd6b9f0fa1f4c42c72e6fa3
176
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0022,22,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,3fb38b9a9dd693b17c2885404241f4165dcf3efceecfc6a8b81595978b5412f7
177
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0024,24,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,03e0e7dab66aab478d4b00425471f87cd03f5b9b7d17657c33e1d2a83d243ca8
178
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0036,36,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,85bdce295ce796aac48679df05cb4b1674e913a4eebcc17409e6883afe9fb0cf
179
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0037,37,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,44cd43cc30f3fa3554b66e644c36ea619c2a89345b15540ce4061100c7484c41
180
+ 2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0042,42,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,b32dfb7d293b019c42b86c9be45092cfd6d2b06bbd9fe76cbdecf19d0a777022
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RoboDojo Evaluation Summary
2
+
3
+ ## Overview (by capability dimension)
4
+
5
+ Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
6
+
7
+ | Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
8
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
9
+ | Bagel-Robotics-1_original_selection | 92.1% † (116/126) | 28.33 ±1.71 | 35.20 ±1.35 | 34.98 ±2.48 | 43.68 ±2.30 | 40.10 ±1.10 | 58.84 ±0.43 | 67.73 ±2.66 | 67.73 ±2.66 | 1.65 ±1.15 | 2.29 ±1.15 | 34.56 ±0.79 | 41.55 ±0.82 |
10
+
11
+ ## Generalization: Standard vs Random (25 episodes each)
12
+
13
+ Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
14
+
15
+ | Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
16
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
17
+ | Bagel-Robotics-1_original_selection | 45.67 ±2.68 | 51.77 ±2.32 | 11.00 ±2.23 | 18.63 ±2.18 | -34.67 | -33.14 |
18
+
19
+ ### Per-task Standard vs Random
20
+
21
+ Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
22
+
23
+ #### Bagel-Robotics-1_original_selection
24
+
25
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
26
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
27
+ | <nobr>stack_bowls</nobr> | 96 | 96.60 | 92 | 93.20 | 84 | 86.40 | 91 ±6 | 92.07 ±5.10 |
28
+ | <nobr>stack_bowls_random</nobr> | 0 | 14.40 | 12 | 24.60 | 8 | 21.80 | 7 ±6 | 20.27 ±5.10 |
29
+ | <nobr>push_T</nobr> | 0 | 0.00 | 4 | 4.00 | 4 | 4.00 | 3 ±2 | 2.67 ±2.00 |
30
+ | <nobr>push_T_random</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
31
+ | <nobr>pack_objects_into_box</nobr> | 0 | 29.40 | 16 | 42.60 | 8 | 34.60 | 8 ±8 | 35.53 ±6.60 |
32
+ | <nobr>pack_objects_into_box_random</nobr> | 4 | 23.60 | 8 | 28.20 | 4 | 25.20 | 5 ±2 | 25.67 ±2.30 |
33
+ | <nobr>fold_clothes</nobr> | 68 | 70.40 | 80 | 80.80 | 64 | 68.00 | 71 ±8 | 73.07 ±6.40 |
34
+ | <nobr>fold_clothes_random</nobr> | 64 | 65.60 | 64 | 68.00 | 84 | 85.60 | 71 ±10 | 73.07 ±10.00 |
35
+ | <nobr>hang_mugs</nobr> | 60 | 71.00 | 56 | 67.40 | 56 | 69.00 | 57 ±2 | 69.13 ±1.80 |
36
+ | <nobr>hang_mugs_random</nobr> | 0 | 11.80 | 0 | 9.60 | 0 | 12.20 | 0 ±0 | 11.20 ±1.30 |
37
+ | <nobr>sweep_blocks</nobr> | 16 | 16.00 | 8 | 8.00 | 8 | 8.00 | 11 ±4 | 10.67 ±4.00 |
38
+ | <nobr>sweep_blocks_random</nobr> | 4 | 4.00 | 0 | 0.00 | 8 | 8.00 | 4 ±4 | 4.00 ±4.00 |
39
+ | <nobr>pour_liquid_into_cup</nobr> | 52 | 52.00 | 68 | 68.00 | 64 | 64.00 | 61 ±8 | 61.33 ±8.00 |
40
+ | <nobr>pour_liquid_into_cup_random</nobr> | 8 | 8.00 | 20 | 20.00 | 28 | 28.00 | 19 ±10 | 18.67 ±10.00 |
41
+ | <nobr>make_toast</nobr> | 56 | 71.00 | 72 | 79.00 | 60 | 67.00 | 63 ±8 | 72.33 ±6.00 |
42
+ | <nobr>make_toast_random</nobr> | 8 | 24.00 | 16 | 26.00 | 8 | 23.00 | 11 ±4 | 24.33 ±1.50 |
43
+ | <nobr>arrange_largest_number</nobr> | 12 | 16.60 | 4 | 13.00 | 0 | 8.60 | 5 ±6 | 12.73 ±4.00 |
44
+ | <nobr>arrange_largest_number_random</nobr> | 0 | 4.60 | 4 | 9.00 | 0 | 6.60 | 1 ±2 | 6.73 ±2.20 |
45
+ | <nobr>sort_nesting_dolls_by_size</nobr> | 36 | 36.00 | 36 | 36.00 | 28 | 28.00 | 33 ±4 | 33.33 ±4.00 |
46
+ | <nobr>sort_nesting_dolls_by_size_random</nobr> | 4 | 4.00 | 4 | 4.00 | 0 | 0.00 | 3 ±2 | 2.67 ±2.00 |
47
+ | <nobr>store_laptop_and_headphones</nobr> | 68 | 78.40 | 68 | 76.80 | 48 | 60.80 | 61 ±10 | 72.00 ±8.80 |
48
+ | <nobr>store_laptop_and_headphones_random</nobr> | 4 | 20.00 | 12 | 25.60 | 20 | 33.60 | 12 ±8 | 26.40 ±6.80 |
49
+ | <nobr>stack_blocks</nobr> | 72 | 76.20 | 88 | 89.80 | 92 | 93.20 | 84 ±10 | 86.40 ±8.50 |
50
+ | <nobr>stack_blocks_random</nobr> | 0 | 10.80 | 0 | 10.20 | 0 | 10.80 | 0 ±0 | 10.60 ±0.30 |
51
+ | **Avg Standard** | 44.7 | 51.13 | 49.3 | 54.88 | 43.0 | 49.30 | 46 ±3 | 51.77 ±2.79 |
52
+ | **Avg Random** | 8.0 | 15.90 | 11.7 | 18.77 | 13.3 | 21.23 | 11 ±3 | 18.63 ±2.67 |
53
+
54
+ ## Bagel-Robotics-1_original_selection
55
+
56
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
57
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
58
+ | <nobr>align_blocks</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
59
+ | <nobr>arrange_largest_number</nobr> | 6 | 10.60 | 4 | 11.00 | 0 | 7.60 | 3 ±3 | 9.73 ±1.70 |
60
+ | <nobr>build_tower</nobr> | 80 | 85.60 | 64 | 73.60 | 68 | 75.40 | 71 ±8 | 78.20 ±6.00 |
61
+ | <nobr>classify_objects</nobr> | 46 | 56.70 | 32 | 44.50 | 42 | 51.10 | 40 ±7 | 50.77 ±6.10 |
62
+ | <nobr>classify_objects_by_language</nobr> | 0 | 3.60 | 0 | 2.60 | 6 | 9.20 | 2 ±3 | 5.13 ±3.30 |
63
+ | <nobr>cover_blocks</nobr> | 100 | 100.00 | 100 | 100.00 | 100 | 100.00 | 100 ±0 | 100.00 ±0.00 |
64
+ | <nobr>deposit_coin</nobr> | | | 22 | 26.80 | | | | |
65
+ | <nobr>fasten_screws</nobr> | 48 | 72.20 | 36 | 66.80 | 40 | 68.20 | 41 ±6 | 69.07 ±2.70 |
66
+ | <nobr>fill_egg_holder</nobr> | | | | | | | | |
67
+ | <nobr>fill_pen_holder</nobr> | 36 | 53.90 | 22 | 43.90 | 30 | 52.40 | 29 ±7 | 50.07 ±5.00 |
68
+ | <nobr>fold_clothes</nobr> | 66 | 68.00 | 72 | 74.40 | 74 | 76.80 | 71 ±4 | 73.07 ±4.40 |
69
+ | <nobr>general_pickup</nobr> | | | | | 16 | 16.00 | | |
70
+ | <nobr>hang_mugs</nobr> | 30 | 41.40 | 28 | 38.50 | 28 | 40.60 | 29 ±1 | 40.17 ±1.45 |
71
+ | <nobr>imitate_sorting_sequence</nobr> | | | | | | | | |
72
+ | <nobr>insert_key</nobr> | 0 | 15.00 | 0 | 15.00 | 0 | 15.00 | 0 ±0 | 15.00 ±0.00 |
73
+ | <nobr>insert_tubes</nobr> | 72 | 82.80 | 64 | 75.20 | 60 | 74.00 | 65 ±6 | 77.33 ±4.40 |
74
+ | <nobr>make_kong</nobr> | 84 | 84.00 | 86 | 86.00 | 88 | 88.00 | 86 ±2 | 86.00 ±2.00 |
75
+ | <nobr>make_toast</nobr> | 32 | 47.50 | 44 | 52.50 | 34 | 45.00 | 37 ±6 | 48.33 ±3.75 |
76
+ | <nobr>match_and_pick_from_conveyor</nobr> | 82 | 82.00 | 92 | 92.00 | 86 | 86.00 | 87 ±5 | 86.67 ±5.00 |
77
+ | <nobr>organize_table</nobr> | 14 | 57.50 | 18 | 64.50 | 12 | 58.00 | 15 ±3 | 60.00 ±3.50 |
78
+ | <nobr>pack_objects_into_box</nobr> | 2 | 26.50 | 12 | 35.40 | 6 | 29.90 | 7 ±5 | 30.60 ±4.45 |
79
+ | <nobr>pick_from_conveyor_by_image</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
80
+ | <nobr>play_Xylophone</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
81
+ | <nobr>play_stacking_toy</nobr> | 14 | 31.80 | 12 | 32.00 | 8 | 27.20 | 11 ±3 | 30.33 ±2.40 |
82
+ | <nobr>play_tic_tac_toe</nobr> | 0 | 35.80 | 2 | 40.50 | 6 | 33.90 | 3 ±3 | 36.73 ±3.30 |
83
+ | <nobr>plug_in_charger</nobr> | 12 | 12.00 | 4 | 4.00 | 8 | 8.00 | 8 ±4 | 8.00 ±4.00 |
84
+ | <nobr>pour_balls_into_vase</nobr> | 48 | 48.00 | 62 | 62.00 | 78 | 78.00 | 63 ±15 | 62.67 ±15.00 |
85
+ | <nobr>pour_by_language</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
86
+ | <nobr>pour_liquid_into_cup</nobr> | 30 | 30.00 | 44 | 44.00 | 46 | 46.00 | 40 ±8 | 40.00 ±8.00 |
87
+ | <nobr>press_by_number</nobr> | 98 | 98.00 | 100 | 100.00 | 100 | 100.00 | 99 ±1 | 99.33 ±1.00 |
88
+ | <nobr>push_T</nobr> | 0 | 0.00 | 2 | 2.00 | 2 | 2.00 | 1 ±1 | 1.33 ±1.00 |
89
+ | <nobr>put_bottles_into_dustbin</nobr> | 94 | 96.40 | 98 | 98.80 | 98 | 98.80 | 97 ±2 | 98.00 ±1.20 |
90
+ | <nobr>solve_equation</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
91
+ | <nobr>sort_nesting_dolls_by_size</nobr> | 20 | 20.00 | 20 | 20.00 | 14 | 14.00 | 18 ±3 | 18.00 ±3.00 |
92
+ | <nobr>stack_blocks</nobr> | 36 | 43.50 | 44 | 50.00 | 46 | 52.00 | 42 ±5 | 48.50 ±4.25 |
93
+ | <nobr>stack_blocks_by_language</nobr> | 4 | 4.80 | 8 | 9.20 | 4 | 6.00 | 5 ±2 | 6.67 ±2.20 |
94
+ | <nobr>stack_bowls</nobr> | 48 | 55.50 | 52 | 58.90 | 46 | 54.10 | 49 ±3 | 56.17 ±2.40 |
95
+ | <nobr>store_laptop_and_headphones</nobr> | 36 | 49.20 | 40 | 51.20 | 34 | 47.20 | 37 ±3 | 49.20 ±2.00 |
96
+ | <nobr>store_tools_in_toolbox</nobr> | 0 | 0.50 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.17 ±0.25 |
97
+ | <nobr>swap_T</nobr> | 22 | 22.00 | 22 | 22.00 | 32 | 32.00 | 25 ±5 | 25.33 ±5.00 |
98
+ | <nobr>swap_blocks</nobr> | 18 | 18.00 | 32 | 32.00 | 32 | 32.00 | 27 ±7 | 27.33 ±7.00 |
99
+ | <nobr>sweep_blocks</nobr> | 10 | 10.00 | 4 | 4.00 | 8 | 8.00 | 7 ±3 | 7.33 ±3.00 |
100
+ | **Avg (per seed)** | 31.3 | 38.49 | 31.8 | 39.32 | 32.1 | 39.29 | 32 ±0 | 39.03 ±0.41 |
101
+
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # RoboDojo Evaluation Summary
2
+
3
+ ## Overview (by capability dimension)
4
+
5
+ Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
6
+
7
+ | Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
8
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
9
+ | Bagel-Robotics-1_with_labeled_replacements_and_repair | 100% | 28.33 ±1.71 | 35.20 ±1.35 | 34.08 ±1.90 | 42.43 ±1.42 | 35.33 ±0.82 | 53.03 ±0.28 | 56.56 ±2.06 | 56.89 ±2.17 | 3.67 ±0.31 | 4.25 ±0.25 | 31.59 ±0.33 | 38.36 ±0.46 |
10
+
11
+ ## Generalization: Standard vs Random (25 episodes each)
12
+
13
+ Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
14
+
15
+ | Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
16
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
17
+ | Bagel-Robotics-1_with_labeled_replacements_and_repair | 45.67 ±2.68 | 51.77 ±2.32 | 11.00 ±2.23 | 18.63 ±2.18 | -34.67 | -33.14 |
18
+
19
+ ### Per-task Standard vs Random
20
+
21
+ Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
22
+
23
+ #### Bagel-Robotics-1_with_labeled_replacements_and_repair
24
+
25
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
26
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
27
+ | <nobr>stack_bowls</nobr> | 96 | 96.60 | 92 | 93.20 | 84 | 86.40 | 91 ±6 | 92.07 ±5.10 |
28
+ | <nobr>stack_bowls_random</nobr> | 0 | 14.40 | 12 | 24.60 | 8 | 21.80 | 7 ±6 | 20.27 ±5.10 |
29
+ | <nobr>push_T</nobr> | 0 | 0.00 | 4 | 4.00 | 4 | 4.00 | 3 ±2 | 2.67 ±2.00 |
30
+ | <nobr>push_T_random</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
31
+ | <nobr>pack_objects_into_box</nobr> | 0 | 29.40 | 16 | 42.60 | 8 | 34.60 | 8 ±8 | 35.53 ±6.60 |
32
+ | <nobr>pack_objects_into_box_random</nobr> | 4 | 23.60 | 8 | 28.20 | 4 | 25.20 | 5 ±2 | 25.67 ±2.30 |
33
+ | <nobr>fold_clothes</nobr> | 68 | 70.40 | 80 | 80.80 | 64 | 68.00 | 71 ±8 | 73.07 ±6.40 |
34
+ | <nobr>fold_clothes_random</nobr> | 64 | 65.60 | 64 | 68.00 | 84 | 85.60 | 71 ±10 | 73.07 ±10.00 |
35
+ | <nobr>hang_mugs</nobr> | 60 | 71.00 | 56 | 67.40 | 56 | 69.00 | 57 ±2 | 69.13 ±1.80 |
36
+ | <nobr>hang_mugs_random</nobr> | 0 | 11.80 | 0 | 9.60 | 0 | 12.20 | 0 ±0 | 11.20 ±1.30 |
37
+ | <nobr>sweep_blocks</nobr> | 16 | 16.00 | 8 | 8.00 | 8 | 8.00 | 11 ±4 | 10.67 ±4.00 |
38
+ | <nobr>sweep_blocks_random</nobr> | 4 | 4.00 | 0 | 0.00 | 8 | 8.00 | 4 ±4 | 4.00 ±4.00 |
39
+ | <nobr>pour_liquid_into_cup</nobr> | 52 | 52.00 | 68 | 68.00 | 64 | 64.00 | 61 ±8 | 61.33 ±8.00 |
40
+ | <nobr>pour_liquid_into_cup_random</nobr> | 8 | 8.00 | 20 | 20.00 | 28 | 28.00 | 19 ±10 | 18.67 ±10.00 |
41
+ | <nobr>make_toast</nobr> | 56 | 71.00 | 72 | 79.00 | 60 | 67.00 | 63 ±8 | 72.33 ±6.00 |
42
+ | <nobr>make_toast_random</nobr> | 8 | 24.00 | 16 | 26.00 | 8 | 23.00 | 11 ±4 | 24.33 ±1.50 |
43
+ | <nobr>arrange_largest_number</nobr> | 12 | 16.60 | 4 | 13.00 | 0 | 8.60 | 5 ±6 | 12.73 ±4.00 |
44
+ | <nobr>arrange_largest_number_random</nobr> | 0 | 4.60 | 4 | 9.00 | 0 | 6.60 | 1 ±2 | 6.73 ±2.20 |
45
+ | <nobr>sort_nesting_dolls_by_size</nobr> | 36 | 36.00 | 36 | 36.00 | 28 | 28.00 | 33 ±4 | 33.33 ±4.00 |
46
+ | <nobr>sort_nesting_dolls_by_size_random</nobr> | 4 | 4.00 | 4 | 4.00 | 0 | 0.00 | 3 ±2 | 2.67 ±2.00 |
47
+ | <nobr>store_laptop_and_headphones</nobr> | 68 | 78.40 | 68 | 76.80 | 48 | 60.80 | 61 ±10 | 72.00 ±8.80 |
48
+ | <nobr>store_laptop_and_headphones_random</nobr> | 4 | 20.00 | 12 | 25.60 | 20 | 33.60 | 12 ±8 | 26.40 ±6.80 |
49
+ | <nobr>stack_blocks</nobr> | 72 | 76.20 | 88 | 89.80 | 92 | 93.20 | 84 ±10 | 86.40 ±8.50 |
50
+ | <nobr>stack_blocks_random</nobr> | 0 | 10.80 | 0 | 10.20 | 0 | 10.80 | 0 ±0 | 10.60 ±0.30 |
51
+ | **Avg Standard** | 44.7 | 51.13 | 49.3 | 54.88 | 43.0 | 49.30 | 46 ±3 | 51.77 ±2.79 |
52
+ | **Avg Random** | 8.0 | 15.90 | 11.7 | 18.77 | 13.3 | 21.23 | 11 ±3 | 18.63 ±2.67 |
53
+
54
+ ## Bagel-Robotics-1_with_labeled_replacements_and_repair
55
+
56
+ | Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
57
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
58
+ | <nobr>align_blocks</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
59
+ | <nobr>arrange_largest_number</nobr> | 6 | 10.60 | 4 | 11.00 | 0 | 7.60 | 3 ±3 | 9.73 ±1.70 |
60
+ | <nobr>build_tower</nobr> | 80 | 85.60 | 64 | 73.60 | 68 | 75.40 | 71 ±8 | 78.20 ±6.00 |
61
+ | <nobr>classify_objects</nobr> | 46 | 56.70 | 32 | 44.50 | 42 | 51.10 | 40 ±7 | 50.77 ±6.10 |
62
+ | <nobr>classify_objects_by_language</nobr> | 0 | 3.60 | 0 | 2.60 | 6 | 9.20 | 2 ±3 | 5.13 ±3.30 |
63
+ | <nobr>cover_blocks</nobr> | 100 | 100.00 | 100 | 100.00 | 100 | 100.00 | 100 ±0 | 100.00 ±0.00 |
64
+ | <nobr>deposit_coin</nobr> | 28 | 32.40 | 22 | 26.80 | 24 | 28.40 | 25 ±3 | 29.20 ±2.80 |
65
+ | <nobr>fasten_screws</nobr> | 48 | 72.20 | 36 | 66.80 | 40 | 68.20 | 41 ±6 | 69.07 ±2.70 |
66
+ | <nobr>fill_egg_holder</nobr> | 2 | 11.00 | 4 | 13.80 | 0 | 12.20 | 2 ±2 | 12.33 ±1.40 |
67
+ | <nobr>fill_pen_holder</nobr> | 36 | 53.90 | 22 | 43.90 | 30 | 52.40 | 29 ±7 | 50.07 ±5.00 |
68
+ | <nobr>fold_clothes</nobr> | 66 | 68.00 | 72 | 74.40 | 74 | 76.80 | 71 ±4 | 73.07 ±4.40 |
69
+ | <nobr>general_pickup</nobr> | 26 | 26.00 | 24 | 24.00 | 16 | 16.00 | 22 ±5 | 22.00 ±5.00 |
70
+ | <nobr>hang_mugs</nobr> | 30 | 41.40 | 28 | 38.50 | 28 | 40.60 | 29 ±1 | 40.17 ±1.45 |
71
+ | <nobr>imitate_sorting_sequence</nobr> | 2 | 3.20 | 0 | 2.00 | 0 | 2.90 | 1 ±1 | 2.70 ±0.60 |
72
+ | <nobr>insert_key</nobr> | 0 | 15.00 | 0 | 15.00 | 0 | 15.00 | 0 ±0 | 15.00 ±0.00 |
73
+ | <nobr>insert_tubes</nobr> | 72 | 82.80 | 64 | 75.20 | 60 | 74.00 | 65 ±6 | 77.33 ±4.40 |
74
+ | <nobr>make_kong</nobr> | 84 | 84.00 | 86 | 86.00 | 88 | 88.00 | 86 ±2 | 86.00 ±2.00 |
75
+ | <nobr>make_toast</nobr> | 32 | 47.50 | 44 | 52.50 | 34 | 45.00 | 37 ±6 | 48.33 ±3.75 |
76
+ | <nobr>match_and_pick_from_conveyor</nobr> | 82 | 82.00 | 92 | 92.00 | 86 | 86.00 | 87 ±5 | 86.67 ±5.00 |
77
+ | <nobr>organize_table</nobr> | 14 | 57.50 | 18 | 64.50 | 12 | 58.00 | 15 ±3 | 60.00 ±3.50 |
78
+ | <nobr>pack_objects_into_box</nobr> | 2 | 26.50 | 12 | 35.40 | 6 | 29.90 | 7 ±5 | 30.60 ±4.45 |
79
+ | <nobr>pick_from_conveyor_by_image</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
80
+ | <nobr>play_Xylophone</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
81
+ | <nobr>play_stacking_toy</nobr> | 14 | 31.80 | 12 | 32.00 | 8 | 27.20 | 11 ±3 | 30.33 ±2.40 |
82
+ | <nobr>play_tic_tac_toe</nobr> | 0 | 35.80 | 2 | 40.50 | 6 | 33.90 | 3 ±3 | 36.73 ±3.30 |
83
+ | <nobr>plug_in_charger</nobr> | 12 | 12.00 | 4 | 4.00 | 8 | 8.00 | 8 ±4 | 8.00 ±4.00 |
84
+ | <nobr>pour_balls_into_vase</nobr> | 48 | 48.00 | 62 | 62.00 | 78 | 78.00 | 63 ±15 | 62.67 ±15.00 |
85
+ | <nobr>pour_by_language</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
86
+ | <nobr>pour_liquid_into_cup</nobr> | 30 | 30.00 | 44 | 44.00 | 46 | 46.00 | 40 ±8 | 40.00 ±8.00 |
87
+ | <nobr>press_by_number</nobr> | 98 | 98.00 | 100 | 100.00 | 100 | 100.00 | 99 ±1 | 99.33 ±1.00 |
88
+ | <nobr>push_T</nobr> | 0 | 0.00 | 2 | 2.00 | 2 | 2.00 | 1 ±1 | 1.33 ±1.00 |
89
+ | <nobr>put_bottles_into_dustbin</nobr> | 94 | 96.40 | 98 | 98.80 | 98 | 98.80 | 97 ±2 | 98.00 ±1.20 |
90
+ | <nobr>solve_equation</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
91
+ | <nobr>sort_nesting_dolls_by_size</nobr> | 20 | 20.00 | 20 | 20.00 | 14 | 14.00 | 18 ±3 | 18.00 ±3.00 |
92
+ | <nobr>stack_blocks</nobr> | 36 | 43.50 | 44 | 50.00 | 46 | 52.00 | 42 ±5 | 48.50 ±4.25 |
93
+ | <nobr>stack_blocks_by_language</nobr> | 4 | 4.80 | 8 | 9.20 | 4 | 6.00 | 5 ±2 | 6.67 ±2.20 |
94
+ | <nobr>stack_bowls</nobr> | 48 | 55.50 | 52 | 58.90 | 46 | 54.10 | 49 ±3 | 56.17 ±2.40 |
95
+ | <nobr>store_laptop_and_headphones</nobr> | 36 | 49.20 | 40 | 51.20 | 34 | 47.20 | 37 ±3 | 49.20 ±2.00 |
96
+ | <nobr>store_tools_in_toolbox</nobr> | 0 | 0.50 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.17 ±0.25 |
97
+ | <nobr>swap_T</nobr> | 22 | 22.00 | 22 | 22.00 | 32 | 32.00 | 25 ±5 | 25.33 ±5.00 |
98
+ | <nobr>swap_blocks</nobr> | 18 | 18.00 | 32 | 32.00 | 32 | 32.00 | 27 ±7 | 27.33 ±7.00 |
99
+ | <nobr>sweep_blocks</nobr> | 10 | 10.00 | 4 | 4.00 | 8 | 8.00 | 7 ±3 | 7.33 ±3.00 |
100
+ | **Avg (per seed)** | 29.7 | 36.56 | 30.2 | 37.45 | 30.4 | 37.52 | 30 ±0 | 37.18 ±0.48 |
101
+
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ seed,suite,task,episode,layout_id,record_type,attempt_count
2
+ 0,long_horizon,fill_egg_holder,fill_egg_holder--0028,28,original,4
3
+ 0,long_horizon,fill_egg_holder,fill_egg_holder--0035,35,original,4
4
+ 0,long_horizon,fill_egg_holder,fill_egg_holder--0037,37,original,4
5
+ 1,memory,imitate_sorting_sequence,imitate_sorting_sequence--0047,47,original,2
6
+ 2,long_horizon,fill_egg_holder,fill_egg_holder--0013,13,original,4
7
+ 2,long_horizon,fill_egg_holder,fill_egg_holder--0018,18,original,4
8
+ 2,long_horizon,fill_egg_holder,fill_egg_holder--0040,40,original,3
9
+ 2,long_horizon,fill_egg_holder,fill_egg_holder--0051,51,reserve_candidate_failed,1
10
+ 0,precision,deposit_coin,deposit_coin--0031,31,original,4
11
+ 0,precision,deposit_coin,deposit_coin--0036,36,original,4
12
+ 0,precision,deposit_coin,deposit_coin--0037,37,original,4
13
+ 0,precision,deposit_coin,deposit_coin--0041,41,original,4
14
+ 0,precision,deposit_coin,deposit_coin--0043,43,original,4
15
+ 0,precision,deposit_coin,deposit_coin--0050,50,reserve_candidate_failed,3
16
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0004,4,original,3
17
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0008,8,original,3
18
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0017,17,original,3
19
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0029,29,original,3
20
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0036,36,original,3
21
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0048,48,original,3
22
+ 0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0049,49,original,3
23
+ 0,open,general_pickup,general_pickup--0029,29,original,3
24
+ 2,precision,deposit_coin,deposit_coin--0024,24,original,4
25
+ 2,precision,deposit_coin,deposit_coin--0033,33,original,4
26
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0016,16,original,3
27
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0024,24,original,3
28
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0032,32,original,3
29
+ 2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0043,43,original,3
30
+ 1,long_horizon,fill_egg_holder,fill_egg_holder--0015,15,original,
31
+ 1,long_horizon,fill_egg_holder,fill_egg_holder--0020,20,original,
32
+ 1,open,general_pickup,general_pickup--0005,5,original,
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json ADDED
@@ -0,0 +1,213 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model": "Bagel-Robotics-1",
3
+ "checkpoint": 1000,
4
+ "huggingface_repository": "bageldotcom/Bagel-Robotics-1",
5
+ "huggingface_revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
6
+ "seeds": [
7
+ 0,
8
+ 1,
9
+ 2
10
+ ],
11
+ "reported_outcomes": 6300,
12
+ "original_selection_outcomes": 6271,
13
+ "later_replacement_outcomes": 28,
14
+ "repaired_variants": 1,
15
+ "preexisting_seed1_precision_selected_substitutions": 4,
16
+ "unscored_originals": 29,
17
+ "failed_replacement_candidates": 2,
18
+ "checkpoint_hashes": {
19
+ "action.pt": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
20
+ "video.pt": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
21
+ "compact_architecture.json": "a7f6696ab9a9beca785531c2402b81ccd1550745cb6e9ccaad67f658630f6b32",
22
+ "dataset_stats.json": "2649d4e3ff3d6d400c9953b7c20c79541474f9c38a8a9916b9caacecd8578f16"
23
+ },
24
+ "source_verification": {
25
+ "primary_archive_manifest_sha256": "b825a5418e7746e3e44d6c25b1dbbeb891353542a7b2f32542064cec850e612f",
26
+ "primary_archive_files_verified": 41025,
27
+ "primary_native_receipts_checked": 3150,
28
+ "seed1_overlap_verified_and_deduplicated": 750,
29
+ "normalized_episode_source_sha256": "b12ed6dd73e4525b3b857431bfb167de1464488164635fb8870a286fc110c4cb",
30
+ "previous_report_verification_sha256": "a438594b0de99cf5300a251d76c8aad50893faf47136e44391bf3163266d6063",
31
+ "complementary_node2_export_sha256": "9cc1fff72c51a69fcbe04d28164187f92d67311648acd5aa3788f1aacc5af409",
32
+ "campaign_protocols_verified": 34
33
+ },
34
+ "summary_commit": "f57e1633ffed59761bc8e568cfe7530c4b33381c",
35
+ "summary_variation": "population standard deviation in overview; half-range in native per-task tables",
36
+ "benchmark_base_commit": "08b7ee46034c3d0c8a389b2b7bfc138f4d55ee4d",
37
+ "local_runtime_patches": true,
38
+ "reported_suite_scores": [
39
+ {
40
+ "seed": 0,
41
+ "suite": "precision",
42
+ "n": 400,
43
+ "success": 144,
44
+ "fail": 256,
45
+ "sr": 36.0,
46
+ "score": 43.5
47
+ },
48
+ {
49
+ "seed": 0,
50
+ "suite": "memory",
51
+ "n": 300,
52
+ "success": 161,
53
+ "fail": 139,
54
+ "sr": 53.666666666666664,
55
+ "score": 53.86666666666667
56
+ },
57
+ {
58
+ "seed": 0,
59
+ "suite": "generalization",
60
+ "n": 600,
61
+ "success": 158,
62
+ "fail": 442,
63
+ "sr": 26.333333333333332,
64
+ "score": 33.516666666666666
65
+ },
66
+ {
67
+ "seed": 0,
68
+ "suite": "long_horizon",
69
+ "n": 400,
70
+ "success": 145,
71
+ "fail": 255,
72
+ "sr": 36.25,
73
+ "score": 53.3875
74
+ },
75
+ {
76
+ "seed": 0,
77
+ "suite": "open",
78
+ "n": 400,
79
+ "success": 15,
80
+ "fail": 385,
81
+ "sr": 3.75,
82
+ "score": 4.3625
83
+ },
84
+ {
85
+ "seed": 1,
86
+ "suite": "precision",
87
+ "n": 400,
88
+ "success": 126,
89
+ "fail": 274,
90
+ "sr": 31.5,
91
+ "score": 40.425
92
+ },
93
+ {
94
+ "seed": 1,
95
+ "suite": "memory",
96
+ "n": 300,
97
+ "success": 173,
98
+ "fail": 127,
99
+ "sr": 57.666666666666664,
100
+ "score": 58.0
101
+ },
102
+ {
103
+ "seed": 1,
104
+ "suite": "generalization",
105
+ "n": 600,
106
+ "success": 183,
107
+ "fail": 417,
108
+ "sr": 30.5,
109
+ "score": 36.825
110
+ },
111
+ {
112
+ "seed": 1,
113
+ "suite": "long_horizon",
114
+ "n": 400,
115
+ "success": 137,
116
+ "fail": 263,
117
+ "sr": 34.25,
118
+ "score": 53.0
119
+ },
120
+ {
121
+ "seed": 1,
122
+ "suite": "open",
123
+ "n": 400,
124
+ "success": 16,
125
+ "fail": 384,
126
+ "sr": 4.0,
127
+ "score": 4.475
128
+ },
129
+ {
130
+ "seed": 2,
131
+ "suite": "precision",
132
+ "n": 400,
133
+ "success": 139,
134
+ "fail": 261,
135
+ "sr": 34.75,
136
+ "score": 43.375
137
+ },
138
+ {
139
+ "seed": 2,
140
+ "suite": "memory",
141
+ "n": 300,
142
+ "success": 175,
143
+ "fail": 125,
144
+ "sr": 58.333333333333336,
145
+ "score": 58.81666666666667
146
+ },
147
+ {
148
+ "seed": 2,
149
+ "suite": "generalization",
150
+ "n": 600,
151
+ "success": 169,
152
+ "fail": 431,
153
+ "sr": 28.166666666666668,
154
+ "score": 35.266666666666666
155
+ },
156
+ {
157
+ "seed": 2,
158
+ "suite": "long_horizon",
159
+ "n": 400,
160
+ "success": 142,
161
+ "fail": 258,
162
+ "sr": 35.5,
163
+ "score": 52.7
164
+ },
165
+ {
166
+ "seed": 2,
167
+ "suite": "open",
168
+ "n": 400,
169
+ "success": 13,
170
+ "fail": 387,
171
+ "sr": 3.25,
172
+ "score": 3.9
173
+ }
174
+ ],
175
+ "five_suite_scores_by_seed": [
176
+ {
177
+ "sr": 31.2,
178
+ "score": 37.72666666666667
179
+ },
180
+ {
181
+ "sr": 31.583333333333332,
182
+ "score": 38.545
183
+ },
184
+ {
185
+ "sr": 32.0,
186
+ "score": 38.81166666666667
187
+ }
188
+ ],
189
+ "three_seed_overall": {
190
+ "success_rate_mean": 31.594444444444445,
191
+ "success_rate_population_sd": 0.3266931206219816,
192
+ "partial_score_mean": 38.361111111111114,
193
+ "partial_score_population_sd": 0.461640258637678
194
+ },
195
+ "video_files_in_export": false,
196
+ "source_records_modified": false,
197
+ "public_export_verification": {
198
+ "episode_rows_checked_against_retained_source": 6300,
199
+ "successes_all_seeds": 1896,
200
+ "unique_scored_identities": 6300,
201
+ "complete_native_task_seed_cells_reported": 126,
202
+ "complete_native_task_seed_cells_original_only": 116,
203
+ "native_task_seed_values_independently_checked": 126,
204
+ "native_summary_byte_copy_verified": true,
205
+ "native_summary_overview_matches_independent_calculation": true,
206
+ "unchanged_seed1_files_verified": true,
207
+ "summary_sha256": "141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f",
208
+ "original_summary_sha256": "c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5",
209
+ "source_ledger_sha256": "b12ed6dd73e4525b3b857431bfb167de1464488164635fb8870a286fc110c4cb",
210
+ "native_summary_script_sha256": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2",
211
+ "native_launcher_sha256": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3"
212
+ }
213
+ }
inference/policy/Bagel_Robotics_1/evidence/pr-audit.json CHANGED
@@ -16,8 +16,11 @@
16
  "readiness_scope": "Opening a review PR; not maintainer acceptance or verified leaderboard publication",
17
  "remaining_gates": [
18
  "maintainer agreement on proposed eval-only integration and release timeline",
19
- "assigned evaluator and private credential handoff",
20
  "official multi-seed evaluation and leaderboard review",
21
- "public artifact release for verified leaderboard publication"
22
- ]
 
 
 
 
23
  }
 
16
  "readiness_scope": "Opening a review PR; not maintainer acceptance or verified leaderboard publication",
17
  "remaining_gates": [
18
  "maintainer agreement on proposed eval-only integration and release timeline",
 
19
  "official multi-seed evaluation and leaderboard review",
20
+ "scheduled technical report and training/pruning/export code release"
21
+ ],
22
+ "public_checkpoint_access": {
23
+ "passed": true,
24
+ "evidence": "public-checkpoint-access.json"
25
+ }
26
  }
inference/policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json ADDED
@@ -0,0 +1,187 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "checked_at_utc": "2026-11-07T14:51:23.788607+00:00",
3
+ "repo_id": "bageldotcom/Bagel-Robotics-1",
4
+ "private": false,
5
+ "gated": false,
6
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
7
+ "checkpoint_identity": "790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c",
8
+ "authentication": "No Authorization header for API/GET probes; token environment removed, isolated empty Hugging Face home/cache, and implicit token disabled for the published metadata downloader.",
9
+ "metadata_downloader": {
10
+ "passed": true,
11
+ "huggingface_hub": "0.36.0 (submission-pinned version)",
12
+ "fully_downloaded_and_sha256_verified": [
13
+ "COMPLETE.json",
14
+ "compact_architecture.json",
15
+ "dataset_stats.json",
16
+ "validation.json"
17
+ ]
18
+ },
19
+ "artifact_download_checks": [
20
+ {
21
+ "repo": "bageldotcom/Bagel-Robotics-1",
22
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
23
+ "file": "COMPLETE.json",
24
+ "size": 2491,
25
+ "sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
26
+ "remote_identity": "0afb4065d4e25d28c0ae3e6ca1bdae7c2cbd1ae5",
27
+ "anonymous_get_status": 206,
28
+ "bytes_read": 2491,
29
+ "content_range": "bytes 0-2490/2491"
30
+ },
31
+ {
32
+ "repo": "bageldotcom/Bagel-Robotics-1",
33
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
34
+ "file": "action.pt",
35
+ "size": 4057916577,
36
+ "sha256": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
37
+ "remote_identity": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
38
+ "anonymous_get_status": 206,
39
+ "bytes_read": 4096,
40
+ "content_range": "bytes 0-4095/4057916577"
41
+ },
42
+ {
43
+ "repo": "bageldotcom/Bagel-Robotics-1",
44
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
45
+ "file": "compact_architecture.json",
46
+ "size": 6563270,
47
+ "sha256": "a7f6696ab9a9beca785531c2402b81ccd1550745cb6e9ccaad67f658630f6b32",
48
+ "remote_identity": "d49d2e581a33844e824bac2396a6e08d8719a151",
49
+ "anonymous_get_status": 206,
50
+ "bytes_read": 4096,
51
+ "content_range": "bytes 0-4095/6563270"
52
+ },
53
+ {
54
+ "repo": "bageldotcom/Bagel-Robotics-1",
55
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
56
+ "file": "dataset_stats.json",
57
+ "size": 100256,
58
+ "sha256": "2649d4e3ff3d6d400c9953b7c20c79541474f9c38a8a9916b9caacecd8578f16",
59
+ "remote_identity": "53027727e6b22c1817fe861b6a91c434d8d8473d",
60
+ "anonymous_get_status": 206,
61
+ "bytes_read": 4096,
62
+ "content_range": "bytes 0-4095/100256"
63
+ },
64
+ {
65
+ "repo": "bageldotcom/Bagel-Robotics-1",
66
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
67
+ "file": "validation.json",
68
+ "size": 88781,
69
+ "sha256": "9806ad2b0b58f879351da58eede46507a79433ec9e4eb8f37f415cf58f2ef70e",
70
+ "remote_identity": "0b71b75d4730d4a4b9c3f856330fbf2e27fa8abc",
71
+ "anonymous_get_status": 206,
72
+ "bytes_read": 4096,
73
+ "content_range": "bytes 0-4095/88781"
74
+ },
75
+ {
76
+ "repo": "bageldotcom/Bagel-Robotics-1",
77
+ "revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
78
+ "file": "video.pt",
79
+ "size": 24937214042,
80
+ "sha256": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
81
+ "remote_identity": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
82
+ "anonymous_get_status": 206,
83
+ "bytes_read": 4096,
84
+ "content_range": "bytes 0-4095/24937214042"
85
+ },
86
+ {
87
+ "repo": "Haodong082399/VPP2",
88
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
89
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/LICENSE.txt",
90
+ "size": 11357,
91
+ "remote_identity": "261eeb9e9f8b2b4b0d119366dda99c6fd7d35c64",
92
+ "identity_algorithm": "git_blob_sha1",
93
+ "anonymous_get_status": 206,
94
+ "bytes_read": 4096,
95
+ "content_range": "bytes 0-4095/11357"
96
+ },
97
+ {
98
+ "repo": "Haodong082399/VPP2",
99
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
100
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/Wan2.1_VAE.pth",
101
+ "size": 507609880,
102
+ "remote_identity": "38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981",
103
+ "identity_algorithm": "sha256",
104
+ "anonymous_get_status": 206,
105
+ "bytes_read": 4096,
106
+ "content_range": "bytes 0-4095/507609880"
107
+ },
108
+ {
109
+ "repo": "Haodong082399/VPP2",
110
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
111
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/config.json",
112
+ "size": 250,
113
+ "remote_identity": "f5a12b285ec9c2497cafc4a4443fc6ba61b1a30b",
114
+ "identity_algorithm": "git_blob_sha1",
115
+ "anonymous_get_status": 206,
116
+ "bytes_read": 250,
117
+ "content_range": "bytes 0-249/250"
118
+ },
119
+ {
120
+ "repo": "Haodong082399/VPP2",
121
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
122
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/special_tokens_map.json",
123
+ "size": 6623,
124
+ "remote_identity": "14855e7052ffbb595057dfd791d293c1c940db2c",
125
+ "identity_algorithm": "git_blob_sha1",
126
+ "anonymous_get_status": 206,
127
+ "bytes_read": 4096,
128
+ "content_range": "bytes 0-4095/6623"
129
+ },
130
+ {
131
+ "repo": "Haodong082399/VPP2",
132
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
133
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/spiece.model",
134
+ "size": 4548313,
135
+ "remote_identity": "e3909a67b780650b35cf529ac782ad2b6b26e6d1f849d3fbb6a872905f452458",
136
+ "identity_algorithm": "sha256",
137
+ "anonymous_get_status": 206,
138
+ "bytes_read": 4096,
139
+ "content_range": "bytes 0-4095/4548313"
140
+ },
141
+ {
142
+ "repo": "Haodong082399/VPP2",
143
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
144
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/tokenizer.json",
145
+ "size": 16837417,
146
+ "remote_identity": "6e197b4d3dbd71da14b4eb255f4fa91c9c1f2068b20a2de2472967ca3d22602b",
147
+ "identity_algorithm": "sha256",
148
+ "anonymous_get_status": 206,
149
+ "bytes_read": 4096,
150
+ "content_range": "bytes 0-4095/16837417"
151
+ },
152
+ {
153
+ "repo": "Haodong082399/VPP2",
154
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
155
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/tokenizer_config.json",
156
+ "size": 61728,
157
+ "remote_identity": "4e1cc1cd85599ce0b47fd0a746af188fe4043ff2",
158
+ "identity_algorithm": "git_blob_sha1",
159
+ "anonymous_get_status": 206,
160
+ "bytes_read": 4096,
161
+ "content_range": "bytes 0-4095/61728"
162
+ },
163
+ {
164
+ "repo": "Haodong082399/VPP2",
165
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
166
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth",
167
+ "size": 4772359047,
168
+ "remote_identity": "628c9998b613391f193eb67ff68da9667d75f492911e4eb3decf23460a158c38",
169
+ "identity_algorithm": "sha256",
170
+ "anonymous_get_status": 206,
171
+ "bytes_read": 4096,
172
+ "content_range": "bytes 0-4095/4772359047"
173
+ },
174
+ {
175
+ "repo": "Haodong082399/VPP2",
176
+ "revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
177
+ "file": "checkpoints/Wan2.1-I2V-14B-480P/models_t5_umt5-xxl-enc-bf16.pth",
178
+ "size": 11361920418,
179
+ "remote_identity": "7cace0da2b446bbbbc57d031ab6cf163a3d59b366da94e5afe36745b746fd81d",
180
+ "identity_algorithm": "sha256",
181
+ "anonymous_get_status": 206,
182
+ "bytes_read": 4096,
183
+ "content_range": "bytes 0-4095/11361920418"
184
+ }
185
+ ],
186
+ "scope": "Anonymous GET access and pinned remote identity/size checked for six checkpoint files and nine encoder/tokenizer assets. Four checkpoint JSON files downloaded fully and SHA-256 verified with the published downloader. Large tensors were range-probed, not downloaded and rehashed in full; earlier full tensor verification remains recorded in VALIDATION.md."
187
+ }
inference/policy/Bagel_Robotics_1/runtime/LICENSE CHANGED
@@ -22,3 +22,9 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
22
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24
  SOFTWARE.
 
 
 
 
 
 
 
22
  LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
23
  OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
24
  SOFTWARE.
25
+
26
+ Component attribution
27
+ VPP2/FastWAM source retains the MIT copyright and terms above.
28
+ Wan2.1/DiffSynth-derived components retain Apache-2.0 terms; see licenses/Apache-2.0.txt.
29
+ Source identities and packaging changes are recorded in the adapter's source-provenance.json.
30
+ Model weights and datasets retain their respective upstream terms.
inference/policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md DELETED
@@ -1,8 +0,0 @@
1
- # Runtime source notices
2
-
3
- Reused source components from [VPP2](https://github.com/roboterax/video-prediction-policy-2)/FastWAM retain the MIT notice in LICENSE.
4
- Wan2.1/DiffSynth-derived components retain Apache-2.0 terms, included in
5
- licenses/Apache-2.0.txt. Source attribution is retained in individual modules
6
- where present. Source hashes and packaging changes accompany the adapter in
7
- source-provenance.json. Model weights and datasets are separate artifacts with
8
- their own upstream terms; this code license does not relicense those artifacts.
 
 
 
 
 
 
 
 
 
inference/policy/Bagel_Robotics_1/runtime/pyproject.toml CHANGED
@@ -8,7 +8,7 @@ version = "0.1.0"
8
  description = "Isolated inference runtime for Bagel-Robotics-1 recovery checkpoint 1000"
9
  requires-python = ">=3.10"
10
  license = "MIT AND Apache-2.0"
11
- license-files = ["LICENSE", "THIRD_PARTY_NOTICES.md", "licenses/*.txt"]
12
  dependencies = [
13
  "torch==2.11.0", "torchvision==0.26.0", "numpy==1.26.4",
14
  "transformers==4.49.0", "huggingface-hub==0.36.0", "hydra-core==1.3.2",
 
8
  description = "Isolated inference runtime for Bagel-Robotics-1 recovery checkpoint 1000"
9
  requires-python = ">=3.10"
10
  license = "MIT AND Apache-2.0"
11
+ license-files = ["LICENSE", "licenses/*.txt"]
12
  dependencies = [
13
  "torch==2.11.0", "torchvision==0.26.0", "numpy==1.26.4",
14
  "transformers==4.49.0", "huggingface-hub==0.36.0", "hydra-core==1.3.2",
inference/policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py CHANGED
@@ -1,4 +1,4 @@
1
- # See LICENSE and THIRD_PARTY_NOTICES.md for retained source notices.
2
  from contextlib import contextmanager, nullcontext
3
  import os
4
  from pathlib import Path
 
1
+ # See LICENSE for retained source notices.
2
  from contextlib import contextmanager, nullcontext
3
  import os
4
  from pathlib import Path
inference/policy/Bagel_Robotics_1/source-provenance.json CHANGED
@@ -125,7 +125,7 @@
125
  {
126
  "source": "models/wan21_14b/vpp2.py",
127
  "source_sha256": "8c419af905a8ffa93d77c827a283b7aeff744f3a461c390783eef8a333c31756",
128
- "packaged_sha256": "41e8632ad7898bf3a63cf7c874179adae609583ceb1b4dadf2b4c22669865323"
129
  },
130
  {
131
  "source": "models/wan21_14b/wan21.py",
@@ -188,7 +188,8 @@
188
  "Recorded deterministic backend controls applied at startup",
189
  "Remove unreachable adapter branches for unsupported action spaces; preserve EE16 math",
190
  "Restore evaluated processor, image transform and action/state merger dependencies under the isolated namespace",
191
- "Recursively map serialized Hydra targets to bagel_cp1000; preserve tensor keys and all non-target configuration values"
 
192
  ],
193
  "adapter_files": [
194
  {
 
125
  {
126
  "source": "models/wan21_14b/vpp2.py",
127
  "source_sha256": "8c419af905a8ffa93d77c827a283b7aeff744f3a461c390783eef8a333c31756",
128
+ "packaged_sha256": "a1df6528f7f9536b2ff4ad3913fa718d40dfb1061dc35e9fa0e28e2e1fa3bf96"
129
  },
130
  {
131
  "source": "models/wan21_14b/wan21.py",
 
188
  "Recorded deterministic backend controls applied at startup",
189
  "Remove unreachable adapter branches for unsupported action spaces; preserve EE16 math",
190
  "Restore evaluated processor, image transform and action/state merger dependencies under the isolated namespace",
191
+ "Recursively map serialized Hydra targets to bagel_cp1000; preserve tensor keys and all non-target configuration values",
192
+ "Consolidate explanatory notices into LICENSE; update the license reference comment without changing executable code"
193
  ],
194
  "adapter_files": [
195
  {