docs: synchronize public release artifacts
Browse files- README.md +13 -21
- RELEASE.json +1 -1
- inference/README.md +4 -4
- inference/SHA256SUMS +31 -15
- inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md +355 -103
- inference/policy/Bagel_Robotics_1/LICENSE +6 -0
- inference/policy/Bagel_Robotics_1/README.md +6 -6
- inference/policy/Bagel_Robotics_1/REEVALUATION.md +12 -10
- inference/policy/Bagel_Robotics_1/THIRD_PARTY_NOTICES.md +0 -13
- inference/policy/Bagel_Robotics_1/VALIDATION.md +13 -3
- inference/policy/Bagel_Robotics_1/artifact-manifest.json +1 -1
- inference/policy/Bagel_Robotics_1/download_checkpoint.py +1 -1
- inference/policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md +102 -0
- inference/policy/Bagel_Robotics_1/evaluation/seed-1/summary.md +101 -0
- inference/policy/Bagel_Robotics_1/evaluation/seed-1/verification.json +21 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes +3 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md +5 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/SHA256SUMS +14 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv +0 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py +106 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md +5 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json +0 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv +30 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md +5 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md +5 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv +180 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md +101 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md +101 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv +32 -0
- inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json +213 -0
- inference/policy/Bagel_Robotics_1/evidence/pr-audit.json +6 -3
- inference/policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json +187 -0
- inference/policy/Bagel_Robotics_1/runtime/LICENSE +6 -0
- inference/policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md +0 -8
- inference/policy/Bagel_Robotics_1/runtime/pyproject.toml +1 -1
- inference/policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py +1 -1
- inference/policy/Bagel_Robotics_1/source-provenance.json +3 -2
README.md
CHANGED
|
@@ -2,14 +2,12 @@
|
|
| 2 |
language: en
|
| 3 |
tags:
|
| 4 |
- robotics
|
| 5 |
-
- vpp2
|
| 6 |
- structured-pruning
|
| 7 |
- robodojo
|
| 8 |
-
---
|
| 9 |
|
| 10 |
# Bagel-Robotics-1 — checkpoint 1000
|
| 11 |
|
| 12 |
-
Bagel-Robotics-1 combines
|
| 13 |
|
| 14 |
Inference release of the **exact checkpoint 1000** used in the Bagel-Robotics-1 evaluation campaign. **The repository is public and ungated.** Anyone can download the checkpoint and inference files without a Hugging Face account or access token.
|
| 15 |
|
|
@@ -19,15 +17,13 @@ The video expert has **12,468,409,664 parameters**, 23.95% fewer than its record
|
|
| 19 |
|
| 20 |
- Original checkpoint revision: `2651631c5d08819088475e8778676f5cf6d9afcf`.
|
| 21 |
- Identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
|
| 22 |
-
- [artifact-manifest.json](artifact-manifest.json) pins all six bundle files and the separate public encoder/tokenizer assets.
|
| 23 |
-
- [SHA256SUMS](SHA256SUMS) records the unchanged SHA-256 hashes of all six bundle files.
|
| 24 |
|
| 25 |
-
The release adds documentation and inference packaging without changing any checkpoint file. The original revision is preserved, so historical provenance remains accessible in Git history.
|
| 26 |
|
| 27 |
## Download the exact checkpoint
|
| 28 |
|
| 29 |
-
The checkpoint is publicly downloadable at the pinned revision below. No login or access token is required.
|
| 30 |
-
|
| 31 |
```bash
|
| 32 |
hf download bageldotcom/Bagel-Robotics-1 \
|
| 33 |
--revision 2651631c5d08819088475e8778676f5cf6d9afcf \
|
|
@@ -35,35 +31,31 @@ hf download bageldotcom/Bagel-Robotics-1 \
|
|
| 35 |
--local-dir /absolute/path/to/checkpoints/cp1000
|
| 36 |
```
|
| 37 |
|
| 38 |
-
There are approximately 29.0 GB of checkpoint tensors. Keep all six files together. The inference downloader checks their sizes and SHA-256 hashes and downloads approximately 16.7 GB of separately pinned VAE/conditioning/tokenizer assets.
|
| 39 |
|
| 40 |
## Inference adapter and reevaluation
|
| 41 |
|
| 42 |
-
The standalone XPolicyLab adapter is provided under [`inference/`](inference/)
|
| 43 |
|
| 44 |
```bash
|
| 45 |
hf download bageldotcom/Bagel-Robotics-1 \
|
| 46 |
-
--revision bagel-robotics-1-cp1000-inference-v2 \
|
| 47 |
--include 'inference/**' \
|
| 48 |
--local-dir /absolute/path/to/Bagel-Robotics-1-release
|
| 49 |
|
| 50 |
-
|
| 51 |
-
git -C /absolute/path/to/workspace/XPolicyLab checkout 6baccd57ff958570d0db1588370289ee36c8bb70
|
| 52 |
python /absolute/path/to/Bagel-Robotics-1-release/inference/install_adapter.py /absolute/path/to/workspace/XPolicyLab
|
| 53 |
```
|
| 54 |
|
| 55 |
-
Follow
|
| 56 |
-
|
| 57 |
-
The adapter uses `policy_name=Bagel_Robotics_1`, `ckpt_name=cp1000`, ARX X5 end-effector actions, and one environment per policy process. Fixed inference settings: BF16; predict 32 actions and execute 16; ten Euler steps; shift 1; 90% new / 10% previous temporal blending. Qualification used a 96 GB RTX PRO 6000 Blackwell, Python 3.10.21 and PyTorch 2.11.0+cu130; a lower minimum GPU capacity has not been qualified.
|
| 58 |
|
| 59 |
-
|
| 60 |
|
| 61 |
## Validation and release scope
|
| 62 |
|
| 63 |
-
The adapter
|
| 64 |
|
| 65 |
-
The
|
| 66 |
|
| 67 |
-
## Attribution
|
| 68 |
|
| 69 |
-
VPP2 is the base model used to validate the Bagel-Robotics-1 pruning, recovery-training and data-selection approach.
|
|
|
|
| 2 |
language: en
|
| 3 |
tags:
|
| 4 |
- robotics
|
|
|
|
| 5 |
- structured-pruning
|
| 6 |
- robodojo
|
|
|
|
| 7 |
|
| 8 |
# Bagel-Robotics-1 — checkpoint 1000
|
| 9 |
|
| 10 |
+
Bagel-Robotics-1 combines task-critical data selection, adaptive structured pruning and recovery training to produce a compact world-action model. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
|
| 11 |
|
| 12 |
Inference release of the **exact checkpoint 1000** used in the Bagel-Robotics-1 evaluation campaign. **The repository is public and ungated.** Anyone can download the checkpoint and inference files without a Hugging Face account or access token.
|
| 13 |
|
|
|
|
| 17 |
|
| 18 |
- Original checkpoint revision: `2651631c5d08819088475e8778676f5cf6d9afcf`.
|
| 19 |
- Identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
|
| 20 |
+
- [`artifact-manifest.json`](artifact-manifest.json) pins all six bundle files and the separate public encoder/tokenizer assets.
|
| 21 |
+
- [`SHA256SUMS`](SHA256SUMS) records the unchanged SHA-256 hashes of all six bundle files.
|
| 22 |
|
| 23 |
+
The release adds documentation and inference packaging without changing any checkpoint file. The original revision is preserved, so historical provenance remains accessible in Git history. `COMPLETE.json` is the original export receipt, retained unchanged for integrity and not used as a current evaluation-status claim.
|
| 24 |
|
| 25 |
## Download the exact checkpoint
|
| 26 |
|
|
|
|
|
|
|
| 27 |
```bash
|
| 28 |
hf download bageldotcom/Bagel-Robotics-1 \
|
| 29 |
--revision 2651631c5d08819088475e8778676f5cf6d9afcf \
|
|
|
|
| 31 |
--local-dir /absolute/path/to/checkpoints/cp1000
|
| 32 |
```
|
| 33 |
|
| 34 |
+
There are approximately 29.0 GB of checkpoint tensors. Keep all six files together. The inference downloader checks their sizes and SHA-256 hashes and downloads approximately 16.7 GB of separately pinned VAE/conditioning/tokenizer assets.
|
| 35 |
|
| 36 |
## Inference adapter and reevaluation
|
| 37 |
|
| 38 |
+
The standalone XPolicyLab adapter is provided under [`inference/`](inference/). The current public adapter follows the instructions in [`inference/README.md`](inference/README.md); no private research repository or access token is required.
|
| 39 |
|
| 40 |
```bash
|
| 41 |
hf download bageldotcom/Bagel-Robotics-1 \
|
|
|
|
| 42 |
--include 'inference/**' \
|
| 43 |
--local-dir /absolute/path/to/Bagel-Robotics-1-release
|
| 44 |
|
| 45 |
+
# Then install the adapter into an existing XPolicyLab checkout.
|
|
|
|
| 46 |
python /absolute/path/to/Bagel-Robotics-1-release/inference/install_adapter.py /absolute/path/to/workspace/XPolicyLab
|
| 47 |
```
|
| 48 |
|
| 49 |
+
Follow [`inference/policy/Bagel_Robotics_1/README.md`](inference/policy/Bagel_Robotics_1/README.md) to install the Python 3.10 environment, download and verify assets, and run both debug checks. Follow [`REEVALUATION.md`](inference/policy/Bagel_Robotics_1/REEVALUATION.md) for task, suite and three-seed commands. Install RoboDojo and simulator assets separately using its [official guide](https://robodojo-benchmark.com/doc/usage/install-and-download/).
|
|
|
|
|
|
|
| 50 |
|
| 51 |
+
The adapter uses `policy_name=Bagel_Robotics_1`, `ckpt_name=cp1000`, ARX X5 end-effector actions and one environment per policy process. Fixed inference settings: BF16; predict 32 actions and execute 16; ten Euler steps; shift 1; 90% new / 10% previous temporal blending.
|
| 52 |
|
| 53 |
## Validation and release scope
|
| 54 |
|
| 55 |
+
The adapter qualification includes 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes and eight exact action/history/reset/backend checks on tested recordings. Full multi-seed official leaderboard validation remains separate. Historical research scores and scene exceptions are documented in [`EVALUATION_RESULTS.md`](inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md); they are not official leaderboard results.
|
| 56 |
|
| 57 |
+
The training, pruning and export implementation is publicly available in the [Bagel-Robotics-1 GitHub repository](https://github.com/bageldotcom/Bagel-Robotics-1). The technical report remains planned for **November 20, 2026**. This Hugging Face repository contains the inference release and weights; it does not claim that optimizer state or exact training continuation can be reconstructed from weights alone.
|
| 58 |
|
| 59 |
+
## Attribution
|
| 60 |
|
| 61 |
+
VPP2 is acknowledged as the base model used to validate the Bagel-Robotics-1 pruning, recovery-training and data-selection approach. Applicable terms are retained in the repository license files.
|
RELEASE.json
CHANGED
|
@@ -30,7 +30,7 @@
|
|
| 30 |
"size": 24937214042
|
| 31 |
}
|
| 32 |
},
|
| 33 |
-
"scope": "Public, ungated inference checkpoint and supporting adapter; no login or access token required; training
|
| 34 |
"historical_provenance": "Original checkpoint revision and checksums preserved. Older commits retain original provenance and export records; the current release view contains inference materials.",
|
| 35 |
"official_leaderboard_result": false
|
| 36 |
}
|
|
|
|
| 30 |
"size": 24937214042
|
| 31 |
}
|
| 32 |
},
|
| 33 |
+
"scope": "Public, ungated inference checkpoint and supporting adapter; no login or access token required; training, pruning and export source are published separately in GitHub; technical report planned for November 20, 2026",
|
| 34 |
"historical_provenance": "Original checkpoint revision and checksums preserved. Older commits retain original provenance and export records; the current release view contains inference materials.",
|
| 35 |
"official_leaderboard_result": false
|
| 36 |
}
|
inference/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
# Bagel-Robotics-1 inference adapter
|
| 2 |
|
| 3 |
-
Bagel-Robotics-1 combines
|
| 4 |
|
| 5 |
-
This folder is the standalone XPolicyLab adapter for
|
| 6 |
|
| 7 |
Install this adapter into an existing XPolicyLab checkout (the installer refuses to overwrite an existing policy):
|
| 8 |
|
|
@@ -10,8 +10,8 @@ Install this adapter into an existing XPolicyLab checkout (the installer refuses
|
|
| 10 |
python install_adapter.py /absolute/path/to/XPolicyLab
|
| 11 |
```
|
| 12 |
|
| 13 |
-
Then follow [
|
| 14 |
|
| 15 |
-
The adapter is prepared against XPolicyLab `6baccd57ff958570d0db1588370289ee36c8bb70`; source and dependency pins are preserved in the policy folder. The
|
| 16 |
|
| 17 |
The repository is public and ungated. The pinned checkpoint and inference files can be downloaded without a Hugging Face account or access token.
|
|
|
|
| 1 |
# Bagel-Robotics-1 inference adapter
|
| 2 |
|
| 3 |
+
Bagel-Robotics-1 combines task-critical data selection, adaptive structured pruning and recovery training to produce a compact world-action model. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
|
| 4 |
|
| 5 |
+
This folder is the standalone XPolicyLab adapter for checkpoint 1000. It contains inference code, configuration and evaluation instructions. Model tensors download separately from [`bageldotcom/Bagel-Robotics-1`](https://huggingface.co/bageldotcom/Bagel-Robotics-1), pinned to `2651631c5d08819088475e8778676f5cf6d9afcf` and verified against `policy/Bagel_Robotics_1/artifact-manifest.json`.
|
| 6 |
|
| 7 |
Install this adapter into an existing XPolicyLab checkout (the installer refuses to overwrite an existing policy):
|
| 8 |
|
|
|
|
| 10 |
python install_adapter.py /absolute/path/to/XPolicyLab
|
| 11 |
```
|
| 12 |
|
| 13 |
+
Then follow [`policy/Bagel_Robotics_1/README.md`](policy/Bagel_Robotics_1/README.md) for the Python 3.10 environment, exact checkpoint and encoder download, and debug checks. [`REEVALUATION.md`](policy/Bagel_Robotics_1/REEVALUATION.md) gives native simulator task, suite and three-seed commands. RoboDojo and its assets are installed separately.
|
| 14 |
|
| 15 |
+
The adapter is prepared against XPolicyLab `6baccd57ff958570d0db1588370289ee36c8bb70`; source and dependency pins are preserved in the policy folder. The training, pruning and export implementation is publicly available in the [Bagel-Robotics-1 repository](https://github.com/bageldotcom/Bagel-Robotics-1). The technical report remains planned for November 20, 2026.
|
| 16 |
|
| 17 |
The repository is public and ungated. The pinned checkpoint and inference files can be downloaded without a Hugging Face account or access token.
|
inference/SHA256SUMS
CHANGED
|
@@ -1,36 +1,52 @@
|
|
| 1 |
-
|
| 2 |
db9db816c8dd0f432068df166bbe3d9aa36ee4d47c5239b4862f73afa3843b06 install_adapter.py
|
| 3 |
6c9437916a0dc5b6ed5fd56b361a4a1894be910e6965c9570e8683cc74a40813 policy/Bagel_Robotics_1/.gitignore
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
4a4d71a75e893be9ddf802b4e3c853f2010f8ccc277e7b7a5ce2ae1e17614116 policy/Bagel_Robotics_1/VALIDATION.md
|
| 10 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/__init__.py
|
| 11 |
-
|
| 12 |
76fec0d61c9344d8d2c5f61853a93269c8b19ae16d08de673f2ced5bdfe08b79 policy/Bagel_Robotics_1/assets.py
|
| 13 |
2a16b03892ede7aa0a93cfd65e01a239498aa992bc4f1affb9cdde54253dae08 policy/Bagel_Robotics_1/backend.py
|
| 14 |
68f337b21b1e517e1df27550e64fbc542b644528adc8f3f47fb8e44849185af6 policy/Bagel_Robotics_1/deploy.py
|
| 15 |
6f25e548de5e0e10ed797ba65f59c7b64c986d340d7917d46d9aa92091c29b5f policy/Bagel_Robotics_1/deploy.yml
|
| 16 |
-
|
| 17 |
467e70c368f1574f6a39afdb3c84d6b10e8430bdbf638d9742a58585c7e60bc3 policy/Bagel_Robotics_1/download_checkpoint.sh
|
| 18 |
daae036dc45b13f90f1b5b5c9e813262d62585aba467534e2402d4933e8dd97c policy/Bagel_Robotics_1/eval.sh
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
6110ad701ed4c7dd9768b61add6df64ace15c7109ccafdace4ea8ab8934b23cb policy/Bagel_Robotics_1/evidence/action-comparison.json
|
| 20 |
2c8b3a55f75117533bf7ec0cfa8cfa7c42cd9b51d2517161fc022e8e15c5b9d3 policy/Bagel_Robotics_1/evidence/five-suite-results.json
|
| 21 |
3187c8c75bb85c393b65f23838e8163fd201081b82b492601b2a6c7bb552d43a policy/Bagel_Robotics_1/evidence/gpu-validation.json
|
| 22 |
f73d49a6b9c96824eddb76ae146b720a1d2feafba69c2ab7e9222c9167f3985f policy/Bagel_Robotics_1/evidence/local-results.json
|
| 23 |
-
|
|
|
|
| 24 |
d842f9639df62df7fddc84f095771b438f02f955605e221e4f7b160893592454 policy/Bagel_Robotics_1/install.sh
|
| 25 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/Apache-2.0.txt
|
| 26 |
7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe policy/Bagel_Robotics_1/licenses/RoboDojo-MIT.txt
|
| 27 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/XPolicyLab.txt
|
| 28 |
56418464b2a88e7aea31d436ee42851dfababad97db9178abdce318b5a7e621b policy/Bagel_Robotics_1/model.py
|
| 29 |
2d1698608d28de6395f8c8794460b292f0233d2a4a5ef76a5d83152631ffe389 policy/Bagel_Robotics_1/requirements-transport.txt
|
| 30 |
-
|
| 31 |
-
43973f510e70c55382293f19573eb81c42e965f5286f52650ca920d35ad8df77 policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md
|
| 32 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/runtime/licenses/Apache-2.0.txt
|
| 33 |
-
|
| 34 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/__init__.py
|
| 35 |
70e67d196cb46263677ebe873c5837657d6b8f90ca40cb6845f4782f5377ba2d policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/checkpoint_compat.py
|
| 36 |
8729d8e1386c74a0e3bd1c800a7c883725c9d00441f19bcbe7e40c2f8ff7af84 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/data.py
|
|
@@ -55,7 +71,7 @@ ce632db536b479c96a7160968eb314deb4e2dd1ab1b986e68f5dd8dea0ed6468 policy/Bagel_R
|
|
| 55 |
5f42547a8571a2758ea6d4285834d5eaafc67285c6db42d44ec3c317e8402cc2 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/mot.py
|
| 56 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/__init__.py
|
| 57 |
15de04574c1d8ccb794ca6560b1e501e5c83cc86104bb80f1eb3cc3b0324ffe0 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/scheduler_continuous.py
|
| 58 |
-
|
| 59 |
d203d5ec0f6c913b052e22202eff0fe238e087040ae0de9843b7a22a5533b5f3 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan21.py
|
| 60 |
f65e44c9d1b7811d2996b3fa22e6fe4c2ac5e6edf1fc2c14251cf582bf8d1f05 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_clip.py
|
| 61 |
516b8dd09d576898c8760968ada4b617f5fe6a7c3fd945787898ae326df8e9a1 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_dit.py
|
|
@@ -68,7 +84,7 @@ dd3afea396e0ba3e619a1a66121f3c41dd0ca4174e1cf1f9f459b200f23e2690 policy/Bagel_R
|
|
| 68 |
9b0ba19de9a64458b12a0fb24f0811f49d9e9fca5d8b815f5f5ecd3b7886fd63 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/utils/pytorch_utils.py
|
| 69 |
113091a0c7fa38ecda3aae470bb6f151d0d034cfc187d4d752350f8cb9d71191 policy/Bagel_Robotics_1/setup_eval_env_client.sh
|
| 70 |
428ada84e7b5d2c131118e7c8c47a65fb60bc34ce49db94b2e33608c354b93b9 policy/Bagel_Robotics_1/setup_eval_policy_server.sh
|
| 71 |
-
|
| 72 |
42be4b09c56377f4686b2e722de760fa3bafd1c6b92605ada269d94e19efc95e policy/Bagel_Robotics_1/temporal_ensemble.py
|
| 73 |
d5a7f0219ee4fc94c32add51121b83bf2138950a39de67cd42d9a76dddf391eb policy/Bagel_Robotics_1/tests/fixtures/checkpoint_processor.json
|
| 74 |
add0937c0762cba1395426f8c46fc3b5b9bc2d6da63a51202b7b781a53732db9 policy/Bagel_Robotics_1/tests/test_backend.py
|
|
|
|
| 1 |
+
b54511ea4ae7a120abde0d9edf3acc4a67ab35d96c276dfb857f09410beac4fd README.md
|
| 2 |
db9db816c8dd0f432068df166bbe3d9aa36ee4d47c5239b4862f73afa3843b06 install_adapter.py
|
| 3 |
6c9437916a0dc5b6ed5fd56b361a4a1894be910e6965c9570e8683cc74a40813 policy/Bagel_Robotics_1/.gitignore
|
| 4 |
+
0ccef2aa651ff17cc56c91985f81c235d92122bcda70fe9ab864091bf63ac59b policy/Bagel_Robotics_1/EVALUATION_RESULTS.md
|
| 5 |
+
ae9584319bcb20cfb02cf4784c34b098ed7ac4bba7c1d8ad8c9a22eda572a21d policy/Bagel_Robotics_1/LICENSE
|
| 6 |
+
6724c8e65d6eb246ada70f28959a9da869ebb66af3e7b0418f6f44687e786ab9 policy/Bagel_Robotics_1/README.md
|
| 7 |
+
dbfe8357fa1ee02f1a4ca9922c6f0c845172ead932a1d306063103659f5cf1cd policy/Bagel_Robotics_1/REEVALUATION.md
|
| 8 |
+
81452c82f6cf9648197d4fe66134f6e1c9e7c6524ae7f209602d7d39b25836dc policy/Bagel_Robotics_1/VALIDATION.md
|
|
|
|
| 9 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/__init__.py
|
| 10 |
+
674af7f9f237559d7c73100f5dd0c6836a65aea4a2ee9c6defae3a6efa0ca94e policy/Bagel_Robotics_1/artifact-manifest.json
|
| 11 |
76fec0d61c9344d8d2c5f61853a93269c8b19ae16d08de673f2ced5bdfe08b79 policy/Bagel_Robotics_1/assets.py
|
| 12 |
2a16b03892ede7aa0a93cfd65e01a239498aa992bc4f1affb9cdde54253dae08 policy/Bagel_Robotics_1/backend.py
|
| 13 |
68f337b21b1e517e1df27550e64fbc542b644528adc8f3f47fb8e44849185af6 policy/Bagel_Robotics_1/deploy.py
|
| 14 |
6f25e548de5e0e10ed797ba65f59c7b64c986d340d7917d46d9aa92091c29b5f policy/Bagel_Robotics_1/deploy.yml
|
| 15 |
+
410fea45c8c15349cbcafefd0dae3c9be6947b67b6eb495e9436ec79217bab85 policy/Bagel_Robotics_1/download_checkpoint.py
|
| 16 |
467e70c368f1574f6a39afdb3c84d6b10e8430bdbf638d9742a58585c7e60bc3 policy/Bagel_Robotics_1/download_checkpoint.sh
|
| 17 |
daae036dc45b13f90f1b5b5c9e813262d62585aba467534e2402d4933e8dd97c policy/Bagel_Robotics_1/eval.sh
|
| 18 |
+
08fa69475293a505de2d74173827621fe27886be5723e38f1e0118e1529604c4 policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md
|
| 19 |
+
a2946a8c62a7cf3d1e64d8e8655d462533e2a3937b49756bbd458ab81586cad9 policy/Bagel_Robotics_1/evaluation/seed-1/summary.md
|
| 20 |
+
ec35c92be6c3bc2792da1fa2888620c9fb3bea79c37aa5ed7f26927a867d8f8b policy/Bagel_Robotics_1/evaluation/seed-1/verification.json
|
| 21 |
+
a79a06b50af6ef7df8f313c0cb84e954f9a328f4eac1c394216de0681ade251f policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes
|
| 22 |
+
c965c48fbaea7e1dd5852ff556d11c441a4897245a9b132fe55afbbdd1488314 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md
|
| 23 |
+
6ed678932036e827103cd89de0c4748515c54b178eb7862c201eb2976d28af21 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv
|
| 24 |
+
0b19a2cc51364baaacb070c5245eef274b5e9b4393b550ca30365fb4d0c603fb policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py
|
| 25 |
+
7a2ac06a26b8496271a4e5c1d334ed9693d1b96e8058b99834e5e52eaa46d842 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md
|
| 26 |
+
b5284fdaf2d0eb3dd7dfc36e91d2842183d4695b2317c1338f145ed78694db57 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json
|
| 27 |
+
5e710be09f95a3200c1c33755f667b9f95797b4ef5d696d3ee006d3dc6e09f92 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv
|
| 28 |
+
2a7fb1a4ba1d8374b4f0317644f75ba360db17f43a0f22a3c5da50cf02da3544 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md
|
| 29 |
+
c0fe627bdbde293db26714658eac7695d2f6cb5d6e66842a3d29f93edcc70c66 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md
|
| 30 |
+
147306c5805b47d1ecaa7179db7f9a15b1b682e492630e6ab685160b4c23ab63 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv
|
| 31 |
+
c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md
|
| 32 |
+
141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md
|
| 33 |
+
ab9c45c7766c3760304aeb53c517dbc7ccebc97f970016d4a845ccf2df936141 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv
|
| 34 |
+
958dbd046ff0d94562a5281f391ba3dfb7274f7f020a40bf191df87fae1d3044 policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json
|
| 35 |
6110ad701ed4c7dd9768b61add6df64ace15c7109ccafdace4ea8ab8934b23cb policy/Bagel_Robotics_1/evidence/action-comparison.json
|
| 36 |
2c8b3a55f75117533bf7ec0cfa8cfa7c42cd9b51d2517161fc022e8e15c5b9d3 policy/Bagel_Robotics_1/evidence/five-suite-results.json
|
| 37 |
3187c8c75bb85c393b65f23838e8163fd201081b82b492601b2a6c7bb552d43a policy/Bagel_Robotics_1/evidence/gpu-validation.json
|
| 38 |
f73d49a6b9c96824eddb76ae146b720a1d2feafba69c2ab7e9222c9167f3985f policy/Bagel_Robotics_1/evidence/local-results.json
|
| 39 |
+
7ad05fcefc49a0878e86969757f70c649201e0155ee07cd125eee228e69892f3 policy/Bagel_Robotics_1/evidence/pr-audit.json
|
| 40 |
+
e97227862bc82a11d9163bf6a928885949d5888c155c4a516d33c5115239e747 policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json
|
| 41 |
d842f9639df62df7fddc84f095771b438f02f955605e221e4f7b160893592454 policy/Bagel_Robotics_1/install.sh
|
| 42 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/Apache-2.0.txt
|
| 43 |
7794bb06af8fe5485ca912454ad1665ccd7846c45f9c2b1258658e480948cdbe policy/Bagel_Robotics_1/licenses/RoboDojo-MIT.txt
|
| 44 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/licenses/XPolicyLab.txt
|
| 45 |
56418464b2a88e7aea31d436ee42851dfababad97db9178abdce318b5a7e621b policy/Bagel_Robotics_1/model.py
|
| 46 |
2d1698608d28de6395f8c8794460b292f0233d2a4a5ef76a5d83152631ffe389 policy/Bagel_Robotics_1/requirements-transport.txt
|
| 47 |
+
3c289cd7e240d09910bfdfa562a63647f09289430b55eb16efbf20ea5456a6c0 policy/Bagel_Robotics_1/runtime/LICENSE
|
|
|
|
| 48 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 policy/Bagel_Robotics_1/runtime/licenses/Apache-2.0.txt
|
| 49 |
+
eb166587ec972e0df7f2385b759abb405d678359c1f97817998f189c902ca98a policy/Bagel_Robotics_1/runtime/pyproject.toml
|
| 50 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/__init__.py
|
| 51 |
70e67d196cb46263677ebe873c5837657d6b8f90ca40cb6845f4782f5377ba2d policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/checkpoint_compat.py
|
| 52 |
8729d8e1386c74a0e3bd1c800a7c883725c9d00441f19bcbe7e40c2f8ff7af84 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/data.py
|
|
|
|
| 71 |
5f42547a8571a2758ea6d4285834d5eaafc67285c6db42d44ec3c317e8402cc2 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/mot.py
|
| 72 |
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/__init__.py
|
| 73 |
15de04574c1d8ccb794ca6560b1e501e5c83cc86104bb80f1eb3cc3b0324ffe0 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/schedulers/scheduler_continuous.py
|
| 74 |
+
a1df6528f7f9536b2ff4ad3913fa718d40dfb1061dc35e9fa0e28e2e1fa3bf96 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py
|
| 75 |
d203d5ec0f6c913b052e22202eff0fe238e087040ae0de9843b7a22a5533b5f3 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan21.py
|
| 76 |
f65e44c9d1b7811d2996b3fa22e6fe4c2ac5e6edf1fc2c14251cf582bf8d1f05 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_clip.py
|
| 77 |
516b8dd09d576898c8760968ada4b617f5fe6a7c3fd945787898ae326df8e9a1 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/wan_video_dit.py
|
|
|
|
| 84 |
9b0ba19de9a64458b12a0fb24f0811f49d9e9fca5d8b815f5f5ecd3b7886fd63 policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/utils/pytorch_utils.py
|
| 85 |
113091a0c7fa38ecda3aae470bb6f151d0d034cfc187d4d752350f8cb9d71191 policy/Bagel_Robotics_1/setup_eval_env_client.sh
|
| 86 |
428ada84e7b5d2c131118e7c8c47a65fb60bc34ce49db94b2e33608c354b93b9 policy/Bagel_Robotics_1/setup_eval_policy_server.sh
|
| 87 |
+
0be927dbf173f09fd2177f60c02100fc280757d5edc8c5d7fd52dc3e21217487 policy/Bagel_Robotics_1/source-provenance.json
|
| 88 |
42be4b09c56377f4686b2e722de760fa3bafd1c6b92605ada269d94e19efc95e policy/Bagel_Robotics_1/temporal_ensemble.py
|
| 89 |
d5a7f0219ee4fc94c32add51121b83bf2138950a39de67cd42d9a76dddf391eb policy/Bagel_Robotics_1/tests/fixtures/checkpoint_processor.json
|
| 90 |
add0937c0762cba1395426f8c46fc3b5b9bc2d6da63a51202b7b781a53732db9 policy/Bagel_Robotics_1/tests/test_backend.py
|
inference/policy/Bagel_Robotics_1/EVALUATION_RESULTS.md
CHANGED
|
@@ -1,103 +1,355 @@
|
|
| 1 |
-
# Bagel-Robotics-1 results
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
|
| 8 |
-
|
|
| 9 |
-
|
|
| 10 |
-
|
|
| 11 |
-
|
|
| 12 |
-
|
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
|
| 39 |
-
|
|
| 40 |
-
|
|
| 41 |
-
|
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
|
| 52 |
-
|
|
| 53 |
-
|
|
| 54 |
-
|
|
| 55 |
-
|
|
| 56 |
-
|
|
| 57 |
-
|
|
| 58 |
-
|
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
|
| 68 |
-
|
|
| 69 |
-
|
|
| 70 |
-
|
|
| 71 |
-
|
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
|
| 83 |
-
|
|
| 84 |
-
|
|
| 85 |
-
|
|
| 86 |
-
|
|
| 87 |
-
|
|
| 88 |
-
|
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
|
| 93 |
-
|
|
| 94 |
-
|
|
| 95 |
-
|
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Bagel-Robotics-1: three-seed local results
|
| 2 |
+
|
| 3 |
+
## Overview
|
| 4 |
+
|
| 5 |
+
Checkpoint 1000 (AdaPrune Final), seeds **0, 1 and 2**: five suites, 42 tasks and **6,300 reported outcomes**. The selection includes 28 later replacement layouts, one repaired Memory episode and four earlier Seed-1 Precision substitutions; the complete appendix is at the end of this report.
|
| 6 |
+
|
| 7 |
+
| Suite | Successes / reported outcomes (3 seeds) | Success rate (%) | Partial score /100 |
|
| 8 |
+
| --- | ---: | ---: | ---: |
|
| 9 |
+
| Precision | 409/1200 | 34.08 ± 1.90 | 42.43 ± 1.42 |
|
| 10 |
+
| Memory | 509/900 | 56.56 ± 2.06 | 56.89 ± 2.17 |
|
| 11 |
+
| Generalization | 510/1800 | 28.33 ± 1.71 | 35.20 ± 1.35 |
|
| 12 |
+
| Long Horizon | 424/1200 | 35.33 ± 0.82 | 53.03 ± 0.28 |
|
| 13 |
+
| Open | 44/1200 | 3.67 ± 0.31 | 4.25 ± 0.25 |
|
| 14 |
+
| **Five-suite average** | **6,300 outcomes** | **31.59 ± 0.33** | **38.36 ± 0.46** |
|
| 15 |
+
|
| 16 |
+
**Reading the scores:** success means full task completion. Partial scores are out of 100 and include credited progress in both successful and failed episodes. All ± values in this report are population standard deviations across the three seeds. Each seed's overall score weights the five suites equally; it is not the total success count divided by 6,300.
|
| 17 |
+
|
| 18 |
+
**Checkpoint and inference:** [bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1/tree/2651631c5d08819088475e8778676f5cf6d9afcf), pinned revision `2651631c5d08819088475e8778676f5cf6d9afcf`; BF16, ten Euler steps, shift 1, 32 predicted / 16 executed actions and 90/10 temporal blending. These are retained local evaluation results; packaged-adapter checks are recorded in [VALIDATION.md](VALIDATION.md).
|
| 19 |
+
|
| 20 |
+
## Individual seeds
|
| 21 |
+
|
| 22 |
+
| Seed | Suite | Scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
|
| 23 |
+
| --- | --- | ---: | ---: | ---: | ---: | ---: |
|
| 24 |
+
| 0 | Precision | 400/400 | 144 | 256 | 36.00 | 43.50 |
|
| 25 |
+
| 0 | Memory | 300/300 | 161 | 139 | 53.67 | 53.87 |
|
| 26 |
+
| 0 | Generalization | 600/600 | 158 | 442 | 26.33 | 33.52 |
|
| 27 |
+
| 0 | Long Horizon | 400/400 | 145 | 255 | 36.25 | 53.39 |
|
| 28 |
+
| 0 | Open | 400/400 | 15 | 385 | 3.75 | 4.36 |
|
| 29 |
+
| **0** | **Equal-weight five-suite average** | **2,100/2,100** | **623** | **1477** | **31.20** | **37.73** |
|
| 30 |
+
| 1 | Precision | 400/400 | 126 | 274 | 31.50 | 40.42 |
|
| 31 |
+
| 1 | Memory | 300/300 | 173 | 127 | 57.67 | 58.00 |
|
| 32 |
+
| 1 | Generalization | 600/600 | 183 | 417 | 30.50 | 36.83 |
|
| 33 |
+
| 1 | Long Horizon | 400/400 | 137 | 263 | 34.25 | 53.00 |
|
| 34 |
+
| 1 | Open | 400/400 | 16 | 384 | 4.00 | 4.47 |
|
| 35 |
+
| **1** | **Equal-weight five-suite average** | **2,100/2,100** | **635** | **1465** | **31.58** | **38.55** |
|
| 36 |
+
| 2 | Precision | 400/400 | 139 | 261 | 34.75 | 43.38 |
|
| 37 |
+
| 2 | Memory | 300/300 | 175 | 125 | 58.33 | 58.82 |
|
| 38 |
+
| 2 | Generalization | 600/600 | 169 | 431 | 28.17 | 35.27 |
|
| 39 |
+
| 2 | Long Horizon | 400/400 | 142 | 258 | 35.50 | 52.70 |
|
| 40 |
+
| 2 | Open | 400/400 | 13 | 387 | 3.25 | 3.90 |
|
| 41 |
+
| **2** | **Equal-weight five-suite average** | **2,100/2,100** | **638** | **1462** | **32.00** | **38.81** |
|
| 42 |
+
|
| 43 |
+
## Task results
|
| 44 |
+
|
| 45 |
+
Each task has 50 reported outcomes per seed. Generalization combines 25 standard and 25 randomized episodes. Success columns show successes/evaluated episodes and percentage; failures equal evaluated episodes minus successes. Expand a suite to see its tasks.
|
| 46 |
+
|
| 47 |
+
<details>
|
| 48 |
+
<summary>Precision</summary>
|
| 49 |
+
|
| 50 |
+
| Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
|
| 51 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 52 |
+
| Build tower | 40/50 (80.00%) | 85.60 | 32/50 (64.00%) | 73.60 | 34/50 (68.00%) | 75.40 | 70.67 ± 6.80 | 78.20 ± 5.28 |
|
| 53 |
+
| Deposit coin | 14/50 (28.00%) | 32.40 | 11/50 (22.00%) | 26.80 | 12/50 (24.00%) | 28.40 | 24.67 ± 2.49 | 29.20 ± 2.36 |
|
| 54 |
+
| Fasten screws | 24/50 (48.00%) | 72.20 | 18/50 (36.00%) | 66.80 | 20/50 (40.00%) | 68.20 | 41.33 ± 4.99 | 69.07 ± 2.29 |
|
| 55 |
+
| Insert key | 0/50 (0.00%) | 15.00 | 0/50 (0.00%) | 15.00 | 0/50 (0.00%) | 15.00 | 0.00 ± 0.00 | 15.00 ± 0.00 |
|
| 56 |
+
| Insert tubes | 36/50 (72.00%) | 82.80 | 32/50 (64.00%) | 75.20 | 30/50 (60.00%) | 74.00 | 65.33 ± 4.99 | 77.33 ± 3.90 |
|
| 57 |
+
| Play xylophone | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
|
| 58 |
+
| Plug in charger | 6/50 (12.00%) | 12.00 | 2/50 (4.00%) | 4.00 | 4/50 (8.00%) | 8.00 | 8.00 ± 3.27 | 8.00 ± 3.27 |
|
| 59 |
+
| Pour balls into vase | 24/50 (48.00%) | 48.00 | 31/50 (62.00%) | 62.00 | 39/50 (78.00%) | 78.00 | 62.67 ± 12.26 | 62.67 ± 12.26 |
|
| 60 |
+
|
| 61 |
+
</details>
|
| 62 |
+
|
| 63 |
+
|
| 64 |
+
<details>
|
| 65 |
+
<summary>Memory</summary>
|
| 66 |
+
|
| 67 |
+
| Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
|
| 68 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 69 |
+
| Cover blocks | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 100.00 ± 0.00 | 100.00 ± 0.00 |
|
| 70 |
+
| Imitate sorting sequence | 1/50 (2.00%) | 3.20 | 0/50 (0.00%) | 2.00 | 0/50 (0.00%) | 2.90 | 0.67 ± 0.94 | 2.70 ± 0.51 |
|
| 71 |
+
| Match and pick from conveyor | 41/50 (82.00%) | 82.00 | 46/50 (92.00%) | 92.00 | 43/50 (86.00%) | 86.00 | 86.67 ± 4.11 | 86.67 ± 4.11 |
|
| 72 |
+
| Press by number | 49/50 (98.00%) | 98.00 | 50/50 (100.00%) | 100.00 | 50/50 (100.00%) | 100.00 | 99.33 ± 0.94 | 99.33 ± 0.94 |
|
| 73 |
+
| Swap t | 11/50 (22.00%) | 22.00 | 11/50 (22.00%) | 22.00 | 16/50 (32.00%) | 32.00 | 25.33 ± 4.71 | 25.33 ± 4.71 |
|
| 74 |
+
| Swap blocks | 9/50 (18.00%) | 18.00 | 16/50 (32.00%) | 32.00 | 16/50 (32.00%) | 32.00 | 27.33 ± 6.60 | 27.33 ± 6.60 |
|
| 75 |
+
|
| 76 |
+
</details>
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
<details>
|
| 80 |
+
<summary>Generalization</summary>
|
| 81 |
+
|
| 82 |
+
| Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
|
| 83 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 84 |
+
| Arrange largest number | 3/50 (6.00%) | 10.60 | 2/50 (4.00%) | 11.00 | 0/50 (0.00%) | 7.60 | 3.33 ± 2.49 | 9.73 ± 1.52 |
|
| 85 |
+
| Fold clothes | 33/50 (66.00%) | 68.00 | 36/50 (72.00%) | 74.40 | 37/50 (74.00%) | 76.80 | 70.67 ± 3.40 | 73.07 ± 3.71 |
|
| 86 |
+
| Hang mugs | 15/50 (30.00%) | 41.40 | 14/50 (28.00%) | 38.50 | 14/50 (28.00%) | 40.60 | 28.67 ± 0.94 | 40.17 ± 1.22 |
|
| 87 |
+
| Make toast | 16/50 (32.00%) | 47.50 | 22/50 (44.00%) | 52.50 | 17/50 (34.00%) | 45.00 | 36.67 ± 5.25 | 48.33 ± 3.12 |
|
| 88 |
+
| Pack objects into box | 1/50 (2.00%) | 26.50 | 6/50 (12.00%) | 35.40 | 3/50 (6.00%) | 29.90 | 6.67 ± 4.11 | 30.60 ± 3.67 |
|
| 89 |
+
| Pour liquid into cup | 15/50 (30.00%) | 30.00 | 22/50 (44.00%) | 44.00 | 23/50 (46.00%) | 46.00 | 40.00 ± 7.12 | 40.00 ± 7.12 |
|
| 90 |
+
| Push t | 0/50 (0.00%) | 0.00 | 1/50 (2.00%) | 2.00 | 1/50 (2.00%) | 2.00 | 1.33 ± 0.94 | 1.33 ± 0.94 |
|
| 91 |
+
| Sort nesting dolls by size | 10/50 (20.00%) | 20.00 | 10/50 (20.00%) | 20.00 | 7/50 (14.00%) | 14.00 | 18.00 ± 2.83 | 18.00 ± 2.83 |
|
| 92 |
+
| Stack blocks | 18/50 (36.00%) | 43.50 | 22/50 (44.00%) | 50.00 | 23/50 (46.00%) | 52.00 | 42.00 ± 4.32 | 48.50 ± 3.63 |
|
| 93 |
+
| Stack bowls | 24/50 (48.00%) | 55.50 | 26/50 (52.00%) | 58.90 | 23/50 (46.00%) | 54.10 | 48.67 ± 2.49 | 56.17 ± 2.02 |
|
| 94 |
+
| Store laptop and headphones | 18/50 (36.00%) | 49.20 | 20/50 (40.00%) | 51.20 | 17/50 (34.00%) | 47.20 | 36.67 ± 2.49 | 49.20 ± 1.63 |
|
| 95 |
+
| Sweep blocks | 5/50 (10.00%) | 10.00 | 2/50 (4.00%) | 4.00 | 4/50 (8.00%) | 8.00 | 7.33 ± 2.49 | 7.33 ± 2.49 |
|
| 96 |
+
|
| 97 |
+
</details>
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
<details>
|
| 101 |
+
<summary>Long Horizon</summary>
|
| 102 |
+
|
| 103 |
+
| Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
|
| 104 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 105 |
+
| Classify objects | 23/50 (46.00%) | 56.70 | 16/50 (32.00%) | 44.50 | 21/50 (42.00%) | 51.10 | 40.00 ± 5.89 | 50.77 ± 4.99 |
|
| 106 |
+
| Fill egg holder | 1/50 (2.00%) | 11.00 | 2/50 (4.00%) | 13.80 | 0/50 (0.00%) | 12.20 | 2.00 ± 1.63 | 12.33 ± 1.15 |
|
| 107 |
+
| Fill pen holder | 18/50 (36.00%) | 53.90 | 11/50 (22.00%) | 43.90 | 15/50 (30.00%) | 52.40 | 29.33 ± 5.73 | 50.07 ± 4.40 |
|
| 108 |
+
| Make kong | 42/50 (84.00%) | 84.00 | 43/50 (86.00%) | 86.00 | 44/50 (88.00%) | 88.00 | 86.00 ± 1.63 | 86.00 ± 1.63 |
|
| 109 |
+
| Organize table | 7/50 (14.00%) | 57.50 | 9/50 (18.00%) | 64.50 | 6/50 (12.00%) | 58.00 | 14.67 ± 2.49 | 60.00 ± 3.19 |
|
| 110 |
+
| Play stacking toy | 7/50 (14.00%) | 31.80 | 6/50 (12.00%) | 32.00 | 4/50 (8.00%) | 27.20 | 11.33 ± 2.49 | 30.33 ± 2.22 |
|
| 111 |
+
| Play tic tac toe | 0/50 (0.00%) | 35.80 | 1/50 (2.00%) | 40.50 | 3/50 (6.00%) | 33.90 | 2.67 ± 2.49 | 36.73 ± 2.77 |
|
| 112 |
+
| Put bottles into dustbin | 47/50 (94.00%) | 96.40 | 49/50 (98.00%) | 98.80 | 49/50 (98.00%) | 98.80 | 96.67 ± 1.89 | 98.00 ± 1.13 |
|
| 113 |
+
|
| 114 |
+
</details>
|
| 115 |
+
|
| 116 |
+
|
| 117 |
+
<details>
|
| 118 |
+
<summary>Open</summary>
|
| 119 |
+
|
| 120 |
+
| Task | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 | Mean success rate (%) ± SD | Mean partial /100 ± SD |
|
| 121 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 122 |
+
| Align blocks | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
|
| 123 |
+
| Classify objects by language | 0/50 (0.00%) | 3.60 | 0/50 (0.00%) | 2.60 | 3/50 (6.00%) | 9.20 | 2.00 ± 2.83 | 5.13 ± 2.90 |
|
| 124 |
+
| General pickup | 13/50 (26.00%) | 26.00 | 12/50 (24.00%) | 24.00 | 8/50 (16.00%) | 16.00 | 22.00 ± 4.32 | 22.00 ± 4.32 |
|
| 125 |
+
| Pick from conveyor by image | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
|
| 126 |
+
| Pour by language | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
|
| 127 |
+
| Solve equation | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.00 ± 0.00 |
|
| 128 |
+
| Stack blocks by language | 2/50 (4.00%) | 4.80 | 4/50 (8.00%) | 9.20 | 2/50 (4.00%) | 6.00 | 5.33 ± 1.89 | 6.67 ± 1.86 |
|
| 129 |
+
| Store tools in toolbox | 0/50 (0.00%) | 0.50 | 0/50 (0.00%) | 0.00 | 0/50 (0.00%) | 0.00 | 0.00 ± 0.00 | 0.17 ± 0.24 |
|
| 130 |
+
|
| 131 |
+
</details>
|
| 132 |
+
|
| 133 |
+
## Generalization split
|
| 134 |
+
|
| 135 |
+
<details>
|
| 136 |
+
<summary>Standard versus randomized results</summary>
|
| 137 |
+
|
| 138 |
+
Each split has 300 episodes per seed. The following rows preserve the 25-episode task halves.
|
| 139 |
+
|
| 140 |
+
| Seed | Split | Successes / scored | Success rate (%) | Partial /100 |
|
| 141 |
+
| --- | --- | ---: | ---: | ---: |
|
| 142 |
+
| 0 | Standard | 134/300 | 44.67 | 51.13 |
|
| 143 |
+
| 0 | Randomized | 24/300 | 8.00 | 15.90 |
|
| 144 |
+
| 1 | Standard | 148/300 | 49.33 | 54.88 |
|
| 145 |
+
| 1 | Randomized | 35/300 | 11.67 | 18.77 |
|
| 146 |
+
| 2 | Standard | 129/300 | 43.00 | 49.30 |
|
| 147 |
+
| 2 | Randomized | 40/300 | 13.33 | 21.23 |
|
| 148 |
+
|
| 149 |
+
| Task / split | Seed 0 successes (%) | Seed 0 partial /100 | Seed 1 successes (%) | Seed 1 partial /100 | Seed 2 successes (%) | Seed 2 partial /100 |
|
| 150 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 151 |
+
| Arrange largest number / standard | 3/25 (12.00%) | 16.60 | 1/25 (4.00%) | 13.00 | 0/25 (0.00%) | 8.60 |
|
| 152 |
+
| Arrange largest number / randomized | 0/25 (0.00%) | 4.60 | 1/25 (4.00%) | 9.00 | 0/25 (0.00%) | 6.60 |
|
| 153 |
+
| Fold clothes / standard | 17/25 (68.00%) | 70.40 | 20/25 (80.00%) | 80.80 | 16/25 (64.00%) | 68.00 |
|
| 154 |
+
| Fold clothes / randomized | 16/25 (64.00%) | 65.60 | 16/25 (64.00%) | 68.00 | 21/25 (84.00%) | 85.60 |
|
| 155 |
+
| Hang mugs / standard | 15/25 (60.00%) | 71.00 | 14/25 (56.00%) | 67.40 | 14/25 (56.00%) | 69.00 |
|
| 156 |
+
| Hang mugs / randomized | 0/25 (0.00%) | 11.80 | 0/25 (0.00%) | 9.60 | 0/25 (0.00%) | 12.20 |
|
| 157 |
+
| Make toast / standard | 14/25 (56.00%) | 71.00 | 18/25 (72.00%) | 79.00 | 15/25 (60.00%) | 67.00 |
|
| 158 |
+
| Make toast / randomized | 2/25 (8.00%) | 24.00 | 4/25 (16.00%) | 26.00 | 2/25 (8.00%) | 23.00 |
|
| 159 |
+
| Pack objects into box / standard | 0/25 (0.00%) | 29.40 | 4/25 (16.00%) | 42.60 | 2/25 (8.00%) | 34.60 |
|
| 160 |
+
| Pack objects into box / randomized | 1/25 (4.00%) | 23.60 | 2/25 (8.00%) | 28.20 | 1/25 (4.00%) | 25.20 |
|
| 161 |
+
| Pour liquid into cup / standard | 13/25 (52.00%) | 52.00 | 17/25 (68.00%) | 68.00 | 16/25 (64.00%) | 64.00 |
|
| 162 |
+
| Pour liquid into cup / randomized | 2/25 (8.00%) | 8.00 | 5/25 (20.00%) | 20.00 | 7/25 (28.00%) | 28.00 |
|
| 163 |
+
| Push t / standard | 0/25 (0.00%) | 0.00 | 1/25 (4.00%) | 4.00 | 1/25 (4.00%) | 4.00 |
|
| 164 |
+
| Push t / randomized | 0/25 (0.00%) | 0.00 | 0/25 (0.00%) | 0.00 | 0/25 (0.00%) | 0.00 |
|
| 165 |
+
| Sort nesting dolls by size / standard | 9/25 (36.00%) | 36.00 | 9/25 (36.00%) | 36.00 | 7/25 (28.00%) | 28.00 |
|
| 166 |
+
| Sort nesting dolls by size / randomized | 1/25 (4.00%) | 4.00 | 1/25 (4.00%) | 4.00 | 0/25 (0.00%) | 0.00 |
|
| 167 |
+
| Stack blocks / standard | 18/25 (72.00%) | 76.20 | 22/25 (88.00%) | 89.80 | 23/25 (92.00%) | 93.20 |
|
| 168 |
+
| Stack blocks / randomized | 0/25 (0.00%) | 10.80 | 0/25 (0.00%) | 10.20 | 0/25 (0.00%) | 10.80 |
|
| 169 |
+
| Stack bowls / standard | 24/25 (96.00%) | 96.60 | 23/25 (92.00%) | 93.20 | 21/25 (84.00%) | 86.40 |
|
| 170 |
+
| Stack bowls / randomized | 0/25 (0.00%) | 14.40 | 3/25 (12.00%) | 24.60 | 2/25 (8.00%) | 21.80 |
|
| 171 |
+
| Store laptop and headphones / standard | 17/25 (68.00%) | 78.40 | 17/25 (68.00%) | 76.80 | 12/25 (48.00%) | 60.80 |
|
| 172 |
+
| Store laptop and headphones / randomized | 1/25 (4.00%) | 20.00 | 3/25 (12.00%) | 25.60 | 5/25 (20.00%) | 33.60 |
|
| 173 |
+
| Sweep blocks / standard | 4/25 (16.00%) | 16.00 | 2/25 (8.00%) | 8.00 | 2/25 (8.00%) | 8.00 |
|
| 174 |
+
| Sweep blocks / randomized | 1/25 (4.00%) | 4.00 | 0/25 (0.00%) | 0.00 | 2/25 (8.00%) | 8.00 |
|
| 175 |
+
|
| 176 |
+
|
| 177 |
+
</details>
|
| 178 |
+
|
| 179 |
+
## Reproduction files
|
| 180 |
+
|
| 181 |
+
<details>
|
| 182 |
+
<summary>Official summary, episode data and regeneration instructions</summary>
|
| 183 |
+
|
| 184 |
+
- [Download the unchanged RoboDojo summary](evaluation/seeds-0-1-2/summary.md).
|
| 185 |
+
- [Episode data and verification files](evaluation/seeds-0-1-2/): all 6,300 episode scores, replacement and retry ledgers, unscored originals, protocol records, checksums and the original-selection summary.
|
| 186 |
+
|
| 187 |
+
The native summary's overview uses the same standard deviation as this report; its task-table ± values instead use half the highest-to-lowest range. Its final task-weighted row is a different average: use the five-suite Overview for the headline. The earlier Notion report used sample standard deviation; its means are identical, but its ± values are larger. Neither describes a confidence interval. The native policy label identifies replacements and repair and represents this same checkpoint.
|
| 188 |
+
|
| 189 |
+
### Evaluation and summary versions
|
| 190 |
+
|
| 191 |
+
- Evaluation lineage: frozen RoboDojo snapshot based on `08b7ee46034c3d0c8a389b2b7bfc138f4d55ee4d`, with local runtime/reproducibility patches. All 34 recorded protocol manifests match the retained baseline's 55 task-source hashes; 23 also record and match all 55 configuration hashes. Eleven cloud protocol manifests do not include those configuration hashes.
|
| 192 |
+
- Summary script revision: `f57e1633ffed59761bc8e568cfe7530c4b33381c`, verified byte-for-byte. Generating the summary at this revision does not change the benchmark version that produced the episodes.
|
| 193 |
+
- This compilation joins the retained cloud archive, earlier verified seed-1 package and completed node-2 records. The 750 overlapping seed-1 outcomes agree and are included once. Seed 2 Open comes from the completed retained cloud campaign.
|
| 194 |
+
|
| 195 |
+
Evaluation/model seeds 0/1/2 are recorded in each protocol. The native simulator reset uses the selected layout identity.
|
| 196 |
+
|
| 197 |
+
### Regenerate the summary
|
| 198 |
+
|
| 199 |
+
Use a separate RoboDojo checkout at the pinned summary revision; do not change an active evaluation checkout. No GPU, model weights or simulator installation is needed to summarize these scores.
|
| 200 |
+
|
| 201 |
+
```bash
|
| 202 |
+
git clone https://github.com/RoboDojo-Benchmark/RoboDojo.git /tmp/robodojo-summary-source
|
| 203 |
+
git -C /tmp/robodojo-summary-source checkout f57e1633ffed59761bc8e568cfe7530c4b33381c
|
| 204 |
+
|
| 205 |
+
# From policy/Bagel_Robotics_1:
|
| 206 |
+
cd evaluation/seeds-0-1-2
|
| 207 |
+
sha256sum -c SHA256SUMS
|
| 208 |
+
python3 generate_summary.py --robodojo-root /tmp/robodojo-summary-source \
|
| 209 |
+
--output-dir /tmp/bagel-robotics-1-reported-summary
|
| 210 |
+
cmp summary.md /tmp/bagel-robotics-1-reported-summary/summary.md
|
| 211 |
+
|
| 212 |
+
python3 generate_summary.py --robodojo-root /tmp/robodojo-summary-source \
|
| 213 |
+
--view original --output-dir /tmp/bagel-robotics-1-original-summary
|
| 214 |
+
cmp summary-original-selection.md /tmp/bagel-robotics-1-original-summary/summary.md
|
| 215 |
+
```
|
| 216 |
+
|
| 217 |
+
The output directories must be new or empty. `generate_summary.py` verifies the two pinned script hashes, groups the published score fields into 162 native task/seed JSON files and invokes **`bash scripts/robodojo.sh summarize` unchanged**. Actual layout IDs and outcomes are preserved. These grouped JSONs are a score projection of the retained native records, not new simulator runs or byte-identical copies of the original one-episode files.
|
| 218 |
+
|
| 219 |
+
This score-only package omits rollout videos. The unmodified summarizer consequently reports missing videos in `summarize.log`; no placeholder videos are created. For model re-evaluation, use [REEVALUATION.md](REEVALUATION.md).
|
| 220 |
+
|
| 221 |
+
</details>
|
| 222 |
+
|
| 223 |
+
## Replacement and repair appendix
|
| 224 |
+
|
| 225 |
+
All reported outcomes retain their native success Boolean and partial score. Replacement scenes use supplied unused layouts; each is counted once in place of its unscored original. Failed startup attempts are not task failures and are not included in scored denominators.
|
| 226 |
+
|
| 227 |
+
### Outcome accounting
|
| 228 |
+
|
| 229 |
+
| Seed | Scored original selection | Later replacement layouts | Repaired variant | Reported outcomes |
|
| 230 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 231 |
+
| 0 | 2084 | 16 | 0 | 2100 |
|
| 232 |
+
| 1 | 2096 | 3 | 1 | 2100 |
|
| 233 |
+
| 2 | 2091 | 9 | 0 | 2100 |
|
| 234 |
+
| **Total** | **6,271** | **28** | **1** | **6,300** |
|
| 235 |
+
|
| 236 |
+
### Complete replacement and repair ledger
|
| 237 |
+
|
| 238 |
+
Partial scores below are out of 100. All listed originals remain unscored in the unchanged-original view.
|
| 239 |
+
|
| 240 |
+
| Seed | Task | Original layout | Reported layout | Type | Full success | Partial /100 |
|
| 241 |
+
| --- | --- | ---: | ---: | --- | --- | ---: |
|
| 242 |
+
| 0 | Fill egg holder | 28 | 50 | Supplied replacement | No | 40.00 |
|
| 243 |
+
| 0 | Fill egg holder | 35 | 51 | Supplied replacement | No | 0.00 |
|
| 244 |
+
| 0 | Fill egg holder | 37 | 52 | Supplied replacement | No | 0.00 |
|
| 245 |
+
| 0 | Imitate sorting sequence | 4 | 50 | Supplied replacement | No | 0.00 |
|
| 246 |
+
| 0 | Imitate sorting sequence | 8 | 51 | Supplied replacement | No | 5.00 |
|
| 247 |
+
| 0 | Imitate sorting sequence | 17 | 52 | Supplied replacement | No | 0.00 |
|
| 248 |
+
| 0 | Imitate sorting sequence | 29 | 53 | Supplied replacement | No | 0.00 |
|
| 249 |
+
| 0 | Imitate sorting sequence | 36 | 54 | Supplied replacement | No | 0.00 |
|
| 250 |
+
| 0 | Imitate sorting sequence | 48 | 55 | Supplied replacement | No | 0.00 |
|
| 251 |
+
| 0 | Imitate sorting sequence | 49 | 56 | Supplied replacement | No | 0.00 |
|
| 252 |
+
| 0 | General pickup | 29 | 50 | Supplied replacement | Yes | 100.00 |
|
| 253 |
+
| 0 | Deposit coin | 36 | 51 | Supplied replacement | No | 0.00 |
|
| 254 |
+
| 0 | Deposit coin | 37 | 52 | Supplied replacement | Yes | 100.00 |
|
| 255 |
+
| 0 | Deposit coin | 41 | 53 | Supplied replacement | No | 0.00 |
|
| 256 |
+
| 0 | Deposit coin | 43 | 54 | Supplied replacement | No | 0.00 |
|
| 257 |
+
| 0 | Deposit coin | 31 | 55 | Supplied replacement | No | 0.00 |
|
| 258 |
+
| 1 | Fill egg holder | 15 | 50 | Supplied replacement | No | 0.00 |
|
| 259 |
+
| 1 | Fill egg holder | 20 | 51 | Supplied replacement | No | 25.00 |
|
| 260 |
+
| 1 | Imitate sorting sequence | 47 | 47 | Repaired same layout | No | 0.00 |
|
| 261 |
+
| 1 | General pickup | 5 | 50 | Supplied replacement | No | 0.00 |
|
| 262 |
+
| 2 | Fill egg holder | 13 | 50 | Supplied replacement | No | 25.00 |
|
| 263 |
+
| 2 | Fill egg holder | 18 | 52 | Supplied replacement | No | 0.00 |
|
| 264 |
+
| 2 | Fill egg holder | 40 | 53 | Supplied replacement | No | 10.00 |
|
| 265 |
+
| 2 | Imitate sorting sequence | 16 | 50 | Supplied replacement | No | 0.00 |
|
| 266 |
+
| 2 | Imitate sorting sequence | 24 | 51 | Supplied replacement | No | 15.00 |
|
| 267 |
+
| 2 | Imitate sorting sequence | 32 | 52 | Supplied replacement | No | 0.00 |
|
| 268 |
+
| 2 | Imitate sorting sequence | 43 | 53 | Supplied replacement | No | 0.00 |
|
| 269 |
+
| 2 | Deposit coin | 24 | 50 | Supplied replacement | No | 0.00 |
|
| 270 |
+
| 2 | Deposit coin | 33 | 51 | Supplied replacement | No | 0.00 |
|
| 271 |
+
|
| 272 |
+
|
| 273 |
+
### Earlier Seed 1 Precision selection
|
| 274 |
+
|
| 275 |
+
The frozen Deposit Coin selection already contained supplied layouts **50, 51, 52 and 53**, with original IDs **4, 8, 39 and 42** omitted. These are four pre-existing selected substitutions, separate from the 28 later recovery replacements above. No pairwise mapping between these two sets is inferred. “Original selection” means the frozen manifest used by that run, not necessarily layouts 0–49. Their scored outcomes are:
|
| 276 |
+
|
| 277 |
+
| Selected layout | Full success | Partial /100 |
|
| 278 |
+
| --- | --- | ---: |
|
| 279 |
+
| 50 | Yes | 100.00 |
|
| 280 |
+
| 51 | No | 0.00 |
|
| 281 |
+
| 52 | No | 20.00 |
|
| 282 |
+
| 53 | No | 0.00 |
|
| 283 |
+
|
| 284 |
+
### Why replacements were used
|
| 285 |
+
|
| 286 |
+
- Deposit Coin, General Pickup and Fill Egg Holder originals failed native startup/stability validation before policy actions. Sorting Sequence originals failed the benchmark demonstration-validity check. Original attempts are retained in the source records.
|
| 287 |
+
- Seed 0 Deposit Coin reserve 50 failed all three startup attempts; reserve 55 was used for original layout 31. Reserve 50 contributes no score.
|
| 288 |
+
- Seed 2 Fill Egg Holder reserve 51 also failed startup and contributes no score. The scored replacements are 13→50, 18→52 and 40→53.
|
| 289 |
+
|
| 290 |
+
### Memory Seed 1: repaired Sorting Sequence layout 47
|
| 291 |
+
|
| 292 |
+
The original supplied demonstration was invalid. A separately recorded repair changes the demonstration timing, adding **1,431 physics frames at 0.004 seconds/frame (5.724 seconds)**. The policy step budget remains 1,600. This repaired variant completed as a **valid failure, partial score 0**, after 629 policy steps. It is not an unchanged-original result.
|
| 293 |
+
|
| 294 |
+
- Original demonstration SHA-256: `170c63d412e6168d6d71dd58d5870f39d3caf85836ca0bb79d441742e7f6083e`.
|
| 295 |
+
- Repaired demonstration SHA-256: `526c9c572d1e7a635114da6e8cf436fd4ad12451927e25e8940da0cd143a1182`.
|
| 296 |
+
- Repair protocol: `8c8f9894f60ac3a905644871b9675d9b6c87ba4e3f52e7228951b49ac1623d75`.
|
| 297 |
+
|
| 298 |
+
### Completed scores recovered after shutdown errors
|
| 299 |
+
|
| 300 |
+
| Seed | Task / layout | Retained native outcome | Recovery |
|
| 301 |
+
| --- | --- | --- | --- |
|
| 302 |
+
| 1 | Match and Pick from Conveyor / 30 | Success; partial 100/100; 388 actions | Native evaluation completed before simulator teardown exit −6. Completion was reconciled from retained evidence; no rerun or score change. |
|
| 303 |
+
| 2 | Insert Tubes / 25 | Failure; partial 40/100; 500 actions | Native evaluation completed before shutdown timeout. Completion was reconciled; the original error receipt and native score remain unchanged. |
|
| 304 |
+
|
| 305 |
+
### Retries of original scenes
|
| 306 |
+
|
| 307 |
+
The `retried-originals.csv` ledger in the episode-data directory lists scored originals with more than one recorded attempt or a later attempt index: **80 for seed 0, 66 for seed 1 and 33 for seed 2**. They are counted once, with their original scene identity. Retained campaigns expose attempt history differently; this field is not a uniform count of extra evaluated episodes. Seed 2 Swap T layout 3, for example, completed on its second original attempt with failure and partial zero.
|
| 308 |
+
|
| 309 |
+
The original-selection-only tables below show the scores before the 28 later replacements and repaired variant were included. This appendix records the local selection; it does not imply committee approval of substitutions.
|
| 310 |
+
|
| 311 |
+
### Original-selection-only comparisons
|
| 312 |
+
|
| 313 |
+
<details>
|
| 314 |
+
<summary>Suite scores and affected tasks before later replacements and repair</summary>
|
| 315 |
+
|
| 316 |
+
This view excludes the 28 later replacement layouts and the repaired Memory variant. It contains **6,271 scored original-selection outcomes**. Invalid original episodes remain unscored, not zero-valued failures. Seed 1 Precision retains its historical frozen selection, including the four earlier substitutions listed above.
|
| 317 |
+
|
| 318 |
+
These tables divide by the number of scored outcomes shown. The unchanged native `summary-original-selection.md` instead leaves incomplete task/seed cells blank and excludes them from its aggregates, so its coverage and averages differ.
|
| 319 |
+
|
| 320 |
+
| Seed | Suite | Scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
|
| 321 |
+
| --- | --- | ---: | ---: | ---: | ---: | ---: |
|
| 322 |
+
| 0 | Precision | 395/400 | 143 | 252 | 36.20 | 43.80 |
|
| 323 |
+
| 0 | Memory | 293/300 | 161 | 132 | 54.95 | 55.14 |
|
| 324 |
+
| 0 | Generalization | 600/600 | 158 | 442 | 26.33 | 33.52 |
|
| 325 |
+
| 0 | Long Horizon | 397/400 | 145 | 252 | 36.52 | 53.69 |
|
| 326 |
+
| 0 | Open | 399/400 | 14 | 385 | 3.51 | 4.12 |
|
| 327 |
+
| 1 | Precision | 400/400 | 126 | 274 | 31.50 | 40.42 |
|
| 328 |
+
| 1 | Memory | 299/300 | 173 | 126 | 57.86 | 58.19 |
|
| 329 |
+
| 1 | Generalization | 600/600 | 183 | 417 | 30.50 | 36.83 |
|
| 330 |
+
| 1 | Long Horizon | 398/400 | 137 | 261 | 34.42 | 53.20 |
|
| 331 |
+
| 1 | Open | 399/400 | 16 | 383 | 4.01 | 4.49 |
|
| 332 |
+
| 2 | Precision | 398/400 | 139 | 259 | 34.92 | 43.59 |
|
| 333 |
+
| 2 | Memory | 296/300 | 175 | 121 | 59.12 | 59.56 |
|
| 334 |
+
| 2 | Generalization | 600/600 | 169 | 431 | 28.17 | 35.27 |
|
| 335 |
+
| 2 | Long Horizon | 397/400 | 142 | 255 | 35.77 | 53.01 |
|
| 336 |
+
| 2 | Open | 400/400 | 13 | 387 | 3.25 | 3.90 |
|
| 337 |
+
|
| 338 |
+
#### Tasks affected by later replacements or repair
|
| 339 |
+
|
| 340 |
+
| Seed | Task | Original scored / planned | Successes | Failures | Success rate (%) | Partial /100 |
|
| 341 |
+
| --- | --- | ---: | ---: | ---: | ---: | ---: |
|
| 342 |
+
| 0 | Deposit coin | 45/50 | 13 | 32 | 28.89 | 33.78 |
|
| 343 |
+
| 0 | Fill egg holder | 47/50 | 1 | 46 | 2.13 | 10.85 |
|
| 344 |
+
| 0 | General pickup | 49/50 | 12 | 37 | 24.49 | 24.49 |
|
| 345 |
+
| 0 | Imitate sorting sequence | 43/50 | 1 | 42 | 2.33 | 3.60 |
|
| 346 |
+
| 1 | Fill egg holder | 48/50 | 2 | 46 | 4.17 | 13.85 |
|
| 347 |
+
| 1 | General pickup | 49/50 | 12 | 37 | 24.49 | 24.49 |
|
| 348 |
+
| 1 | Imitate sorting sequence | 49/50 | 0 | 49 | 0.00 | 2.04 |
|
| 349 |
+
| 2 | Deposit coin | 48/50 | 12 | 36 | 25.00 | 29.58 |
|
| 350 |
+
| 2 | Fill egg holder | 47/50 | 0 | 47 | 0.00 | 12.23 |
|
| 351 |
+
| 2 | Imitate sorting sequence | 46/50 | 0 | 46 | 0.00 | 2.83 |
|
| 352 |
+
|
| 353 |
+
All other task rows match the reported task tables above. The `episode-scores.csv` ledger can be filtered with `record_type=original` to reproduce this view.
|
| 354 |
+
|
| 355 |
+
</details>
|
inference/policy/Bagel_Robotics_1/LICENSE
CHANGED
|
@@ -22,3 +22,9 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
| 22 |
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 23 |
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 24 |
SOFTWARE.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 23 |
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 24 |
SOFTWARE.
|
| 25 |
+
|
| 26 |
+
Component attribution
|
| 27 |
+
VPP2/FastWAM source retains the MIT copyright and terms above.
|
| 28 |
+
Wan2.1/DiffSynth-derived components retain Apache-2.0 terms; see licenses/Apache-2.0.txt.
|
| 29 |
+
The XPolicyLab interface retains its license in licenses/XPolicyLab.txt.
|
| 30 |
+
Model weights and datasets retain their respective upstream terms.
|
inference/policy/Bagel_Robotics_1/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
| 1 |
# Bagel-Robotics-1
|
| 2 |
|
| 3 |
-
**Contributor:** Bagel Labs | **Paper:** Technical report in preparation | **arXiv:** Not yet available | **Original code:** [Bagel-Robotics-1 implementation](.)
|
| 4 |
|
| 5 |
-
Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is
|
| 6 |
|
| 7 |
Shared conventions — argument meanings, checkpoint naming, split-machine deployment, `EVAL_ENV_TYPE` — are documented in the [XPolicyLab README](https://github.com/XPolicyLab/XPolicyLab/blob/main/README.md). Official results: [RoboDojo LeaderBoard](https://robodojo-benchmark.com/LeaderBoard).
|
| 8 |
|
|
@@ -26,7 +26,7 @@ PYTHON_BIN=python3.10 bash install.sh "$BAGEL_POLICY_ENV"
|
|
| 26 |
|
| 27 |
Keep these variables in the same shell. The installer sets up the standalone runtime, XPolicyLab and PyTorch 2.11 / torchvision 0.26 CUDA 13.0 wheels; `TORCH_INDEX_URL` overrides the wheel index. Tested setup: Python 3.10.21, PyTorch 2.11.0+cu130, RTX PRO 6000 Blackwell (96 GB), 128 GiB host RAM and eight CPUs.
|
| 28 |
|
| 29 |
-
The [checkpoint repository](https://huggingface.co/bageldotcom/Bagel-Robotics-1) is public and ungated. No
|
| 30 |
|
| 31 |
```bash
|
| 32 |
PYTHON_BIN="$BAGEL_POLICY_ENV/bin/python" bash download_checkpoint.sh
|
|
@@ -36,11 +36,11 @@ Allow approximately **45.7 GB** for checkpoint and encoder assets, plus cache an
|
|
| 36 |
|
| 37 |
## Data Processing
|
| 38 |
|
| 39 |
-
Inference receives decoded RGB observations from XPolicyLab. No conversion is required, so `process_data.sh` is omitted
|
| 40 |
|
| 41 |
## Training
|
| 42 |
|
| 43 |
-
This
|
| 44 |
|
| 45 |
## Evaluation
|
| 46 |
|
|
@@ -110,4 +110,4 @@ The server launcher applies the recorded deterministic backend settings. Direct
|
|
| 110 |
|
| 111 |
## Notes
|
| 112 |
|
| 113 |
-
|
|
|
|
| 1 |
# Bagel-Robotics-1
|
| 2 |
|
| 3 |
+
**Contributor:** Bagel Labs | **Paper:** Technical report in preparation | **arXiv:** Not yet available | **Original code:** [Bagel-Robotics-1 implementation](https://github.com/bageldotcom/Bagel-Robotics-1)
|
| 4 |
|
| 5 |
+
Bagel-Robotics-1 combines mask-based structured pruning, recovery training and task-critical data selection to produce a compact WAM with nonuniform FFN widths and cross-attention head counts. Its inference pipeline predicts 32 actions, executes 16 before replanning, and applies 90/10 temporal ensembling. VPP2 is used only as the base model for validation; the training and inference recipe is specific to Bagel-Robotics-1.
|
| 6 |
|
| 7 |
Shared conventions — argument meanings, checkpoint naming, split-machine deployment, `EVAL_ENV_TYPE` — are documented in the [XPolicyLab README](https://github.com/XPolicyLab/XPolicyLab/blob/main/README.md). Official results: [RoboDojo LeaderBoard](https://robodojo-benchmark.com/LeaderBoard).
|
| 8 |
|
|
|
|
| 26 |
|
| 27 |
Keep these variables in the same shell. The installer sets up the standalone runtime, XPolicyLab and PyTorch 2.11 / torchvision 0.26 CUDA 13.0 wheels; `TORCH_INDEX_URL` overrides the wheel index. Tested setup: Python 3.10.21, PyTorch 2.11.0+cu130, RTX PRO 6000 Blackwell (96 GB), 128 GiB host RAM and eight CPUs.
|
| 28 |
|
| 29 |
+
The [checkpoint repository](https://huggingface.co/bageldotcom/Bagel-Robotics-1) is public and ungated. No Hugging Face account or access token is required. Download and verify all assets:
|
| 30 |
|
| 31 |
```bash
|
| 32 |
PYTHON_BIN="$BAGEL_POLICY_ENV/bin/python" bash download_checkpoint.sh
|
|
|
|
| 36 |
|
| 37 |
## Data Processing
|
| 38 |
|
| 39 |
+
Inference receives decoded RGB observations from XPolicyLab. No conversion is required, so `process_data.sh` is omitted from this eval-only adapter. Training uses selected segments from the RoboDojo EE16 LeRobot v3.0 dataset. The public [data preparation guide](https://github.com/bageldotcom/Bagel-Robotics-1/blob/main/docs/DATA.md) provides the pinned dataset, conversion instructions and exact selection manifests.
|
| 40 |
|
| 41 |
## Training
|
| 42 |
|
| 43 |
+
This XPolicyLab adapter remains **eval-only**; `train.sh` is omitted. Dataset selection, mask learning, physical pruning and recovery code are publicly available in [Bagel-Robotics-1](https://github.com/bageldotcom/Bagel-Robotics-1). Follow the [training guide](https://github.com/bageldotcom/Bagel-Robotics-1/blob/main/docs/TRAINING.md) for configurations, required inputs and commands. The technical report remains planned for **November 20, 2026**. Maintainer agreement on the eval-only adapter is requested.
|
| 44 |
|
| 45 |
## Evaluation
|
| 46 |
|
|
|
|
| 110 |
|
| 111 |
## Notes
|
| 112 |
|
| 113 |
+
Three-seed local results (seeds **0, 1 and 2**): **31.59% success / 38.36 mean partial out of 100**, equally weighted across five suites. The report covers 42 tasks and 6,300 outcomes, including labeled replacements and one repaired Memory variant. [Detailed results](EVALUATION_RESULTS.md) contains all suite/task scores, the official summary and the replacement appendix.
|
inference/policy/Bagel_Robotics_1/REEVALUATION.md
CHANGED
|
@@ -2,17 +2,19 @@
|
|
| 2 |
|
| 3 |
Use the [README quickstart](README.md#installation) to install Python 3.10 policy
|
| 4 |
dependencies, download and verify the pinned files, and pass both
|
| 5 |
-
debug checks. The standalone inference runtime is included
|
| 6 |
-
training repository is
|
|
|
|
| 7 |
|
| 8 |
The only policy checkpoint is `cp1000`, from
|
| 9 |
[bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1),
|
| 10 |
revision `2651631c5d08819088475e8778676f5cf6d9afcf`. The downloader and loader reject
|
| 11 |
files that do not match `artifact-manifest.json`. The VAE, image/text encoders and
|
| 12 |
tokenizer are downloaded automatically at their separately pinned public revision.
|
| 13 |
-
The checkpoint
|
| 14 |
-
|
| 15 |
-
|
|
|
|
| 16 |
|
| 17 |
## What a reviewer can reproduce
|
| 18 |
|
|
@@ -23,8 +25,8 @@ two recorded observation sequences. The full five-suite research score table has
|
|
| 23 |
not been rerun end to end through this packaged adapter.
|
| 24 |
|
| 25 |
The commands below start a fresh official evaluation using supplied original
|
| 26 |
-
layouts. They do not recreate the research run's
|
| 27 |
-
|
| 28 |
comparing score totals; retain startup failures for committee review. The public
|
| 29 |
comparison receipts describe the tested scope, but the original research runner
|
| 30 |
and raw comparison fixtures are not included in this PR. They are not required to
|
|
@@ -129,8 +131,8 @@ episodes or combined with the two-episode smoke results.
|
|
| 129 |
The default reevaluation uses original supplied layouts. If a scene fails before
|
| 130 |
any robot action, preserve its startup error and retry history; do not silently
|
| 131 |
replace the scene or score an infrastructure failure as a policy failure. Seek
|
| 132 |
-
the committee's handling decision. The historical research run
|
| 133 |
-
|
| 134 |
[EVALUATION_RESULTS.md](EVALUATION_RESULTS.md); those exceptions are not activated
|
| 135 |
by these commands. Official verification and acceptance remain with RoboDojo.
|
| 136 |
|
|
@@ -138,7 +140,7 @@ by these commands. Official verification and acceptance remain with RoboDojo.
|
|
| 138 |
|
| 139 |
| Symptom | Check |
|
| 140 |
|---|---|
|
| 141 |
-
| HTTP 401/403 or repository not found |
|
| 142 |
| Hash/size mismatch | Check the pinned revision and local file completeness; do not disable verification. |
|
| 143 |
| Missing encoder or tokenizer | Run the full downloader; `--metadata-only` does not fetch tensors or encoders. |
|
| 144 |
| Missing `env_cfg` or simulator | Complete the official workspace setup; `env_cfg/` belongs beside XPolicyLab. |
|
|
|
|
| 2 |
|
| 3 |
Use the [README quickstart](README.md#installation) to install Python 3.10 policy
|
| 4 |
dependencies, download and verify the pinned files, and pass both
|
| 5 |
+
debug checks. The standalone inference runtime is included. The public
|
| 6 |
+
[training repository](https://github.com/bageldotcom/Bagel-Robotics-1) is optional for
|
| 7 |
+
model reevaluation. Install RoboDojo and its assets separately.
|
| 8 |
|
| 9 |
The only policy checkpoint is `cp1000`, from
|
| 10 |
[bageldotcom/Bagel-Robotics-1](https://huggingface.co/bageldotcom/Bagel-Robotics-1),
|
| 11 |
revision `2651631c5d08819088475e8778676f5cf6d9afcf`. The downloader and loader reject
|
| 12 |
files that do not match `artifact-manifest.json`. The VAE, image/text encoders and
|
| 13 |
tokenizer are downloaded automatically at their separately pinned public revision.
|
| 14 |
+
The checkpoint and required encoder assets are public and ungated. No Hugging
|
| 15 |
+
Face account, access token or evaluator access approval is required. Run the
|
| 16 |
+
downloader directly as shown in the README; the pinned revisions and checksum
|
| 17 |
+
checks are unchanged.
|
| 18 |
|
| 19 |
## What a reviewer can reproduce
|
| 20 |
|
|
|
|
| 25 |
not been rerun end to end through this packaged adapter.
|
| 26 |
|
| 27 |
The commands below start a fresh official evaluation using supplied original
|
| 28 |
+
layouts. They do not recreate the research run's historical scene substitutions or repaired
|
| 29 |
+
Memory variant. Compare episode selection and seed coverage before
|
| 30 |
comparing score totals; retain startup failures for committee review. The public
|
| 31 |
comparison receipts describe the tested scope, but the original research runner
|
| 32 |
and raw comparison fixtures are not included in this PR. They are not required to
|
|
|
|
| 131 |
The default reevaluation uses original supplied layouts. If a scene fails before
|
| 132 |
any robot action, preserve its startup error and retry history; do not silently
|
| 133 |
replace the scene or score an infrastructure failure as a policy failure. Seek
|
| 134 |
+
the committee's handling decision. The historical research run includes 28 later replacement layouts, one repaired
|
| 135 |
+
Memory variant and four earlier Seed-1 Precision substitutions, described in
|
| 136 |
[EVALUATION_RESULTS.md](EVALUATION_RESULTS.md); those exceptions are not activated
|
| 137 |
by these commands. Official verification and acceptance remain with RoboDojo.
|
| 138 |
|
|
|
|
| 140 |
|
| 141 |
| Symptom | Check |
|
| 142 |
|---|---|
|
| 143 |
+
| HTTP 401/403 or repository not found | Confirm the canonical repository and pinned revision above. Both repositories are public; no token is required. Retry with `HF_HUB_DISABLE_IMPLICIT_TOKEN=1` to exclude stale cached credentials, and check network/proxy access if the error persists. |
|
| 144 |
| Hash/size mismatch | Check the pinned revision and local file completeness; do not disable verification. |
|
| 145 |
| Missing encoder or tokenizer | Run the full downloader; `--metadata-only` does not fetch tensors or encoders. |
|
| 146 |
| Missing `env_cfg` or simulator | Complete the official workspace setup; `env_cfg/` belongs beside XPolicyLab. |
|
inference/policy/Bagel_Robotics_1/THIRD_PARTY_NOTICES.md
DELETED
|
@@ -1,13 +0,0 @@
|
|
| 1 |
-
# Source and artifact notices
|
| 2 |
-
|
| 3 |
-
- Reused source components from [VPP2](https://github.com/roboterax/video-prediction-policy-2)/FastWAM retain the MIT notice in
|
| 4 |
-
`LICENSE`, copyright 2026 The FastWAM Authors. See `source-provenance.json`
|
| 5 |
-
for the source identities and packaging changes.
|
| 6 |
-
- Wan2.1/DiffSynth-derived model components retain Apache-2.0 terms; see
|
| 7 |
-
`licenses/Apache-2.0.txt` and upstream Wan2.1/DiffSynth source notices.
|
| 8 |
-
- The XPolicyLab interface follows its upstream license, included in
|
| 9 |
-
`licenses/XPolicyLab.txt`. RoboDojo and simulator assets are not bundled.
|
| 10 |
-
- Encoder weights, tokenizer files and model weights have their own upstream
|
| 11 |
-
terms. The code license does not assign a new license to those artifacts.
|
| 12 |
-
- The existing preview PR by Soroush Mehraban was consulted as a reference; its
|
| 13 |
-
alternate checkpoint, code and release promises are not this submission.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
inference/policy/Bagel_Robotics_1/VALIDATION.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
# Adapter validation
|
| 2 |
|
| 3 |
-
Status: **Technical qualification passed: 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes, and all eight exact replay/backend checks. [PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234) is open. The
|
| 4 |
|
| 5 |
| Check | Evidence and scope |
|
| 6 |
|---|---|
|
|
@@ -37,11 +37,21 @@ audit. CPU checks used Python 3.12.7 / PyTorch 2.11.0+cpu with existing test
|
|
| 37 |
dependencies; they do not replace the earlier fresh Python 3.10.21 / CUDA 13.0
|
| 38 |
installation and GPU qualification. Inference code and settings are unchanged.
|
| 39 |
|
| 40 |
-
|
| 41 |
-
|
| 42 |
GPU types, and a full five-suite packaged-adapter run have not been qualified by
|
| 43 |
this audit.
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
## Reproduce the transport checks
|
| 46 |
|
| 47 |
Use the policy README's debug command with the actual checkpoint. Repeat it with `DEBUG_OBS_ENCODED=1`. Both runs must finish normally and print the evaluation-finished marker.
|
|
|
|
| 1 |
# Adapter validation
|
| 2 |
|
| 3 |
+
Status: **Technical qualification passed: 15 CPU tests, both real-checkpoint debug transports, four valid native simulator episodes, and all eight exact replay/backend checks. [PR #234](https://github.com/XPolicyLab/XPolicyLab/pull/234) is open. The checkpoint is public and ungated; no evaluator credential handoff is required. The technical report remains planned for November 20, 2026. Full official benchmark verification remains pending.**
|
| 4 |
|
| 5 |
| Check | Evidence and scope |
|
| 6 |
|---|---|
|
|
|
|
| 37 |
dependencies; they do not replace the earlier fresh Python 3.10.21 / CUDA 13.0
|
| 38 |
installation and GPU qualification. Inference code and settings are unchanged.
|
| 39 |
|
| 40 |
+
That audit used authenticated access; the checkpoint is now public, as verified
|
| 41 |
+
below. A complete fresh-machine simulator/assets installation, different
|
| 42 |
GPU types, and a full five-suite packaged-adapter run have not been qualified by
|
| 43 |
this audit.
|
| 44 |
|
| 45 |
+
## Public checkpoint access (November 7, 2026)
|
| 46 |
+
|
| 47 |
+
The checkpoint and all required encoder/tokenizer assets are publicly accessible
|
| 48 |
+
without authentication. The published metadata downloader passed in an isolated
|
| 49 |
+
empty cache with the submission-pinned `huggingface-hub==0.36.0` and no token.
|
| 50 |
+
All four checkpoint JSON files matched their SHA-256 checksums; all 15 asset URLs
|
| 51 |
+
passed anonymous ranged downloads and matched pinned remote identities and sizes.
|
| 52 |
+
The checkpoint revision, hashes and inference settings are unchanged. Evidence:
|
| 53 |
+
[public-checkpoint-access.json](evidence/public-checkpoint-access.json).
|
| 54 |
+
|
| 55 |
## Reproduce the transport checks
|
| 56 |
|
| 57 |
Use the policy README's debug command with the actual checkpoint. Repeat it with `DEBUG_OBS_ENCODED=1`. Both runs must finish normally and print the evaluation-finished marker.
|
inference/policy/Bagel_Robotics_1/artifact-manifest.json
CHANGED
|
@@ -6,7 +6,7 @@
|
|
| 6 |
"video_parameters": 12468409664,
|
| 7 |
"repo_id": "bageldotcom/Bagel-Robotics-1",
|
| 8 |
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 9 |
-
"private":
|
| 10 |
"files": {
|
| 11 |
"COMPLETE.json": {
|
| 12 |
"sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
|
|
|
|
| 6 |
"video_parameters": 12468409664,
|
| 7 |
"repo_id": "bageldotcom/Bagel-Robotics-1",
|
| 8 |
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 9 |
+
"private": false,
|
| 10 |
"files": {
|
| 11 |
"COMPLETE.json": {
|
| 12 |
"sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
|
inference/policy/Bagel_Robotics_1/download_checkpoint.py
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
"""Download the pinned
|
| 2 |
from pathlib import Path
|
| 3 |
import argparse
|
| 4 |
import json
|
|
|
|
| 1 |
+
"""Download the pinned public CP1000 artifact and encoder assets; no token required."""
|
| 2 |
from pathlib import Path
|
| 3 |
import argparse
|
| 4 |
import json
|
inference/policy/Bagel_Robotics_1/evaluation/seed-1/results-breakdown.md
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Bagel-Robotics-1 — complete reported results
|
| 2 |
+
|
| 3 |
+
Checkpoint 1000, **evaluation seed 1**. This companion report contains the previously reported five-suite totals and all 42 task results. The tables below are a readable companion to the unchanged [native summary](summary.md); all 42 tasks have been verified against retained per-episode native records. Success means full task completion; partial scores measure credited progress, including in failed episodes.
|
| 4 |
+
|
| 5 |
+
All 2,100 reported outcomes were checked against retained native result files and completion records: 1,350 from Node 2, plus 450 Generalization outcomes and 300 reported Memory outcomes from the retained Nebius archive. The Memory archive contains 299 valid originals and a separately labeled repaired variant.
|
| 6 |
+
|
| 7 |
+
| Suite | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 8 |
+
|---|---:|---:|---:|
|
| 9 |
+
| Precision | 126/400 | 31.50% | 40.43 |
|
| 10 |
+
| Memory | 173/300 | 57.67% | 58.00 |
|
| 11 |
+
| Long Horizon | 137/400 | 34.25% | 53.00 |
|
| 12 |
+
| Generalization | 183/600 | 30.50% | 36.82 |
|
| 13 |
+
| Open | 16/400 | 4.00% | 4.48 |
|
| 14 |
+
| **Equal-weight five-suite average** | **635/2,100 total** | **31.58%** | **38.55** |
|
| 15 |
+
|
| 16 |
+
The overall score is the mean of the five suite scores, not successes divided by all episodes.
|
| 17 |
+
|
| 18 |
+
## Scene selection and sources
|
| 19 |
+
|
| 20 |
+
- Memory: 299 original valid results plus the separately repaired sorting-sequence layout 47 (failure, partial 0). Original-only totals: 173/299 successes (57.86%), partial 58.19/100. The original layout remains unscored.
|
| 21 |
+
- Long Horizon: 398 originals plus supplied egg-holder layouts 50 and 51, replacing unscored originals 15 and 20. The replacements scored 0 and 0.25, both failures.
|
| 22 |
+
- Open: 399 originals plus supplied pickup layout 50, replacing unscored original 5. The replacement failed with partial 0.
|
| 23 |
+
- Generalization: 25 standard plus 25 randomized episodes per task. All 12 tasks were checked against native records: three from Node 2 and nine from the retained Nebius archive. The combined standard/random total is 600 outcomes.
|
| 24 |
+
|
| 25 |
+
“Original” refers to the frozen task selections. Deposit Coin uses supplied layouts 50–53 instead of 4, 8, 39 and 42; Imitate Sorting Sequence uses supplied layouts 50–53 instead of 18, 33, 34 and 44. These selections were fixed before the saved runs.
|
| 26 |
+
|
| 27 |
+
## Task-level results
|
| 28 |
+
|
| 29 |
+
### Precision
|
| 30 |
+
|
| 31 |
+
| Task | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 32 |
+
|---|---:|---:|---:|
|
| 33 |
+
| Build tower | 32/50 | 64.00% | 73.60 |
|
| 34 |
+
| Deposit coin | 11/50 | 22.00% | 26.80 |
|
| 35 |
+
| Fasten screws | 18/50 | 36.00% | 66.80 |
|
| 36 |
+
| Insert key | 0/50 | 0.00% | 15.00 |
|
| 37 |
+
| Insert tubes | 32/50 | 64.00% | 75.20 |
|
| 38 |
+
| Play xylophone | 0/50 | 0.00% | 0.00 |
|
| 39 |
+
| Plug in charger | 2/50 | 4.00% | 4.00 |
|
| 40 |
+
| Pour balls into vase | 31/50 | 62.00% | 62.00 |
|
| 41 |
+
|
| 42 |
+
### Memory
|
| 43 |
+
|
| 44 |
+
| Task | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 45 |
+
|---|---:|---:|---:|
|
| 46 |
+
| Cover blocks | 50/50 | 100.00% | 100.00 |
|
| 47 |
+
| Imitate sorting sequence | 0/50 | 0.00% | 2.00 |
|
| 48 |
+
| Match and pick from conveyor | 46/50 | 92.00% | 92.00 |
|
| 49 |
+
| Press by number | 50/50 | 100.00% | 100.00 |
|
| 50 |
+
| Swap T | 11/50 | 22.00% | 22.00 |
|
| 51 |
+
| Swap blocks | 16/50 | 32.00% | 32.00 |
|
| 52 |
+
|
| 53 |
+
### Long Horizon
|
| 54 |
+
|
| 55 |
+
| Task | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 56 |
+
|---|---:|---:|---:|
|
| 57 |
+
| Classify objects | 16/50 | 32.00% | 44.50 |
|
| 58 |
+
| Fill egg holder | 2/50 | 4.00% | 13.80 |
|
| 59 |
+
| Fill pen holder | 11/50 | 22.00% | 43.90 |
|
| 60 |
+
| Make kong | 43/50 | 86.00% | 86.00 |
|
| 61 |
+
| Organize table | 9/50 | 18.00% | 64.50 |
|
| 62 |
+
| Play stacking toy | 6/50 | 12.00% | 32.00 |
|
| 63 |
+
| Play tic tac toe | 1/50 | 2.00% | 40.50 |
|
| 64 |
+
| Put bottles into dustbin | 49/50 | 98.00% | 98.80 |
|
| 65 |
+
|
| 66 |
+
### Generalization
|
| 67 |
+
|
| 68 |
+
| Task | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 69 |
+
|---|---:|---:|---:|
|
| 70 |
+
| Arrange largest number | 2/50 | 4.00% | 11.00 |
|
| 71 |
+
| Fold clothes | 36/50 | 72.00% | 74.40 |
|
| 72 |
+
| Hang mugs | 14/50 | 28.00% | 38.50 |
|
| 73 |
+
| Make toast | 22/50 | 44.00% | 52.50 |
|
| 74 |
+
| Pack objects into box | 6/50 | 12.00% | 35.40 |
|
| 75 |
+
| Pour liquid into cup | 22/50 | 44.00% | 44.00 |
|
| 76 |
+
| Push T | 1/50 | 2.00% | 2.00 |
|
| 77 |
+
| Sort nesting dolls by size | 10/50 | 20.00% | 20.00 |
|
| 78 |
+
| Stack blocks | 22/50 | 44.00% | 50.00 |
|
| 79 |
+
| Stack bowls | 26/50 | 52.00% | 58.90 |
|
| 80 |
+
| Store laptop and headphones | 20/50 | 40.00% | 51.20 |
|
| 81 |
+
| Sweep blocks | 2/50 | 4.00% | 4.00 |
|
| 82 |
+
|
| 83 |
+
### Open
|
| 84 |
+
|
| 85 |
+
| Task | Successes / reported outcomes | Success rate | Mean partial /100 |
|
| 86 |
+
|---|---:|---:|---:|
|
| 87 |
+
| Align blocks | 0/50 | 0.00% | 0.00 |
|
| 88 |
+
| Classify objects by language | 0/50 | 0.00% | 2.60 |
|
| 89 |
+
| General pickup | 12/50 | 24.00% | 24.00 |
|
| 90 |
+
| Pick from conveyor by image | 0/50 | 0.00% | 0.00 |
|
| 91 |
+
| Pour by language | 0/50 | 0.00% | 0.00 |
|
| 92 |
+
| Solve equation | 0/50 | 0.00% | 0.00 |
|
| 93 |
+
| Stack blocks by language | 4/50 | 8.00% | 9.20 |
|
| 94 |
+
| Store tools in toolbox | 0/50 | 0.00% | 0.00 |
|
| 95 |
+
|
| 96 |
+
Checkpoint identity: `790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c`.
|
| 97 |
+
|
| 98 |
+
## Reading the native summary
|
| 99 |
+
|
| 100 |
+
[summary.md](summary.md) is the unchanged output of `bash scripts/robodojo.sh summarize` at [RoboDojo revision f57e163](https://github.com/RoboDojo-Benchmark/RoboDojo/tree/f57e1633ffed59761bc8e568cfe7530c4b33381c), generated from the retained native records. [verification.json](verification.json) records its SHA-256 and the completed checks.
|
| 101 |
+
|
| 102 |
+
All 42 tasks are reported for seed 1. The native summary shows 42/126 (33.3%) because its denominator includes seeds 0, 1 and 2; the other seed columns are blank. Its `±0.00` reflects one available seed, not measured variation across seeds. The native script's rounding can differ by 0.01 from the readable tables above; the underlying episode scores agree. The summary's `with_labeled_replacements_and_repair` suffix identifies the reported scene selection; the model name is Bagel-Robotics-1.
|
inference/policy/Bagel_Robotics_1/evaluation/seed-1/summary.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RoboDojo Evaluation Summary
|
| 2 |
+
|
| 3 |
+
## Overview (by capability dimension)
|
| 4 |
+
|
| 5 |
+
Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
|
| 6 |
+
|
| 7 |
+
| Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
|
| 8 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 9 |
+
| Bagel-Robotics-1_with_labeled_replacements_and_repair | 33.3% † (42/126) | 30.50 ±0.00 | 36.82 ±0.00 | 31.50 ±0.00 | 40.42 ±0.00 | 34.25 ±0.00 | 53.00 ±0.00 | 57.67 ±0.00 | 58.00 ±0.00 | 4.00 ±0.00 | 4.47 ±0.00 | 31.58 ±0.00 | 38.55 ±0.00 |
|
| 10 |
+
|
| 11 |
+
## Generalization: Standard vs Random (25 episodes each)
|
| 12 |
+
|
| 13 |
+
Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
|
| 14 |
+
|
| 15 |
+
| Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
|
| 16 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 17 |
+
| Bagel-Robotics-1_with_labeled_replacements_and_repair | 49.33 ±0.00 | 54.88 ±0.00 | 11.67 ±0.00 | 18.77 ±0.00 | -37.67 | -36.12 |
|
| 18 |
+
|
| 19 |
+
### Per-task Standard vs Random
|
| 20 |
+
|
| 21 |
+
Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
|
| 22 |
+
|
| 23 |
+
#### Bagel-Robotics-1_with_labeled_replacements_and_repair
|
| 24 |
+
|
| 25 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 26 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 27 |
+
| <nobr>stack_bowls</nobr> | | | 92 | 93.20 | | | | |
|
| 28 |
+
| <nobr>stack_bowls_random</nobr> | | | 12 | 24.60 | | | | |
|
| 29 |
+
| <nobr>push_T</nobr> | | | 4 | 4.00 | | | | |
|
| 30 |
+
| <nobr>push_T_random</nobr> | | | 0 | 0.00 | | | | |
|
| 31 |
+
| <nobr>pack_objects_into_box</nobr> | | | 16 | 42.60 | | | | |
|
| 32 |
+
| <nobr>pack_objects_into_box_random</nobr> | | | 8 | 28.20 | | | | |
|
| 33 |
+
| <nobr>fold_clothes</nobr> | | | 80 | 80.80 | | | | |
|
| 34 |
+
| <nobr>fold_clothes_random</nobr> | | | 64 | 68.00 | | | | |
|
| 35 |
+
| <nobr>hang_mugs</nobr> | | | 56 | 67.40 | | | | |
|
| 36 |
+
| <nobr>hang_mugs_random</nobr> | | | 0 | 9.60 | | | | |
|
| 37 |
+
| <nobr>sweep_blocks</nobr> | | | 8 | 8.00 | | | | |
|
| 38 |
+
| <nobr>sweep_blocks_random</nobr> | | | 0 | 0.00 | | | | |
|
| 39 |
+
| <nobr>pour_liquid_into_cup</nobr> | | | 68 | 68.00 | | | | |
|
| 40 |
+
| <nobr>pour_liquid_into_cup_random</nobr> | | | 20 | 20.00 | | | | |
|
| 41 |
+
| <nobr>make_toast</nobr> | | | 72 | 79.00 | | | | |
|
| 42 |
+
| <nobr>make_toast_random</nobr> | | | 16 | 26.00 | | | | |
|
| 43 |
+
| <nobr>arrange_largest_number</nobr> | | | 4 | 13.00 | | | | |
|
| 44 |
+
| <nobr>arrange_largest_number_random</nobr> | | | 4 | 9.00 | | | | |
|
| 45 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | | | 36 | 36.00 | | | | |
|
| 46 |
+
| <nobr>sort_nesting_dolls_by_size_random</nobr> | | | 4 | 4.00 | | | | |
|
| 47 |
+
| <nobr>store_laptop_and_headphones</nobr> | | | 68 | 76.80 | | | | |
|
| 48 |
+
| <nobr>store_laptop_and_headphones_random</nobr> | | | 12 | 25.60 | | | | |
|
| 49 |
+
| <nobr>stack_blocks</nobr> | | | 88 | 89.80 | | | | |
|
| 50 |
+
| <nobr>stack_blocks_random</nobr> | | | 0 | 10.20 | | | | |
|
| 51 |
+
| **Avg Standard** | | | 49.3 | 54.88 | | | 49 ±0 | 54.88 ±0.00 |
|
| 52 |
+
| **Avg Random** | | | 11.7 | 18.77 | | | 12 ±0 | 18.77 ±0.00 |
|
| 53 |
+
|
| 54 |
+
## Bagel-Robotics-1_with_labeled_replacements_and_repair
|
| 55 |
+
|
| 56 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 57 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 58 |
+
| <nobr>align_blocks</nobr> | | | 0 | 0.00 | | | | |
|
| 59 |
+
| <nobr>arrange_largest_number</nobr> | | | 4 | 11.00 | | | | |
|
| 60 |
+
| <nobr>build_tower</nobr> | | | 64 | 73.60 | | | | |
|
| 61 |
+
| <nobr>classify_objects</nobr> | | | 32 | 44.50 | | | | |
|
| 62 |
+
| <nobr>classify_objects_by_language</nobr> | | | 0 | 2.60 | | | | |
|
| 63 |
+
| <nobr>cover_blocks</nobr> | | | 100 | 100.00 | | | | |
|
| 64 |
+
| <nobr>deposit_coin</nobr> | | | 22 | 26.80 | | | | |
|
| 65 |
+
| <nobr>fasten_screws</nobr> | | | 36 | 66.80 | | | | |
|
| 66 |
+
| <nobr>fill_egg_holder</nobr> | | | 4 | 13.80 | | | | |
|
| 67 |
+
| <nobr>fill_pen_holder</nobr> | | | 22 | 43.90 | | | | |
|
| 68 |
+
| <nobr>fold_clothes</nobr> | | | 72 | 74.40 | | | | |
|
| 69 |
+
| <nobr>general_pickup</nobr> | | | 24 | 24.00 | | | | |
|
| 70 |
+
| <nobr>hang_mugs</nobr> | | | 28 | 38.50 | | | | |
|
| 71 |
+
| <nobr>imitate_sorting_sequence</nobr> | | | 0 | 2.00 | | | | |
|
| 72 |
+
| <nobr>insert_key</nobr> | | | 0 | 15.00 | | | | |
|
| 73 |
+
| <nobr>insert_tubes</nobr> | | | 64 | 75.20 | | | | |
|
| 74 |
+
| <nobr>make_kong</nobr> | | | 86 | 86.00 | | | | |
|
| 75 |
+
| <nobr>make_toast</nobr> | | | 44 | 52.50 | | | | |
|
| 76 |
+
| <nobr>match_and_pick_from_conveyor</nobr> | | | 92 | 92.00 | | | | |
|
| 77 |
+
| <nobr>organize_table</nobr> | | | 18 | 64.50 | | | | |
|
| 78 |
+
| <nobr>pack_objects_into_box</nobr> | | | 12 | 35.40 | | | | |
|
| 79 |
+
| <nobr>pick_from_conveyor_by_image</nobr> | | | 0 | 0.00 | | | | |
|
| 80 |
+
| <nobr>play_Xylophone</nobr> | | | 0 | 0.00 | | | | |
|
| 81 |
+
| <nobr>play_stacking_toy</nobr> | | | 12 | 32.00 | | | | |
|
| 82 |
+
| <nobr>play_tic_tac_toe</nobr> | | | 2 | 40.50 | | | | |
|
| 83 |
+
| <nobr>plug_in_charger</nobr> | | | 4 | 4.00 | | | | |
|
| 84 |
+
| <nobr>pour_balls_into_vase</nobr> | | | 62 | 62.00 | | | | |
|
| 85 |
+
| <nobr>pour_by_language</nobr> | | | 0 | 0.00 | | | | |
|
| 86 |
+
| <nobr>pour_liquid_into_cup</nobr> | | | 44 | 44.00 | | | | |
|
| 87 |
+
| <nobr>press_by_number</nobr> | | | 100 | 100.00 | | | | |
|
| 88 |
+
| <nobr>push_T</nobr> | | | 2 | 2.00 | | | | |
|
| 89 |
+
| <nobr>put_bottles_into_dustbin</nobr> | | | 98 | 98.80 | | | | |
|
| 90 |
+
| <nobr>solve_equation</nobr> | | | 0 | 0.00 | | | | |
|
| 91 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | | | 20 | 20.00 | | | | |
|
| 92 |
+
| <nobr>stack_blocks</nobr> | | | 44 | 50.00 | | | | |
|
| 93 |
+
| <nobr>stack_blocks_by_language</nobr> | | | 8 | 9.20 | | | | |
|
| 94 |
+
| <nobr>stack_bowls</nobr> | | | 52 | 58.90 | | | | |
|
| 95 |
+
| <nobr>store_laptop_and_headphones</nobr> | | | 40 | 51.20 | | | | |
|
| 96 |
+
| <nobr>store_tools_in_toolbox</nobr> | | | 0 | 0.00 | | | | |
|
| 97 |
+
| <nobr>swap_T</nobr> | | | 22 | 22.00 | | | | |
|
| 98 |
+
| <nobr>swap_blocks</nobr> | | | 32 | 32.00 | | | | |
|
| 99 |
+
| <nobr>sweep_blocks</nobr> | | | 4 | 4.00 | | | | |
|
| 100 |
+
| **Avg (per seed)** | | | 30.2 | 37.45 | | | 30 ±0 | 37.45 ±0.00 |
|
| 101 |
+
|
inference/policy/Bagel_Robotics_1/evaluation/seed-1/verification.json
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"revision": "f57e1633ffed59761bc8e568cfe7530c4b33381c",
|
| 3 |
+
"files": {
|
| 4 |
+
"scripts/robodojo.sh": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3",
|
| 5 |
+
"scripts/internal/summarize_result.py": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2"
|
| 6 |
+
},
|
| 7 |
+
"unmodified_official_sources_verified": true,
|
| 8 |
+
"source": "https://github.com/RoboDojo-Benchmark/RoboDojo",
|
| 9 |
+
"generated_records": 2100,
|
| 10 |
+
"matching_reported_tasks": 42,
|
| 11 |
+
"sha256_of_native_summary": "a2946a8c62a7cf3d1e64d8e8655d462533e2a3937b49756bbd458ab81586cad9",
|
| 12 |
+
"native_markdown_task_cells_verified": 42,
|
| 13 |
+
"native_summary_copy_identical": true,
|
| 14 |
+
"original_selected_records": 2096,
|
| 15 |
+
"labeled_replacements": 3,
|
| 16 |
+
"separate_repaired_variants": 1,
|
| 17 |
+
"memory_original_layout47_unscored": true,
|
| 18 |
+
"nebius_retention_verified": true,
|
| 19 |
+
"clean_offline_regeneration_identical": true,
|
| 20 |
+
"all_scores_match_previously_reported": true
|
| 21 |
+
}
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/.gitattributes
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Preserve the official summarizer's trailing blank line byte-for-byte.
|
| 2 |
+
summary.md whitespace=-blank-at-eof
|
| 3 |
+
summary-original-selection.md whitespace=-blank-at-eof
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/README.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Three-seed evaluation data
|
| 2 |
+
|
| 3 |
+
This directory retains the unchanged official summaries, episode data, verification records and regeneration script. Results and reproduction instructions are now in one report.
|
| 4 |
+
|
| 5 |
+
[Open the consolidated results report](../../EVALUATION_RESULTS.md#reproduction-files).
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/SHA256SUMS
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
a79a06b50af6ef7df8f313c0cb84e954f9a328f4eac1c394216de0681ade251f .gitattributes
|
| 2 |
+
c965c48fbaea7e1dd5852ff556d11c441a4897245a9b132fe55afbbdd1488314 README.md
|
| 3 |
+
6ed678932036e827103cd89de0c4748515c54b178eb7862c201eb2976d28af21 episode-scores.csv
|
| 4 |
+
0b19a2cc51364baaacb070c5245eef274b5e9b4393b550ca30365fb4d0c603fb generate_summary.py
|
| 5 |
+
7a2ac06a26b8496271a4e5c1d334ed9693d1b96e8058b99834e5e52eaa46d842 original-only-breakdown.md
|
| 6 |
+
b5284fdaf2d0eb3dd7dfc36e91d2842183d4695b2317c1338f145ed78694db57 protocols.json
|
| 7 |
+
5e710be09f95a3200c1c33755f667b9f95797b4ef5d696d3ee006d3dc6e09f92 replacements-and-repairs.csv
|
| 8 |
+
2a7fb1a4ba1d8374b4f0317644f75ba360db17f43a0f22a3c5da50cf02da3544 replacements-and-repairs.md
|
| 9 |
+
c0fe627bdbde293db26714658eac7695d2f6cb5d6e66842a3d29f93edcc70c66 results-breakdown.md
|
| 10 |
+
147306c5805b47d1ecaa7179db7f9a15b1b682e492630e6ab685160b4c23ab63 retried-originals.csv
|
| 11 |
+
c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5 summary-original-selection.md
|
| 12 |
+
141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f summary.md
|
| 13 |
+
ab9c45c7766c3760304aeb53c517dbc7ccebc97f970016d4a845ccf2df936141 unscored-episodes.csv
|
| 14 |
+
958dbd046ff0d94562a5281f391ba3dfb7274f7f020a40bf191df87fae1d3044 verification.json
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/episode-scores.csv
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/generate_summary.py
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""Regenerate the unchanged RoboDojo summary from the published score ledger.
|
| 3 |
+
|
| 4 |
+
This groups already-scored outcomes; it does not run or rescore any episode.
|
| 5 |
+
Python standard library only. See README.md for the pinned RoboDojo checkout.
|
| 6 |
+
"""
|
| 7 |
+
import argparse
|
| 8 |
+
import collections
|
| 9 |
+
import csv
|
| 10 |
+
import hashlib
|
| 11 |
+
import json
|
| 12 |
+
import math
|
| 13 |
+
import os
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
import shutil
|
| 16 |
+
import subprocess
|
| 17 |
+
|
| 18 |
+
SCRIPTS = {
|
| 19 |
+
"scripts/robodojo.sh": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3",
|
| 20 |
+
"scripts/internal/summarize_result.py": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2",
|
| 21 |
+
}
|
| 22 |
+
POLICY = "Bagel-Robotics-1_with_labeled_replacements_and_repair"
|
| 23 |
+
SUITES = {"precision": 400, "memory": 300, "generalization": 600,
|
| 24 |
+
"long_horizon": 400, "open": 400}
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
def main():
|
| 28 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 29 |
+
parser.add_argument("--robodojo-root", type=Path, required=True)
|
| 30 |
+
parser.add_argument("--output-dir", type=Path, required=True,
|
| 31 |
+
help="New or empty output directory; never use a live eval_result directory")
|
| 32 |
+
parser.add_argument("--view", choices=["reported", "original"], default="reported")
|
| 33 |
+
args = parser.parse_args()
|
| 34 |
+
root = args.robodojo_root.resolve()
|
| 35 |
+
for name, expected in SCRIPTS.items():
|
| 36 |
+
if hashlib.sha256((root / name).read_bytes()).hexdigest() != expected:
|
| 37 |
+
parser.error(f"{name} must match RoboDojo f57e1633ffed59761bc8e568cfe7530c4b33381c")
|
| 38 |
+
ledger = Path(__file__).with_name("episode-scores.csv")
|
| 39 |
+
with ledger.open(newline="") as stream:
|
| 40 |
+
rows = list(csv.DictReader(stream))
|
| 41 |
+
if len(rows) != 6300:
|
| 42 |
+
parser.error("Expected 6,300 reported outcomes")
|
| 43 |
+
seen = set()
|
| 44 |
+
for row in rows:
|
| 45 |
+
row["seed"] = int(row["seed"])
|
| 46 |
+
row["layout_id"] = int(row["layout_id"])
|
| 47 |
+
if row["success"] not in ("true", "false"):
|
| 48 |
+
parser.error("Invalid success field")
|
| 49 |
+
row["success"] = row["success"] == "true"
|
| 50 |
+
row["partial_score"] = float(row["partial_score"])
|
| 51 |
+
if not math.isfinite(row["partial_score"]) or not 0 <= row["partial_score"] <= 1:
|
| 52 |
+
parser.error("Invalid native partial score")
|
| 53 |
+
key = row["seed"], row["task"], row["layout_id"]
|
| 54 |
+
if key in seen:
|
| 55 |
+
parser.error(f"Duplicate scored identity: {key}")
|
| 56 |
+
seen.add(key)
|
| 57 |
+
counts = collections.Counter((r["seed"], r["suite"]) for r in rows)
|
| 58 |
+
expected = {(s, suite): count for s in range(3) for suite, count in SUITES.items()}
|
| 59 |
+
if counts != expected:
|
| 60 |
+
parser.error("Seed/suite coverage differs from the published selection")
|
| 61 |
+
selected = rows if args.view == "reported" else [r for r in rows if r["record_type"] == "original"]
|
| 62 |
+
if len(selected) != (6300 if args.view == "reported" else 6271):
|
| 63 |
+
parser.error("Unexpected selection count")
|
| 64 |
+
output = args.output_dir.resolve()
|
| 65 |
+
if output.exists() and any(output.iterdir()):
|
| 66 |
+
parser.error("Output directory must be empty; existing results will not be overwritten")
|
| 67 |
+
output.mkdir(parents=True, exist_ok=True)
|
| 68 |
+
eval_root = output / "eval_result" / "RoboDojo"
|
| 69 |
+
grouped = collections.defaultdict(list)
|
| 70 |
+
for row in selected:
|
| 71 |
+
grouped[row["seed"], row["task"]].append(row)
|
| 72 |
+
policy = POLICY if args.view == "reported" else "Bagel-Robotics-1_original_selection"
|
| 73 |
+
for (seed, task), episodes in sorted(grouped.items()):
|
| 74 |
+
episodes.sort(key=lambda r: r["layout_id"])
|
| 75 |
+
n = len(episodes)
|
| 76 |
+
target = (eval_root / task / policy / "arx_x5" /
|
| 77 |
+
f"{seed}_ckpt_name=cp1000,action_type=ee" / "2026-11-09_00-00-00" / "_result.json")
|
| 78 |
+
target.parent.mkdir(parents=True, exist_ok=True)
|
| 79 |
+
data = {
|
| 80 |
+
"eval_time": n,
|
| 81 |
+
"success_rate": sum(r["success"] for r in episodes) / n,
|
| 82 |
+
"score": 100 * math.fsum(r["partial_score"] for r in episodes) / n,
|
| 83 |
+
"details": {str(r["layout_id"]): {
|
| 84 |
+
"layout_id": r["layout_id"], "success": r["success"], "score": r["partial_score"],
|
| 85 |
+
"record_type": r["record_type"], "replaces_episode": r["replaces_episode"],
|
| 86 |
+
} for r in episodes},
|
| 87 |
+
"_aggregation": {
|
| 88 |
+
"source": "episode-scores.csv", "view": args.view, "checkpoint": 1000,
|
| 89 |
+
"note": "Score-field projection of retained results; outcomes and actual layout IDs unchanged."
|
| 90 |
+
},
|
| 91 |
+
}
|
| 92 |
+
target.write_text(json.dumps(data, indent=2) + "\n")
|
| 93 |
+
env = dict(os.environ, ROBODOJO_EVAL_ROOT=str(eval_root))
|
| 94 |
+
result = subprocess.run(["bash", str(root / "scripts/robodojo.sh"), "summarize"],
|
| 95 |
+
cwd=root, env=env, text=True, stdout=subprocess.PIPE,
|
| 96 |
+
stderr=subprocess.STDOUT, check=False)
|
| 97 |
+
(output / "summarize.log").write_text(result.stdout)
|
| 98 |
+
if result.returncode:
|
| 99 |
+
raise SystemExit(f"Summarizer failed ({result.returncode}); see {output / 'summarize.log'}")
|
| 100 |
+
shutil.copyfile(eval_root / "_summary.md", output / "summary.md")
|
| 101 |
+
print(f"Generated {output / 'summary.md'} from {len(selected)} retained outcomes.")
|
| 102 |
+
print("Score-only export: rollout videos are not copied; native video-check messages remain in summarize.log.")
|
| 103 |
+
|
| 104 |
+
|
| 105 |
+
if __name__ == "__main__":
|
| 106 |
+
main()
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/original-only-breakdown.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Original-selection-only comparisons
|
| 2 |
+
|
| 3 |
+
The original-selection-only suite and affected-task tables are now in the consolidated report appendix.
|
| 4 |
+
|
| 5 |
+
[Open the consolidated results report](../../EVALUATION_RESULTS.md#original-selection-only-comparisons).
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/protocols.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.csv
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed,suite,task,report_task,task_variant,episode,layout_id,record_type,success,partial_score,replaces_episode,attempt_count,protocol,completion_record_sha256
|
| 2 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.4,fill_egg_holder--0028,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,1d93916dbdad621225ec6ffb2db26ee6ed268282df2f14dce9f2ba95b2ae52fc
|
| 3 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0051,51,reserve_replacement,false,0.0,fill_egg_holder--0035,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,4f3ddddc86a5d4f3eafb0be166557bbb19c1117e7c2df55b0608e33aacab0401
|
| 4 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0052,52,reserve_replacement,false,0.0,fill_egg_holder--0037,1,7d5cd70f85fea087a292e933e322e265cfe33c9cefed101cf499349a5a86f000,69c5e4ff890baf37074de001e6c06b49907a02fa7946c22939fc34546119a428
|
| 5 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0050,50,reserve_replacement,false,0.0,imitate_sorting_sequence--0004,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,dd38bacf246a87c684c2e68b7d4e4c7f185ac0547366e5de3a6d22b73f946213
|
| 6 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0051,51,reserve_replacement,false,0.05,imitate_sorting_sequence--0008,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,0abc5fc0c7da5feb900757c0e2fc356c90ce49d40488d538371ba6ffc4274d5a
|
| 7 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0052,52,reserve_replacement,false,0.0,imitate_sorting_sequence--0017,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,9ba41682374f093d94b0a74bf9ea59a064c9cad635deea9c9932b08eead99a83
|
| 8 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0053,53,reserve_replacement,false,0.0,imitate_sorting_sequence--0029,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,02215431e7f053ea82ab92f34938973462210e562b1f5758e7a7ae701d29c300
|
| 9 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0054,54,reserve_replacement,false,0.0,imitate_sorting_sequence--0036,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,9cefe96e30626cab2cae45ff9d22247af7f50ea87c8c62d2b00c802fb8b66a31
|
| 10 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0055,55,reserve_replacement,false,0.0,imitate_sorting_sequence--0048,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,3d13434232bee26e09e6ce1ce09a111ebedbb5dd7d4c9010f2d181f5f0c1d5d4
|
| 11 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0056,56,reserve_replacement,false,0.0,imitate_sorting_sequence--0049,1,f5f3c0ab6d9cd6894a569a17e4d138076b64afb07ce430d10dcbb20e998d51b1,42937ed09990643cf2a5ca623319af8a60b98e2a2fca996b4c07565341169816
|
| 12 |
+
0,open,general_pickup,general_pickup,standard,general_pickup--0050,50,reserve_replacement,true,1.0,general_pickup--0029,1,723d1a0a15d21a3ad95777bf80414a63919dea60ad5656ad7954c55bbd245c22,6f44cc5eff3b39a922cd842e660a8ce3f858a014840a8ec8213c9831f787321c
|
| 13 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0051,51,reserve_replacement,false,0.0,deposit_coin--0036,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,32453dc5b75f7525cc6b442bfe5281fc46e073773be6c26c1de68801e5b819b9
|
| 14 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0052,52,reserve_replacement,true,1.0,deposit_coin--0037,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,a3e89dc4ccd3fab31bedc56c0d321e5198eeff6c8ad7ba20deaa1bb0a9679180
|
| 15 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0053,53,reserve_replacement,false,0.0,deposit_coin--0041,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,0bdcc61a1cdddbb7cb1c693afbf3246a756776bc288981605a8690991ebf01a8
|
| 16 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0054,54,reserve_replacement,false,0.0,deposit_coin--0043,1,675b50268c2433afb342e131361f046416ff0215c4ca61c6c4745d6a0669b3d6,51191edb880199516b11b343561a3bacb9d2d47c491429584b6d89165ca51c91
|
| 17 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0055,55,reserve_replacement,false,0.0,deposit_coin--0031,1,85a5123e2ac96326ad2c877c1f171f5e5b14d819d62951a594cd7c81f5bad14d,cc85bdc9344a35ad4f37414b5ead9cdf6ff3c9bed61c5847c567f00d2fe81bd2
|
| 18 |
+
1,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.0,fill_egg_holder--0015,1,a876503909848ac16a62806bed0177445f8ee0681970f82f1d49150a1ab5b5b6,e997fd981a5855be673538943400865788b20f4850d0da684d9752ff4778bafa
|
| 19 |
+
1,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0051,51,reserve_replacement,false,0.25,fill_egg_holder--0020,1,a876503909848ac16a62806bed0177445f8ee0681970f82f1d49150a1ab5b5b6,8a9be90e270a00edd75615061418570fe83af25e28eb742ce2548d10b792713f
|
| 20 |
+
1,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0047,47,repaired_variant,false,0.0,imitate_sorting_sequence--0047,3,8c8f9894f60ac3a905644871b9675d9b6c87ba4e3f52e7228951b49ac1623d75,9756370436d26011a577e74aaa1f22ed3a88300cd1c5d583677346a05f171274
|
| 21 |
+
1,open,general_pickup,general_pickup,standard,general_pickup--0050,50,reserve_replacement,false,0.0,general_pickup--0005,1,e32aa0e95d1bdfe638c65520e8512c1e0da1cab16744ac560ba2c8ad59dcbc5d,4f2a6cebf43dc12ea30ee63981e720e7afcec93b03ad3f6a4ba460c59f2d9cf4
|
| 22 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0050,50,reserve_replacement,false,0.25,fill_egg_holder--0013,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,e0b2398081fa2dc27515ea5d8515d2dca795224478847f58cc637eabecaf7252
|
| 23 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0052,52,reserve_replacement,false,0.0,fill_egg_holder--0018,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,7d4a1940b244b7abd5211577bf2ef8223f79d78fe1d14ca84829009bf55bbcdc
|
| 24 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0053,53,reserve_replacement,false,0.1,fill_egg_holder--0040,1,fc04849d62572afb7314fbe5a87f0efcb0396e201232616c2658041be996b208,c6b9073d83de5c5bb3566e893f019847454af5da6a3762c45d6c6dbbd70be8af
|
| 25 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0050,50,reserve_replacement,false,0.0,imitate_sorting_sequence--0016,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,a2e784ebe95a5cd15629ef1713e468a47472ea8d49d62470cfe0e2bf1302352f
|
| 26 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0051,51,reserve_replacement,false,0.15,imitate_sorting_sequence--0024,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,bfceb828b8d01210b287caadb9dcacba90002a3d2823caf084e59f98137c7880
|
| 27 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0052,52,reserve_replacement,false,0.0,imitate_sorting_sequence--0032,1,6bfff3ca014b359cd7458a331501b50eddbddf3073a88c43a93f76c630640e36,8057e37a3180f236bc0f65e6ec62dc79335413985d311999b79317f604873072
|
| 28 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0053,53,reserve_replacement,false,0.0,imitate_sorting_sequence--0043,1,d93b3d1ec634a3f609730818701deead33df6bafedb5f7e3afceb81593046ed1,986c4f50cf26c41e31ccb5906d4678117c60de624b1fa140f167d3e8ddc68f0a
|
| 29 |
+
2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0050,50,reserve_replacement,false,0.0,deposit_coin--0024,1,23fa76d39404c5213a6810d134bd9f012655ddd9faf95176523b826cbb2b5913,0137e168203e7dbdb8f82e5340bc2c1cca2bd033f41ec22569511532fa334938
|
| 30 |
+
2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0051,51,reserve_replacement,false,0.0,deposit_coin--0033,1,23fa76d39404c5213a6810d134bd9f012655ddd9faf95176523b826cbb2b5913,d27ce8a007a1c1eee5de705f4c034d66e0f15995be335e18cea58fa3d2fe952b
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/replacements-and-repairs.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Replacement and repair appendix
|
| 2 |
+
|
| 3 |
+
The complete layout ledger, repair details and completion recoveries are now at the end of the consolidated report.
|
| 4 |
+
|
| 5 |
+
[Open the consolidated results report](../../EVALUATION_RESULTS.md#replacement-and-repair-appendix).
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/results-breakdown.md
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Suite and task results
|
| 2 |
+
|
| 3 |
+
All individual-seed suite scores, task scores and the standard/randomized split are now in the consolidated report.
|
| 4 |
+
|
| 5 |
+
[Open the consolidated results report](../../EVALUATION_RESULTS.md#task-results).
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/retried-originals.csv
ADDED
|
@@ -0,0 +1,180 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed,suite,task,report_task,task_variant,episode,layout_id,record_type,success,partial_score,replaces_episode,attempt_count,protocol,completion_record_sha256
|
| 2 |
+
0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0018,18,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,9d2bd22ee5480bec9ef3dd8264506066dfaca97806c27aa9429395ed1468fffc
|
| 3 |
+
0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0019,19,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,7c12a87a4818edd6cec4aec9e186417f630135417c73ac7e943e6e702e9c4640
|
| 4 |
+
0,generalization,arrange_largest_number,arrange_largest_number,standard,arrange_largest_number--0023,23,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,5f3ea13c20b57396c7e61dc8622f63efc604dade39ac762d84c9c18c6e35d170
|
| 5 |
+
0,generalization,arrange_largest_number_random,arrange_largest_number,random,arrange_largest_number_random--0001,1,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,1ce025b50c5915039d92538be05ae3ae8d9371f9b65ed759317bde56d4237ad2
|
| 6 |
+
0,generalization,arrange_largest_number_random,arrange_largest_number,random,arrange_largest_number_random--0006,6,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,8a475f841cb5c56dae3f3feb582210d2bf3bb9822fe170743471c17aeecd8d1a
|
| 7 |
+
0,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0007,7,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ad6c71d820ba07c9f7db95e76721d31d78dd21de59d6bc6946d99474a9a08f67
|
| 8 |
+
0,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0016,16,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,3e0876a9f314bfef50c1d30a7d87eeec8931a8eb9bba7b8e61b65c270be63c66
|
| 9 |
+
0,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0002,2,original,true,1.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,9ff73e28ae5c2abe86c82d735586a5ba6b69664b6eeac96cb73308e37966c05f
|
| 10 |
+
0,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0003,3,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,84ec4e15129abf87f245e8bcbb7a6527655bb152431b48a27f292a8a678c2648
|
| 11 |
+
0,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0010,10,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,54294e8d19bf2d405dbc84da9c96a7a1182f717103230fc980011d65b11bcb4f
|
| 12 |
+
0,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0021,21,original,false,0.15,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,d34347a9037adc764fae8323c8df1cbfc245aa48890bfe5119144c2d9591e5f1
|
| 13 |
+
0,generalization,make_toast,make_toast,standard,make_toast--0011,11,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,38adc2e894856e55c23e26370342325f59df6611b786bbcbd9f4d67a5a2933b9
|
| 14 |
+
0,generalization,pack_objects_into_box,pack_objects_into_box,standard,pack_objects_into_box--0021,21,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,a1eeb9d7ab1c718d04f25c52cb44fe782a977845afe44ef49dacbf9642b94a21
|
| 15 |
+
0,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0013,13,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,9d2e824ac546bd947deb3004f0b3c2a9769bd44ec02220ff67ea9001a5a3af0c
|
| 16 |
+
0,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0017,17,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,cab2e03bfd4c5d3102718f03e90e48dcc3d89d64df5a1c820795eb41f69c8e67
|
| 17 |
+
0,generalization,pour_liquid_into_cup,pour_liquid_into_cup,standard,pour_liquid_into_cup--0001,1,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,043b9ac45a0b5eeab9d0197d02c308cd8d942781c92fc7db2473e0d57fbff38a
|
| 18 |
+
0,generalization,pour_liquid_into_cup,pour_liquid_into_cup,standard,pour_liquid_into_cup--0023,23,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,d5f4b028954a195331bb1238ac4541dad07acaa83aa9d3c822d471dc9f587b80
|
| 19 |
+
0,generalization,push_T,push_T,standard,push_T--0005,5,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,82b3795ca21f7eafc415c2f05cc20960b0a5b34cf3895ecc64882a6fab7e6826
|
| 20 |
+
0,generalization,push_T,push_T,standard,push_T--0017,17,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,69d77913886d700a8763d1a21a5248b8632ffb03b37a46ed9f0d425d3578188e
|
| 21 |
+
0,generalization,push_T_random,push_T,random,push_T_random--0007,7,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,524ebe155d7e5f9eddf4b04caa46d3bd3beb459c8a5e17d2d65cbf235b7c7821
|
| 22 |
+
0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0007,7,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,82e5b52c2d5b5da1e39a110cbb019e48a7d8ca078f640cb865b9ee10a233e781
|
| 23 |
+
0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0011,11,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,9fd0941210f7252588b9929d56f38d3fe5548d9b319ea7d7bdc07f9b1c2e00b3
|
| 24 |
+
0,generalization,sort_nesting_dolls_by_size,sort_nesting_dolls_by_size,standard,sort_nesting_dolls_by_size--0013,13,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,b2ab34be60d1bab7fe6ef20e27adb15c7f5144ff84372fef1702eac7d349123e
|
| 25 |
+
0,generalization,stack_blocks,stack_blocks,standard,stack_blocks--0006,6,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,7d511310959bd655bad242eb6a09ecb1231a577242cc3e93e68f233b64d25403
|
| 26 |
+
0,generalization,stack_bowls_random,stack_bowls,random,stack_bowls_random--0007,7,original,false,0.15,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,9c87454a6d9559a4965b32a3546770b9ab838a127fd197980be9bf7478c7196f
|
| 27 |
+
0,generalization,stack_bowls_random,stack_bowls,random,stack_bowls_random--0023,23,original,false,0.15,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,30d4f55e7aa2ffa1cd9da2c94b7527cecd27da85fe69c95e7a2dec37f26ccb70
|
| 28 |
+
0,generalization,store_laptop_and_headphones,store_laptop_and_headphones,standard,store_laptop_and_headphones--0006,6,original,true,1.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,896663cca4d9f4ad591083afac9d67076cfa70cae5ee87553f9e4c1a59d50183
|
| 29 |
+
0,generalization,store_laptop_and_headphones_random,store_laptop_and_headphones,random,store_laptop_and_headphones_random--0014,14,original,false,0.2,,3,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,5404aba674e150a13358d2fce30ba2bf79bf795cee0b05d897367034a59d9f84
|
| 30 |
+
0,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0010,10,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,01c7847f424303ffd38b070a434eea93bdb3b1d57612505b9aa0d676b2bb014c
|
| 31 |
+
0,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0016,16,original,true,1.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,575f9b0ba6f3c3fd6d15673f31a1ee5a9e234d3f7bee77f5f6e1c0e672cbe851
|
| 32 |
+
0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0002,2,original,false,0.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,63733e1ee948f3c75ca98eca3446e6475a02c3070d5d6424a0e7bdaf21e608df
|
| 33 |
+
0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0003,3,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,b659f71b60fad6edab1b155b3b9a39feaf9b551faedbc59a45eabe5b16fd324b
|
| 34 |
+
0,generalization,sweep_blocks_random,sweep_blocks,random,sweep_blocks_random--0004,4,original,false,0.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,59b362ae4bcbaa0196ba18c06a6458d7df604e9fc49ab3c83098abd635c52b0f
|
| 35 |
+
0,long_horizon,classify_objects,classify_objects,standard,classify_objects--0027,27,original,false,0.15,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ccf5fe1d901adf28c9f0f6ceed35b8cbec95712b1f17ce00fb3b9b053c3da798
|
| 36 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0006,6,original,false,0.25,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,75c977869cdb14ebd90cc2b8a199a51c34b9372f52eeacea7ed39b8467c4f600
|
| 37 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0039,39,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,6ad2f9118f46af9f67096a9f36f6ea741a3d78f7491cd8f89108b09d33091d0c
|
| 38 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder,standard,fill_egg_holder--0048,48,original,false,0.25,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,42af9d17a3d8c9f12e690cea1c98602c98e1cb9f470e66a493cc8e088bb92edb
|
| 39 |
+
0,long_horizon,fill_pen_holder,fill_pen_holder,standard,fill_pen_holder--0025,25,original,true,1.0,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,56d5a40edf56ce754d19a1e38cacffef1bba61b0c9676bf7f39712b70a883ac7
|
| 40 |
+
0,long_horizon,fill_pen_holder,fill_pen_holder,standard,fill_pen_holder--0048,48,original,false,0.4,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,6c9d0c9befd2c74f75e53d847b26037f18d83e636e1297a7ca6ed6bb3c926061
|
| 41 |
+
0,long_horizon,organize_table,organize_table,standard,organize_table--0000,0,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,2e5c1eae898019bd9fba4eb5af3d8d300c86e947863a48da8267acccf6e3b4e3
|
| 42 |
+
0,long_horizon,organize_table,organize_table,standard,organize_table--0013,13,original,false,0.0,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,7bb5875819e0e46d32e050024ed40133999e834b835d548ef490919f6753b2e9
|
| 43 |
+
0,long_horizon,organize_table,organize_table,standard,organize_table--0024,24,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,3ce909d881288aa5e3a7aef65bfeb4e256ab1aa0b2ed3134782f9bd09b4fe5ce
|
| 44 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0001,1,original,false,0.75,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,52c5d76a5158dae269122142a79328ab638c2f5122bbbeeec34b1a0940edabca
|
| 45 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0005,5,original,false,0.5,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,415dc8f729394b39a951a3f57d0223d65acfbfcc98d520e15c08f751528f01d4
|
| 46 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0009,9,original,false,0.75,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,22f9e233fdf59fd854a95473414fe18ab6dc4972ce632f5968f7e6fa80bfdc7a
|
| 47 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0012,12,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,a138013173611e8936a5d79732da540321beaae41448735f49d796872c9ca1fa
|
| 48 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0016,16,original,false,0.5,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,bafe6e83dfc30cf63483783be5e543b5c1830fc9a375fef8202d37566ae2f5ca
|
| 49 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0018,18,original,false,0.3,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,56dc83b882811fe048e048a8091c9867913317ea5171c40340d5e9906dea090f
|
| 50 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0030,30,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,31e915198b5ff931bc37c3fa445a645ec9874abb798f55b25555f1bdd66d840c
|
| 51 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0031,31,original,false,0.1,,3,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,36c173d4d8727c0be548c691bb744182f35b8f298de756679b1021fefd3713bf
|
| 52 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0041,41,original,false,0.5,,2,6951bc2ec7c7c4607e13a9182629b508416ca66d9be671257d13df2dbd009610,3d8f83a609e30e566f05bf59fc330b29f9698d97450bd0fb2102e5c63383cb27
|
| 53 |
+
0,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0042,42,original,false,0.1,,2,a80b33abf32c4847848e2f56b80eac4b8d3b0da2a356c067cdf32a477241a625,626f48a82062746712e69e74d91bcf3499a6b88c6000c7ec5258518161298bdd
|
| 54 |
+
0,long_horizon,put_bottles_into_dustbin,put_bottles_into_dustbin,standard,put_bottles_into_dustbin--0037,37,original,true,1.0,,2,d53fe474dfcbd37c63fc96efa39950854e817552471e23bfed5ae1cb05bf01e0,ad972e1019644f623c968861dbf59d95782867f0c06c9e99da63eaf97662441b
|
| 55 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence,standard,imitate_sorting_sequence--0034,34,original,false,0.0,,2,6b32e48eb89420065ceb5ad01bc5ba3b1cc99086cb424e299e5c9f5281297023,3eed7b82f11a1a3dc8cbf4b0c7da5c0b48056da51cbaeb0891abeb898c205b91
|
| 56 |
+
0,memory,press_by_number,press_by_number,standard,press_by_number--0015,15,original,true,1.0,,2,6b32e48eb89420065ceb5ad01bc5ba3b1cc99086cb424e299e5c9f5281297023,bd7a89fe3452da0f73d1dd2f91d4bcadf8cd5fa3c6b9728290e3144a650e6e0d
|
| 57 |
+
0,open,solve_equation,solve_equation,standard,solve_equation--0014,14,original,false,0.0,,2,847b0e9ac20a32a47b64b977e656d20c666d8ad62b28b2de5e598ea7fe34fa8b,5907291d39a7fbbf5f6ee1adf57796fb4c136d8962619ae538a7baa44a0a26d9
|
| 58 |
+
0,precision,deposit_coin,deposit_coin,standard,deposit_coin--0014,14,original,false,0.2,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,24dbbc8616c706068aafb7d00a779d83f3be9967b5f60b2c3f1ab62e207f0741
|
| 59 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0000,0,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,246eeacdd1e238b5dc8c4eaa586561ab5b77357dcff794d93f8d9699b671cbf0
|
| 60 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0001,1,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6118d38f8f0b13e6e40bba2eb543e48c17e129c9b198ad34936ee1816a09c99e
|
| 61 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0002,2,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b7d42329a34c0b97da922529338d9577b2bb4ebbfe0467d4fd0171aa6447b75d
|
| 62 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0003,3,original,false,0.4,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,f01e22f7e7ba192b8581804c2b4c50977388ae80b00429cb69b5108845ac5d3e
|
| 63 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0004,4,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,9405c6bbbac4661a8b9e9c06018ef6877cbfb4a5bd28defe03cb550f45a105a3
|
| 64 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0005,5,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6a2b01f07552ad59cc5cd4239e9ad15f32b52094965865b6deaaaa57674085e7
|
| 65 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0006,6,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,bbf45c09db9bdf7f18a9a5ed921f70569290290f105052f2d9c8ba63ae6dad8e
|
| 66 |
+
0,precision,insert_tubes,insert_tubes,standard,insert_tubes--0007,7,original,false,0.4,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,caf5a5e6e61edcf91a0636d7650ad6310224449c18ba10c4267c1f4855cc1d23
|
| 67 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0000,0,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,4cc5ae100707d8575ef496db66401dc495f82e8ada7868c1ce489ac8e2aca5d9
|
| 68 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0001,1,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b39a081c41ad873925f783f700c2884ef75b6e2fa8842fa6e96228c2d54e2578
|
| 69 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0002,2,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,aa1cf4c181b7290f703f51d73cf76b7e0689734d8f0475c9f894227c1802ae58
|
| 70 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0003,3,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,68683ff65457e32796f897d2cce79a7f5382e915629adef29479c450744d3910
|
| 71 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0004,4,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,3305f942d10b3724426677e374dafe3acc740264570c41206eee366ed13c8a72
|
| 72 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0005,5,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,1c5136ca12ec6406779090abb08effbf00c883f5544920ec1874a3efc3d2b4c4
|
| 73 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0006,6,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,8ec87867358523905f77573d220dd95d69a51b49b92154f0b7532cf76ed14a4e
|
| 74 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0007,7,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,cb028f0e32fa71e0811df72b8bd27f89a2097fff3c4d5f3df965cd23f559e232
|
| 75 |
+
0,precision,play_Xylophone,play_Xylophone,standard,play_Xylophone--0008,8,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6971a629863f7cfaf4640a5d23b9bb62362156dc6537e4c19c4636e0e53bb66c
|
| 76 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0004,4,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,6b4944be17f0fbed035cb566579f4e1079f5acd6f53719e5c3ba1bbda4c71392
|
| 77 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0015,15,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,a459e0504046ed10c8d077324065026900b16a972a6a7859c9a713835ce66629
|
| 78 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0017,17,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,50691d529242d7394066b5d1fb2fcfbd1c645bb1cea4c6d5fe30135b8ccd644a
|
| 79 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0019,19,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,4594a999afc254e283196bd5f03a4cb1bb66985cb2372d928d64c02ae2f52f81
|
| 80 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0022,22,original,false,0.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,b1ab680a25bd8fd542416db58345853f914a8a2704c78ff3207d223d9e74db39
|
| 81 |
+
0,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0045,45,original,true,1.0,,2,bf975d33cc025cabac4fb4b2c52ca464c01f04468e53b215238979e7a6c79818,aed9928aede3b15106ba6154abe743117649a77cd27995c28fa617949e0698f1
|
| 82 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0000,0,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1787292e76ac9e88a467c3087c8bb30be831d322e390a16ff941cb2d0fe84084
|
| 83 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0001,1,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1777bd3c3e290fad72a3396b0b9161845112410ad2ce6fe8650643de0bb4208f
|
| 84 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0004,4,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,842eb1343a106d10037b389711249f0fad480537793f89b112bb61b8e9bac644
|
| 85 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0006,6,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,5c71f2fe873f06068c9dd9703ae9c690af71e626e7b90c07f4f8a61c75d84601
|
| 86 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0007,7,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b4202fa61804ff4e750421f14428e413c81aa3f909d15df4bcdb11d08d7702cc
|
| 87 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0013,13,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,ccab300ea1dd05cd7e3c3e03103facd684b453c46e2692ed907c20edf10e1c53
|
| 88 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0018,18,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,5999b54b8dafae9842b5cd77d71a45794c21f112e1d91356870d45bf0bcec3b7
|
| 89 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0021,21,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1aed902c59236af04b9068e670d5c5cf96a8a1ca836b4a4d46f304d4c22236c9
|
| 90 |
+
1,generalization,fold_clothes,fold_clothes,standard,fold_clothes--0023,23,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,37b59debc232efa1a3622451954097879ff57487a8c3c8a65c4900096a98294e
|
| 91 |
+
1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0003,3,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b9f15c9ffeb7eabec45ad246cc74a76bab6249e9c37e6958b6570b14c97c6a5b
|
| 92 |
+
1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0004,4,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,8407bbd937cefeab1e828ee7593cdcb1be88a0c0d9a231253aa15a1442ea4a32
|
| 93 |
+
1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0006,6,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,728d6bbed636e7e9b8f988d4bf25e845f39cbfd57a6868a22117407e8c2ca454
|
| 94 |
+
1,generalization,fold_clothes_random,fold_clothes,random,fold_clothes_random--0010,10,original,false,0.2,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7a7e2044f3303973aee1045cdfbe9473295767a24c4df88fad4137c6513228af
|
| 95 |
+
1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0014,14,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,832c1aa79c2fc686eab658b024e83c268883808519364431799ea6c9953431ec
|
| 96 |
+
1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0018,18,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,82daf42a3916b578d786241cb337f033a9c937443fcb01f686a7f9c098afb440
|
| 97 |
+
1,generalization,hang_mugs,hang_mugs,standard,hang_mugs--0022,22,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3e6c0bd42d696ed812d68119a5f1d6c7a47dfe1f905703b5d5b93e836e976e9c
|
| 98 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0001,1,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,c563a0ff7965f74fb6c44f96f9371ab9297d763696839d96bda20dee2f864785
|
| 99 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0006,6,original,false,0.4,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4beaccc987935cd3d3d5d3f51f8dc137c3d48df660e9a42a027611b0b0fdb0dd
|
| 100 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0009,9,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4dba4e99cdd67b6bffe630a4252d21a7ef471f6aef2e69c9aa6f2bd1a8d0d181
|
| 101 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0010,10,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,9c727ca0f896f9b0aa375f2a4a791c5a6d13cea34de4b2cc09bf83fcb7dc5869
|
| 102 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0016,16,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,ea98a5251c7000b2bbe284385cf90c43e59a81fe9b1c0e59c2dfcc5f3425cd80
|
| 103 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0018,18,original,false,0.15,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,e1515bd2bc6705e1827c96e6aff7ae42fda82aa0e93b1f77451ed841326d50d4
|
| 104 |
+
1,generalization,hang_mugs_random,hang_mugs,random,hang_mugs_random--0019,19,original,false,0.4,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,32bd292bcd81f3c43738c5c73f51e7b49343dfc91a6227613a51b2138796c71d
|
| 105 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0002,2,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,396e3ac0e649db529141627e9f0680a2b2336aa102fd0831f31fd6191a026753
|
| 106 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0004,4,original,false,0.1,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b73e57b84cb2a20a59a69cfd8c6f682c03df413e8f3e59afdf90f2798546b9da
|
| 107 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0006,6,original,false,0.25,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,b8c9a1bca6f4ee7cf27ce39fddb83572d56c07c7f70271d18dcf80f945f6ba57
|
| 108 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0009,9,original,false,0.1,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,1ba0ee80bfff5f39d09702fb0f1961aab435b4a09bb6218b33e7e131faf892a0
|
| 109 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0012,12,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,8ce7c78d63cebf227dc154a786af097d24c8fab7b39d6b86c7223552dabd1b4e
|
| 110 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0020,20,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7697c9fd9e9538121b975c5c98c37f64890b7e233f2fbe842aa4128eda2564bd
|
| 111 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0022,22,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,144188a615dee96afece66deeb7218b8c74474ce6a34a783c0317e488e890fe8
|
| 112 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0023,23,original,false,0.1,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,83a4ef649f1b855aa5de54e3a3c1a2e811151a44cf6455960ad7fdeb2500e4ee
|
| 113 |
+
1,generalization,pack_objects_into_box_random,pack_objects_into_box,random,pack_objects_into_box_random--0024,24,original,false,0.25,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3bf75c5308d09c22ecb2007ae4ccb6e9eaae037be622fa82235935b8f9972136
|
| 114 |
+
1,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0021,21,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,3fa829be5f8eb370a72cc09d7f635a1d9fbc79f6e3bea28f5398ea893c8c73ed
|
| 115 |
+
1,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0023,23,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,28a0b6230e15cd428c73f63e4baaf348e0361082073e65005cd82c7dbd30596e
|
| 116 |
+
1,generalization,push_T_random,push_T,random,push_T_random--0010,10,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,c6bdb024677d49ba2878e0bc3d9d82d067f75bd180ada40879a8d3bd3fe948b8
|
| 117 |
+
1,generalization,push_T_random,push_T,random,push_T_random--0011,11,original,false,0.0,,3,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,9df9daea9d85328436a92b3b036b5f54d00b3e918e579ca8d992e6b9e1177b23
|
| 118 |
+
1,generalization,push_T_random,push_T,random,push_T_random--0015,15,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,4dee106852ba4035b151450a9ef78edb345a88ba414b6cc350e0ebcb0bd681c2
|
| 119 |
+
1,generalization,sort_nesting_dolls_by_size_random,sort_nesting_dolls_by_size,random,sort_nesting_dolls_by_size_random--0000,0,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,9341ca1ee851e48b9fb16bb92f1fa25fe0afff95b3a51474b7db4502496d424a
|
| 120 |
+
1,generalization,sort_nesting_dolls_by_size_random,sort_nesting_dolls_by_size,random,sort_nesting_dolls_by_size_random--0001,1,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,8597697c8e092138b70a13b25babcb3d1f32a7beaa9ec976f09f7c6e58088365
|
| 121 |
+
1,generalization,stack_blocks,stack_blocks,standard,stack_blocks--0006,6,original,true,1.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,cd78fc6ba624dc8ef29064aa35d1b4d76df68c92a059b238d60dbb941db1b228
|
| 122 |
+
1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0002,2,original,false,0.15,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,5b0a82132e4acf21052c89b1c886d90e664fb18b00afbc499108b10edce735c1
|
| 123 |
+
1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0011,11,original,false,0.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,6f17f7dfa1c67d6e4668ac3cbbd3bb335e273dd13ac21c536c546eb52eb94b36
|
| 124 |
+
1,generalization,stack_blocks_random,stack_blocks,random,stack_blocks_random--0014,14,original,false,0.15,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,ca56a1313166a2ad70764d0f8ddb860a1545e24f26cba78f8e8bf2ca3f2599c2
|
| 125 |
+
1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0002,2,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,ce1a910006f377dce23cae16ad65db3f3076a991edea6ab76fbfe9c2c5792964
|
| 126 |
+
1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0004,4,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,86e2db0cf9a5bbd7e2cf44c44b32550fbb5d425aacc5c8363370fec0456bb7c7
|
| 127 |
+
1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0014,14,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,81584c3a793ba3ad4116adcaae944e3b2c331afc4e63d3ae52213e8796843a1d
|
| 128 |
+
1,generalization,stack_bowls,stack_bowls,standard,stack_bowls--0015,15,original,true,1.0,,2,7ae157a10853d4931597d2df391e338bdbb3b4a0faa918b7ca6bcb31ad85bb5a,c124fc6274a44db93aea919e3591d292c7df880e4f2af1b82dc21951f3f344b9
|
| 129 |
+
1,generalization,store_laptop_and_headphones,store_laptop_and_headphones,standard,store_laptop_and_headphones--0022,22,original,true,1.0,,2,83627b59eb149de1a7550edf5997418fa89b89905de1bd90ec9dbad66726af30,73d2d6153c18b86d072d4b79fedebd9ca745613e861fccc94e3e52e49204b9d3
|
| 130 |
+
1,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0000,0,original,true,1.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,7633f7a79b4729864957c784248d3cb9c716ba5ba2dffb3c354d1007138a2bfb
|
| 131 |
+
1,generalization,sweep_blocks,sweep_blocks,standard,sweep_blocks--0003,3,original,false,0.0,,2,53a45c93590ef118ddc19283e54ebf869fd1af7f6aed4eeaca388d0c13239827,09859ce9fb0d11a21a50d2007ab9d22600d2e79dcdd64d040bc9bdd9641025ad
|
| 132 |
+
1,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0049,49,original,false,0.75,,2,c2867b1bb38b88d54cbe92103a5cf6d2020c3d44f67c89412182265c4bac8256,aedb082c7c577b11cf7e79cf5a765c14004ca0607872e65a508712f256f8f5b5
|
| 133 |
+
1,memory,match_and_pick_from_conveyor,match_and_pick_from_conveyor,standard,match_and_pick_from_conveyor--0048,48,original,true,1.0,,2,edcf660c7679d8ced9b72d4d57acc29691c8fd5fc91577e82532bcd46e08c38c,c00bbae79ff276f31caf4db6cca23dff08b7bf1fb3d9c605603d4e4ed7837024
|
| 134 |
+
1,memory,swap_T,swap_T,standard,swap_T--0022,22,original,true,1.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,d76d3b07df1264e2dfb78f0729bb33b309d0ad44be6c17dec7850f6f7074f3b8
|
| 135 |
+
1,memory,swap_T,swap_T,standard,swap_T--0025,25,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,fd67e99c97110fe2cfb305d7554871d722bf38fc2c55f411e6f55f878b36dcb1
|
| 136 |
+
1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0000,0,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,688343532fd457745c18daad9ff701a3ed5953fd35743683185a07dffeb3fdf0
|
| 137 |
+
1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0005,5,original,true,1.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,8a5cb5653ef6db169d70460b0055767125f2e1a20d03f7c450598b12407a9c2b
|
| 138 |
+
1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0022,22,original,false,0.0,,2,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,a4b4274c8c2e2bcc1c9157f69b887908b2f11b1aafcc12483976987aa0402e50
|
| 139 |
+
1,memory,swap_blocks,swap_blocks,standard,swap_blocks--0029,29,original,false,0.0,,3,efe3bd77868d37d56ef2180a07aa34a14dec3d38058348e0b765ed74c5894f69,ca1b94e67fb3278f72cf49e297b538676b236c6583d62e15c39ad778ea566a94
|
| 140 |
+
1,open,general_pickup,general_pickup,standard,general_pickup--0047,47,original,false,0.0,,2,92558f946ce17044692364bcb3515da3925bcf3c646266c8dc9ce1f38771384e,a82103e77e58e1c65aa304cde88ddf49b46ba7b7eaf29fb7a4dd22c7fa08dc95
|
| 141 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0034,34,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,24d4754dd80e3d493dca1d9802d1087cfdf7d9f258bfae73efc1ce3364cbaab9
|
| 142 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0037,37,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,0e37dc2edffa0731287627996d448343be9477c30b8fc1fcb66485dd9b5f095b
|
| 143 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0038,38,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,e62f48bbed2636c1d3c097d92411f7da359fc950a6835f3d9f03b75a32c1c768
|
| 144 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0039,39,original,true,1.0,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,4a1367c1b8b5b7bd9bf60b02efc74debe96482760463d91ac9934015ca43101f
|
| 145 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0040,40,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,80110a17e7abba14ed61a363bb80d5969a7e1833e96ebdc558bf1dbff0a3f006
|
| 146 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0041,41,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,7dd88174665764efd6997ca8ab21e94012e22867e57f932fcf65d707f3b0b989
|
| 147 |
+
1,precision,fasten_screws,fasten_screws,standard,fasten_screws--0042,42,original,false,0.5,,2,266df4b5d9b8eac16145600a8c4f9223d15e03285d61473c3bbd8c43b694f211,37454569da3c33a7ac17da8a70c11032313a21eec68993e41cd898225b154118
|
| 148 |
+
2,generalization,pour_liquid_into_cup_random,pour_liquid_into_cup,random,pour_liquid_into_cup_random--0001,1,original,false,0.0,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,742a47472c7ebfdb6216f8b6822e8598753786ef0743cc567499c9a72f490161
|
| 149 |
+
2,long_horizon,make_kong,make_kong,standard,make_kong--0039,39,original,true,1.0,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,d3acf54fdb5b2644c972f175c38f2d8f64ceddb0edd6cfe529249c6bdaa1cdf1
|
| 150 |
+
2,long_horizon,make_kong,make_kong,standard,make_kong--0043,43,original,true,1.0,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,508b76a2e8b3c50015d0cba005c446bacd091aafeefc7ba723ec4d288dec1b6b
|
| 151 |
+
2,long_horizon,play_stacking_toy,play_stacking_toy,standard,play_stacking_toy--0030,30,original,false,0.0,,2,e8a1d6c899cf43a21ba6a747be1317d102485e5effa9aa460ffb2258d3643d9f,d4951fe70c07365ec3819b12bc58a070075421b7fbb77a05901b8cd648ab62b2
|
| 152 |
+
2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0007,7,original,false,0.5,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,bcfa324b282c5695b60188ac32fa58cf0545b683d91119a5181d627fb925ab4e
|
| 153 |
+
2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0009,9,original,false,0.5,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,db195dd7b9fe78c238b36fe5db9510e6ec6cccb2ac739a56eb827666668fccf4
|
| 154 |
+
2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0017,17,original,false,0.75,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,e48c3dc3fba1fc165ee836a869e3f275c3af2af5dc7e930bc5669578c9d9c003
|
| 155 |
+
2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0027,27,original,true,1.0,,2,accff9b8e37358c73ffb7de5e210191d7b303b15b2256930105c40bc6d0426f6,2f7dde4e35e2f494ff6c6404c89159063b9b2136b3e6be33fe6a74a5eae30e2b
|
| 156 |
+
2,long_horizon,play_tic_tac_toe,play_tic_tac_toe,standard,play_tic_tac_toe--0043,43,original,false,0.3,,2,189b3c836cd93047d77ae2d2ddda67cccbf88bc1c65e4d3ac48bdc91691e5a41,feabf3f67f18bbcbcdf0da8af6c02378aaaa1762382e04f28e79613401297f5d
|
| 157 |
+
2,memory,cover_blocks,cover_blocks,standard,cover_blocks--0016,16,original,true,1.0,,2,c48429d0a143dea28dd311abd2e2b34f23140ef95abb554b100b851901427b87,3ac9ca3dfec6261ec5a21decd3bf294466ad300bac07566589cff551a7d02df6
|
| 158 |
+
2,memory,swap_T,swap_T,standard,swap_T--0003,3,original,false,0.0,,2,c48429d0a143dea28dd311abd2e2b34f23140ef95abb554b100b851901427b87,8a53e0ce6050cf1662038fdae6dc0ff7a86ae375439ce7778505d260f149df7d
|
| 159 |
+
2,open,align_blocks,align_blocks,standard,align_blocks--0001,1,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,9a3983149634c3de0f1957de9236db3ce44c2d60a86761dd0c1549b2722e44a2
|
| 160 |
+
2,open,align_blocks,align_blocks,standard,align_blocks--0004,4,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f71dc35e0eecb79cc9a46c6a26674533bcda0a8df3f16479f154f1aeb1f8315a
|
| 161 |
+
2,open,align_blocks,align_blocks,standard,align_blocks--0007,7,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,0673929de298c98cf4c246a694222178499c8b49e1fe0f8a87a2aeef7cdb0d1f
|
| 162 |
+
2,open,align_blocks,align_blocks,standard,align_blocks--0013,13,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f75e751dbcc28079b1ec163b86fda589c02af44ecc4a832d93168050ef0cb579
|
| 163 |
+
2,open,align_blocks,align_blocks,standard,align_blocks--0035,35,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,b672829c413aeaf482fcaa1a43a9e3af1a424c1a8401bbcd36212294eb9e57fa
|
| 164 |
+
2,open,classify_objects_by_language,classify_objects_by_language,standard,classify_objects_by_language--0013,13,original,false,0.0,,2,13fc494c13228764cbed9fae7505e9eafd3e9a578fedd33399b130a583a00cf1,5800d929cd82ae95900db5ce812d2cb2e7dc61b40c13f2a5b2cce4a6354fb745
|
| 165 |
+
2,open,general_pickup,general_pickup,standard,general_pickup--0008,8,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,89d72079bd22da9e7bebb5ceb80cd608322b9ac0bbc02c6536719b8d5743ea41
|
| 166 |
+
2,open,solve_equation,solve_equation,standard,solve_equation--0036,36,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,f15fb41e7f4e14ddd842811bad6821c8a5baee4ea53a7162d0a6decf3e010316
|
| 167 |
+
2,open,solve_equation,solve_equation,standard,solve_equation--0044,44,original,false,0.0,,2,13fc494c13228764cbed9fae7505e9eafd3e9a578fedd33399b130a583a00cf1,44260033ab2274bd58b185f5b71e17d58955de7375a7067fa74adcdbe854a220
|
| 168 |
+
2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0008,8,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,e08259f3647206e7499242d9365f4a3f8685f53fccfaa02630cf7119e44ca7a0
|
| 169 |
+
2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0014,14,original,false,0.0,,2,5b0d543b1efaec362a60ae2abe93a2ae721654746cec32e22671470dabfba170,a931ad87cd18d27058bfeab723932ca2584680bb0043e02240a9939b4d034df4
|
| 170 |
+
2,open,stack_blocks_by_language,stack_blocks_by_language,standard,stack_blocks_by_language--0028,28,original,false,0.0,,2,cad21efb5dfeda30e62a99174e6cd9982e7ba954561d4e05707d9f9c09523ea5,dc28529548f4fb5438d8fa6527631b176baf40903d50c7c0d30517c5c86c3a18
|
| 171 |
+
2,precision,deposit_coin,deposit_coin,standard,deposit_coin--0047,47,original,false,0.2,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,8120914346c578e7a45c58777553d7dc226d042085f915eed9193f38f67f54aa
|
| 172 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0001,1,original,false,0.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,5ee2fbaa4a2a8e3b32edc82e36bdd9a2d6944952176da5e0a5690c1626170864
|
| 173 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0008,8,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,925d8def8e4e5943174327ff0e90a19ac57b2e5977d119dc7d4eaad5a11466bb
|
| 174 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0010,10,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,7ec138965910f52f1ffde2e117922ca3267c525c96fa6f92c4d98e28cd42c1fb
|
| 175 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0016,16,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,623c156da7194b08c9aec332afdcd408157428204dd6b9f0fa1f4c42c72e6fa3
|
| 176 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0022,22,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,3fb38b9a9dd693b17c2885404241f4165dcf3efceecfc6a8b81595978b5412f7
|
| 177 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0024,24,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,03e0e7dab66aab478d4b00425471f87cd03f5b9b7d17657c33e1d2a83d243ca8
|
| 178 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0036,36,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,85bdce295ce796aac48679df05cb4b1674e913a4eebcc17409e6883afe9fb0cf
|
| 179 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0037,37,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,44cd43cc30f3fa3554b66e644c36ea619c2a89345b15540ce4061100c7484c41
|
| 180 |
+
2,precision,pour_balls_into_vase,pour_balls_into_vase,standard,pour_balls_into_vase--0042,42,original,true,1.0,,2,176cebe33dfe37a90a4a91e385ec48a248da09106c8a285d787ed8cab0e5aa32,b32dfb7d293b019c42b86c9be45092cfd6d2b06bbd9fe76cbdecf19d0a777022
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary-original-selection.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RoboDojo Evaluation Summary
|
| 2 |
+
|
| 3 |
+
## Overview (by capability dimension)
|
| 4 |
+
|
| 5 |
+
Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
|
| 6 |
+
|
| 7 |
+
| Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
|
| 8 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 9 |
+
| Bagel-Robotics-1_original_selection | 92.1% † (116/126) | 28.33 ±1.71 | 35.20 ±1.35 | 34.98 ±2.48 | 43.68 ±2.30 | 40.10 ±1.10 | 58.84 ±0.43 | 67.73 ±2.66 | 67.73 ±2.66 | 1.65 ±1.15 | 2.29 ±1.15 | 34.56 ±0.79 | 41.55 ±0.82 |
|
| 10 |
+
|
| 11 |
+
## Generalization: Standard vs Random (25 episodes each)
|
| 12 |
+
|
| 13 |
+
Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
|
| 14 |
+
|
| 15 |
+
| Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
|
| 16 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 17 |
+
| Bagel-Robotics-1_original_selection | 45.67 ±2.68 | 51.77 ±2.32 | 11.00 ±2.23 | 18.63 ±2.18 | -34.67 | -33.14 |
|
| 18 |
+
|
| 19 |
+
### Per-task Standard vs Random
|
| 20 |
+
|
| 21 |
+
Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
|
| 22 |
+
|
| 23 |
+
#### Bagel-Robotics-1_original_selection
|
| 24 |
+
|
| 25 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 26 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 27 |
+
| <nobr>stack_bowls</nobr> | 96 | 96.60 | 92 | 93.20 | 84 | 86.40 | 91 ±6 | 92.07 ±5.10 |
|
| 28 |
+
| <nobr>stack_bowls_random</nobr> | 0 | 14.40 | 12 | 24.60 | 8 | 21.80 | 7 ±6 | 20.27 ±5.10 |
|
| 29 |
+
| <nobr>push_T</nobr> | 0 | 0.00 | 4 | 4.00 | 4 | 4.00 | 3 ±2 | 2.67 ±2.00 |
|
| 30 |
+
| <nobr>push_T_random</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 31 |
+
| <nobr>pack_objects_into_box</nobr> | 0 | 29.40 | 16 | 42.60 | 8 | 34.60 | 8 ±8 | 35.53 ±6.60 |
|
| 32 |
+
| <nobr>pack_objects_into_box_random</nobr> | 4 | 23.60 | 8 | 28.20 | 4 | 25.20 | 5 ±2 | 25.67 ±2.30 |
|
| 33 |
+
| <nobr>fold_clothes</nobr> | 68 | 70.40 | 80 | 80.80 | 64 | 68.00 | 71 ±8 | 73.07 ±6.40 |
|
| 34 |
+
| <nobr>fold_clothes_random</nobr> | 64 | 65.60 | 64 | 68.00 | 84 | 85.60 | 71 ±10 | 73.07 ±10.00 |
|
| 35 |
+
| <nobr>hang_mugs</nobr> | 60 | 71.00 | 56 | 67.40 | 56 | 69.00 | 57 ±2 | 69.13 ±1.80 |
|
| 36 |
+
| <nobr>hang_mugs_random</nobr> | 0 | 11.80 | 0 | 9.60 | 0 | 12.20 | 0 ±0 | 11.20 ±1.30 |
|
| 37 |
+
| <nobr>sweep_blocks</nobr> | 16 | 16.00 | 8 | 8.00 | 8 | 8.00 | 11 ±4 | 10.67 ±4.00 |
|
| 38 |
+
| <nobr>sweep_blocks_random</nobr> | 4 | 4.00 | 0 | 0.00 | 8 | 8.00 | 4 ±4 | 4.00 ±4.00 |
|
| 39 |
+
| <nobr>pour_liquid_into_cup</nobr> | 52 | 52.00 | 68 | 68.00 | 64 | 64.00 | 61 ±8 | 61.33 ±8.00 |
|
| 40 |
+
| <nobr>pour_liquid_into_cup_random</nobr> | 8 | 8.00 | 20 | 20.00 | 28 | 28.00 | 19 ±10 | 18.67 ±10.00 |
|
| 41 |
+
| <nobr>make_toast</nobr> | 56 | 71.00 | 72 | 79.00 | 60 | 67.00 | 63 ±8 | 72.33 ±6.00 |
|
| 42 |
+
| <nobr>make_toast_random</nobr> | 8 | 24.00 | 16 | 26.00 | 8 | 23.00 | 11 ±4 | 24.33 ±1.50 |
|
| 43 |
+
| <nobr>arrange_largest_number</nobr> | 12 | 16.60 | 4 | 13.00 | 0 | 8.60 | 5 ±6 | 12.73 ±4.00 |
|
| 44 |
+
| <nobr>arrange_largest_number_random</nobr> | 0 | 4.60 | 4 | 9.00 | 0 | 6.60 | 1 ±2 | 6.73 ±2.20 |
|
| 45 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | 36 | 36.00 | 36 | 36.00 | 28 | 28.00 | 33 ±4 | 33.33 ±4.00 |
|
| 46 |
+
| <nobr>sort_nesting_dolls_by_size_random</nobr> | 4 | 4.00 | 4 | 4.00 | 0 | 0.00 | 3 ±2 | 2.67 ±2.00 |
|
| 47 |
+
| <nobr>store_laptop_and_headphones</nobr> | 68 | 78.40 | 68 | 76.80 | 48 | 60.80 | 61 ±10 | 72.00 ±8.80 |
|
| 48 |
+
| <nobr>store_laptop_and_headphones_random</nobr> | 4 | 20.00 | 12 | 25.60 | 20 | 33.60 | 12 ±8 | 26.40 ±6.80 |
|
| 49 |
+
| <nobr>stack_blocks</nobr> | 72 | 76.20 | 88 | 89.80 | 92 | 93.20 | 84 ±10 | 86.40 ±8.50 |
|
| 50 |
+
| <nobr>stack_blocks_random</nobr> | 0 | 10.80 | 0 | 10.20 | 0 | 10.80 | 0 ±0 | 10.60 ±0.30 |
|
| 51 |
+
| **Avg Standard** | 44.7 | 51.13 | 49.3 | 54.88 | 43.0 | 49.30 | 46 ±3 | 51.77 ±2.79 |
|
| 52 |
+
| **Avg Random** | 8.0 | 15.90 | 11.7 | 18.77 | 13.3 | 21.23 | 11 ±3 | 18.63 ±2.67 |
|
| 53 |
+
|
| 54 |
+
## Bagel-Robotics-1_original_selection
|
| 55 |
+
|
| 56 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 57 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 58 |
+
| <nobr>align_blocks</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 59 |
+
| <nobr>arrange_largest_number</nobr> | 6 | 10.60 | 4 | 11.00 | 0 | 7.60 | 3 ±3 | 9.73 ±1.70 |
|
| 60 |
+
| <nobr>build_tower</nobr> | 80 | 85.60 | 64 | 73.60 | 68 | 75.40 | 71 ±8 | 78.20 ±6.00 |
|
| 61 |
+
| <nobr>classify_objects</nobr> | 46 | 56.70 | 32 | 44.50 | 42 | 51.10 | 40 ±7 | 50.77 ±6.10 |
|
| 62 |
+
| <nobr>classify_objects_by_language</nobr> | 0 | 3.60 | 0 | 2.60 | 6 | 9.20 | 2 ±3 | 5.13 ±3.30 |
|
| 63 |
+
| <nobr>cover_blocks</nobr> | 100 | 100.00 | 100 | 100.00 | 100 | 100.00 | 100 ±0 | 100.00 ±0.00 |
|
| 64 |
+
| <nobr>deposit_coin</nobr> | | | 22 | 26.80 | | | | |
|
| 65 |
+
| <nobr>fasten_screws</nobr> | 48 | 72.20 | 36 | 66.80 | 40 | 68.20 | 41 ±6 | 69.07 ±2.70 |
|
| 66 |
+
| <nobr>fill_egg_holder</nobr> | | | | | | | | |
|
| 67 |
+
| <nobr>fill_pen_holder</nobr> | 36 | 53.90 | 22 | 43.90 | 30 | 52.40 | 29 ±7 | 50.07 ±5.00 |
|
| 68 |
+
| <nobr>fold_clothes</nobr> | 66 | 68.00 | 72 | 74.40 | 74 | 76.80 | 71 ±4 | 73.07 ±4.40 |
|
| 69 |
+
| <nobr>general_pickup</nobr> | | | | | 16 | 16.00 | | |
|
| 70 |
+
| <nobr>hang_mugs</nobr> | 30 | 41.40 | 28 | 38.50 | 28 | 40.60 | 29 ±1 | 40.17 ±1.45 |
|
| 71 |
+
| <nobr>imitate_sorting_sequence</nobr> | | | | | | | | |
|
| 72 |
+
| <nobr>insert_key</nobr> | 0 | 15.00 | 0 | 15.00 | 0 | 15.00 | 0 ±0 | 15.00 ±0.00 |
|
| 73 |
+
| <nobr>insert_tubes</nobr> | 72 | 82.80 | 64 | 75.20 | 60 | 74.00 | 65 ±6 | 77.33 ±4.40 |
|
| 74 |
+
| <nobr>make_kong</nobr> | 84 | 84.00 | 86 | 86.00 | 88 | 88.00 | 86 ±2 | 86.00 ±2.00 |
|
| 75 |
+
| <nobr>make_toast</nobr> | 32 | 47.50 | 44 | 52.50 | 34 | 45.00 | 37 ±6 | 48.33 ±3.75 |
|
| 76 |
+
| <nobr>match_and_pick_from_conveyor</nobr> | 82 | 82.00 | 92 | 92.00 | 86 | 86.00 | 87 ±5 | 86.67 ±5.00 |
|
| 77 |
+
| <nobr>organize_table</nobr> | 14 | 57.50 | 18 | 64.50 | 12 | 58.00 | 15 ±3 | 60.00 ±3.50 |
|
| 78 |
+
| <nobr>pack_objects_into_box</nobr> | 2 | 26.50 | 12 | 35.40 | 6 | 29.90 | 7 ±5 | 30.60 ±4.45 |
|
| 79 |
+
| <nobr>pick_from_conveyor_by_image</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 80 |
+
| <nobr>play_Xylophone</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 81 |
+
| <nobr>play_stacking_toy</nobr> | 14 | 31.80 | 12 | 32.00 | 8 | 27.20 | 11 ±3 | 30.33 ±2.40 |
|
| 82 |
+
| <nobr>play_tic_tac_toe</nobr> | 0 | 35.80 | 2 | 40.50 | 6 | 33.90 | 3 ±3 | 36.73 ±3.30 |
|
| 83 |
+
| <nobr>plug_in_charger</nobr> | 12 | 12.00 | 4 | 4.00 | 8 | 8.00 | 8 ±4 | 8.00 ±4.00 |
|
| 84 |
+
| <nobr>pour_balls_into_vase</nobr> | 48 | 48.00 | 62 | 62.00 | 78 | 78.00 | 63 ±15 | 62.67 ±15.00 |
|
| 85 |
+
| <nobr>pour_by_language</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 86 |
+
| <nobr>pour_liquid_into_cup</nobr> | 30 | 30.00 | 44 | 44.00 | 46 | 46.00 | 40 ±8 | 40.00 ±8.00 |
|
| 87 |
+
| <nobr>press_by_number</nobr> | 98 | 98.00 | 100 | 100.00 | 100 | 100.00 | 99 ±1 | 99.33 ±1.00 |
|
| 88 |
+
| <nobr>push_T</nobr> | 0 | 0.00 | 2 | 2.00 | 2 | 2.00 | 1 ±1 | 1.33 ±1.00 |
|
| 89 |
+
| <nobr>put_bottles_into_dustbin</nobr> | 94 | 96.40 | 98 | 98.80 | 98 | 98.80 | 97 ±2 | 98.00 ±1.20 |
|
| 90 |
+
| <nobr>solve_equation</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 91 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | 20 | 20.00 | 20 | 20.00 | 14 | 14.00 | 18 ±3 | 18.00 ±3.00 |
|
| 92 |
+
| <nobr>stack_blocks</nobr> | 36 | 43.50 | 44 | 50.00 | 46 | 52.00 | 42 ±5 | 48.50 ±4.25 |
|
| 93 |
+
| <nobr>stack_blocks_by_language</nobr> | 4 | 4.80 | 8 | 9.20 | 4 | 6.00 | 5 ±2 | 6.67 ±2.20 |
|
| 94 |
+
| <nobr>stack_bowls</nobr> | 48 | 55.50 | 52 | 58.90 | 46 | 54.10 | 49 ±3 | 56.17 ±2.40 |
|
| 95 |
+
| <nobr>store_laptop_and_headphones</nobr> | 36 | 49.20 | 40 | 51.20 | 34 | 47.20 | 37 ±3 | 49.20 ±2.00 |
|
| 96 |
+
| <nobr>store_tools_in_toolbox</nobr> | 0 | 0.50 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.17 ±0.25 |
|
| 97 |
+
| <nobr>swap_T</nobr> | 22 | 22.00 | 22 | 22.00 | 32 | 32.00 | 25 ±5 | 25.33 ±5.00 |
|
| 98 |
+
| <nobr>swap_blocks</nobr> | 18 | 18.00 | 32 | 32.00 | 32 | 32.00 | 27 ±7 | 27.33 ±7.00 |
|
| 99 |
+
| <nobr>sweep_blocks</nobr> | 10 | 10.00 | 4 | 4.00 | 8 | 8.00 | 7 ±3 | 7.33 ±3.00 |
|
| 100 |
+
| **Avg (per seed)** | 31.3 | 38.49 | 31.8 | 39.32 | 32.1 | 39.29 | 32 ±0 | 39.03 ±0.41 |
|
| 101 |
+
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/summary.md
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# RoboDojo Evaluation Summary
|
| 2 |
+
|
| 3 |
+
## Overview (by capability dimension)
|
| 4 |
+
|
| 5 |
+
Progress † marks incomplete evaluation (126 = 42 tasks × 3 seeds). Cells show `mean ±std` across seeds; scores reflect completed cells only. Policies are sorted by overall Avg SR.
|
| 6 |
+
|
| 7 |
+
| Policy | Progress | Generalization SR (%) | Generalization Score | Precision SR (%) | Precision Score | Long-Horizon SR (%) | Long-Horizon Score | Memory SR (%) | Memory Score | Open SR (%) | Open Score | Avg SR (%) | Avg Score |
|
| 8 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 9 |
+
| Bagel-Robotics-1_with_labeled_replacements_and_repair | 100% | 28.33 ±1.71 | 35.20 ±1.35 | 34.08 ±1.90 | 42.43 ±1.42 | 35.33 ±0.82 | 53.03 ±0.28 | 56.56 ±2.06 | 56.89 ±2.17 | 3.67 ±0.31 | 4.25 ±0.25 | 31.59 ±0.33 | 38.36 ±0.46 |
|
| 10 |
+
|
| 11 |
+
## Generalization: Standard vs Random (25 episodes each)
|
| 12 |
+
|
| 13 |
+
Each cell shows `mean ±std` across seeds, averaged over the 12 Generalization tasks with completed data for that half. Δ = Random − Standard (mean SR). Sorted by Standard SR.
|
| 14 |
+
|
| 15 |
+
| Policy | Standard SR (%) | Standard Score | Random SR (%) | Random Score | Δ SR | Δ Score |
|
| 16 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 17 |
+
| Bagel-Robotics-1_with_labeled_replacements_and_repair | 45.67 ±2.68 | 51.77 ±2.32 | 11.00 ±2.23 | 18.63 ±2.18 | -34.67 | -33.14 |
|
| 18 |
+
|
| 19 |
+
### Per-task Standard vs Random
|
| 20 |
+
|
| 21 |
+
Additional view only: each of the 12 Generalization tasks is listed as the 25-episode standard split and its `_random` sibling. Columns match the per-policy all-task tables. Existing merged 50-episode scores are unchanged.
|
| 22 |
+
|
| 23 |
+
#### Bagel-Robotics-1_with_labeled_replacements_and_repair
|
| 24 |
+
|
| 25 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 26 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 27 |
+
| <nobr>stack_bowls</nobr> | 96 | 96.60 | 92 | 93.20 | 84 | 86.40 | 91 ±6 | 92.07 ±5.10 |
|
| 28 |
+
| <nobr>stack_bowls_random</nobr> | 0 | 14.40 | 12 | 24.60 | 8 | 21.80 | 7 ±6 | 20.27 ±5.10 |
|
| 29 |
+
| <nobr>push_T</nobr> | 0 | 0.00 | 4 | 4.00 | 4 | 4.00 | 3 ±2 | 2.67 ±2.00 |
|
| 30 |
+
| <nobr>push_T_random</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 31 |
+
| <nobr>pack_objects_into_box</nobr> | 0 | 29.40 | 16 | 42.60 | 8 | 34.60 | 8 ±8 | 35.53 ±6.60 |
|
| 32 |
+
| <nobr>pack_objects_into_box_random</nobr> | 4 | 23.60 | 8 | 28.20 | 4 | 25.20 | 5 ±2 | 25.67 ±2.30 |
|
| 33 |
+
| <nobr>fold_clothes</nobr> | 68 | 70.40 | 80 | 80.80 | 64 | 68.00 | 71 ±8 | 73.07 ±6.40 |
|
| 34 |
+
| <nobr>fold_clothes_random</nobr> | 64 | 65.60 | 64 | 68.00 | 84 | 85.60 | 71 ±10 | 73.07 ±10.00 |
|
| 35 |
+
| <nobr>hang_mugs</nobr> | 60 | 71.00 | 56 | 67.40 | 56 | 69.00 | 57 ±2 | 69.13 ±1.80 |
|
| 36 |
+
| <nobr>hang_mugs_random</nobr> | 0 | 11.80 | 0 | 9.60 | 0 | 12.20 | 0 ±0 | 11.20 ±1.30 |
|
| 37 |
+
| <nobr>sweep_blocks</nobr> | 16 | 16.00 | 8 | 8.00 | 8 | 8.00 | 11 ±4 | 10.67 ±4.00 |
|
| 38 |
+
| <nobr>sweep_blocks_random</nobr> | 4 | 4.00 | 0 | 0.00 | 8 | 8.00 | 4 ±4 | 4.00 ±4.00 |
|
| 39 |
+
| <nobr>pour_liquid_into_cup</nobr> | 52 | 52.00 | 68 | 68.00 | 64 | 64.00 | 61 ±8 | 61.33 ±8.00 |
|
| 40 |
+
| <nobr>pour_liquid_into_cup_random</nobr> | 8 | 8.00 | 20 | 20.00 | 28 | 28.00 | 19 ±10 | 18.67 ±10.00 |
|
| 41 |
+
| <nobr>make_toast</nobr> | 56 | 71.00 | 72 | 79.00 | 60 | 67.00 | 63 ±8 | 72.33 ±6.00 |
|
| 42 |
+
| <nobr>make_toast_random</nobr> | 8 | 24.00 | 16 | 26.00 | 8 | 23.00 | 11 ±4 | 24.33 ±1.50 |
|
| 43 |
+
| <nobr>arrange_largest_number</nobr> | 12 | 16.60 | 4 | 13.00 | 0 | 8.60 | 5 ±6 | 12.73 ±4.00 |
|
| 44 |
+
| <nobr>arrange_largest_number_random</nobr> | 0 | 4.60 | 4 | 9.00 | 0 | 6.60 | 1 ±2 | 6.73 ±2.20 |
|
| 45 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | 36 | 36.00 | 36 | 36.00 | 28 | 28.00 | 33 ±4 | 33.33 ±4.00 |
|
| 46 |
+
| <nobr>sort_nesting_dolls_by_size_random</nobr> | 4 | 4.00 | 4 | 4.00 | 0 | 0.00 | 3 ±2 | 2.67 ±2.00 |
|
| 47 |
+
| <nobr>store_laptop_and_headphones</nobr> | 68 | 78.40 | 68 | 76.80 | 48 | 60.80 | 61 ±10 | 72.00 ±8.80 |
|
| 48 |
+
| <nobr>store_laptop_and_headphones_random</nobr> | 4 | 20.00 | 12 | 25.60 | 20 | 33.60 | 12 ±8 | 26.40 ±6.80 |
|
| 49 |
+
| <nobr>stack_blocks</nobr> | 72 | 76.20 | 88 | 89.80 | 92 | 93.20 | 84 ±10 | 86.40 ±8.50 |
|
| 50 |
+
| <nobr>stack_blocks_random</nobr> | 0 | 10.80 | 0 | 10.20 | 0 | 10.80 | 0 ±0 | 10.60 ±0.30 |
|
| 51 |
+
| **Avg Standard** | 44.7 | 51.13 | 49.3 | 54.88 | 43.0 | 49.30 | 46 ±3 | 51.77 ±2.79 |
|
| 52 |
+
| **Avg Random** | 8.0 | 15.90 | 11.7 | 18.77 | 13.3 | 21.23 | 11 ±3 | 18.63 ±2.67 |
|
| 53 |
+
|
| 54 |
+
## Bagel-Robotics-1_with_labeled_replacements_and_repair
|
| 55 |
+
|
| 56 |
+
| Task | Seed 0 SR (%) | Seed 0 Score | Seed 1 SR (%) | Seed 1 Score | Seed 2 SR (%) | Seed 2 Score | Avg SR (%) | Avg Score |
|
| 57 |
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
|
| 58 |
+
| <nobr>align_blocks</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 59 |
+
| <nobr>arrange_largest_number</nobr> | 6 | 10.60 | 4 | 11.00 | 0 | 7.60 | 3 ±3 | 9.73 ±1.70 |
|
| 60 |
+
| <nobr>build_tower</nobr> | 80 | 85.60 | 64 | 73.60 | 68 | 75.40 | 71 ±8 | 78.20 ±6.00 |
|
| 61 |
+
| <nobr>classify_objects</nobr> | 46 | 56.70 | 32 | 44.50 | 42 | 51.10 | 40 ±7 | 50.77 ±6.10 |
|
| 62 |
+
| <nobr>classify_objects_by_language</nobr> | 0 | 3.60 | 0 | 2.60 | 6 | 9.20 | 2 ±3 | 5.13 ±3.30 |
|
| 63 |
+
| <nobr>cover_blocks</nobr> | 100 | 100.00 | 100 | 100.00 | 100 | 100.00 | 100 ±0 | 100.00 ±0.00 |
|
| 64 |
+
| <nobr>deposit_coin</nobr> | 28 | 32.40 | 22 | 26.80 | 24 | 28.40 | 25 ±3 | 29.20 ±2.80 |
|
| 65 |
+
| <nobr>fasten_screws</nobr> | 48 | 72.20 | 36 | 66.80 | 40 | 68.20 | 41 ±6 | 69.07 ±2.70 |
|
| 66 |
+
| <nobr>fill_egg_holder</nobr> | 2 | 11.00 | 4 | 13.80 | 0 | 12.20 | 2 ±2 | 12.33 ±1.40 |
|
| 67 |
+
| <nobr>fill_pen_holder</nobr> | 36 | 53.90 | 22 | 43.90 | 30 | 52.40 | 29 ±7 | 50.07 ±5.00 |
|
| 68 |
+
| <nobr>fold_clothes</nobr> | 66 | 68.00 | 72 | 74.40 | 74 | 76.80 | 71 ±4 | 73.07 ±4.40 |
|
| 69 |
+
| <nobr>general_pickup</nobr> | 26 | 26.00 | 24 | 24.00 | 16 | 16.00 | 22 ±5 | 22.00 ±5.00 |
|
| 70 |
+
| <nobr>hang_mugs</nobr> | 30 | 41.40 | 28 | 38.50 | 28 | 40.60 | 29 ±1 | 40.17 ±1.45 |
|
| 71 |
+
| <nobr>imitate_sorting_sequence</nobr> | 2 | 3.20 | 0 | 2.00 | 0 | 2.90 | 1 ±1 | 2.70 ±0.60 |
|
| 72 |
+
| <nobr>insert_key</nobr> | 0 | 15.00 | 0 | 15.00 | 0 | 15.00 | 0 ±0 | 15.00 ±0.00 |
|
| 73 |
+
| <nobr>insert_tubes</nobr> | 72 | 82.80 | 64 | 75.20 | 60 | 74.00 | 65 ±6 | 77.33 ±4.40 |
|
| 74 |
+
| <nobr>make_kong</nobr> | 84 | 84.00 | 86 | 86.00 | 88 | 88.00 | 86 ±2 | 86.00 ±2.00 |
|
| 75 |
+
| <nobr>make_toast</nobr> | 32 | 47.50 | 44 | 52.50 | 34 | 45.00 | 37 ±6 | 48.33 ±3.75 |
|
| 76 |
+
| <nobr>match_and_pick_from_conveyor</nobr> | 82 | 82.00 | 92 | 92.00 | 86 | 86.00 | 87 ±5 | 86.67 ±5.00 |
|
| 77 |
+
| <nobr>organize_table</nobr> | 14 | 57.50 | 18 | 64.50 | 12 | 58.00 | 15 ±3 | 60.00 ±3.50 |
|
| 78 |
+
| <nobr>pack_objects_into_box</nobr> | 2 | 26.50 | 12 | 35.40 | 6 | 29.90 | 7 ±5 | 30.60 ±4.45 |
|
| 79 |
+
| <nobr>pick_from_conveyor_by_image</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 80 |
+
| <nobr>play_Xylophone</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 81 |
+
| <nobr>play_stacking_toy</nobr> | 14 | 31.80 | 12 | 32.00 | 8 | 27.20 | 11 ±3 | 30.33 ±2.40 |
|
| 82 |
+
| <nobr>play_tic_tac_toe</nobr> | 0 | 35.80 | 2 | 40.50 | 6 | 33.90 | 3 ±3 | 36.73 ±3.30 |
|
| 83 |
+
| <nobr>plug_in_charger</nobr> | 12 | 12.00 | 4 | 4.00 | 8 | 8.00 | 8 ±4 | 8.00 ±4.00 |
|
| 84 |
+
| <nobr>pour_balls_into_vase</nobr> | 48 | 48.00 | 62 | 62.00 | 78 | 78.00 | 63 ±15 | 62.67 ±15.00 |
|
| 85 |
+
| <nobr>pour_by_language</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 86 |
+
| <nobr>pour_liquid_into_cup</nobr> | 30 | 30.00 | 44 | 44.00 | 46 | 46.00 | 40 ±8 | 40.00 ±8.00 |
|
| 87 |
+
| <nobr>press_by_number</nobr> | 98 | 98.00 | 100 | 100.00 | 100 | 100.00 | 99 ±1 | 99.33 ±1.00 |
|
| 88 |
+
| <nobr>push_T</nobr> | 0 | 0.00 | 2 | 2.00 | 2 | 2.00 | 1 ±1 | 1.33 ±1.00 |
|
| 89 |
+
| <nobr>put_bottles_into_dustbin</nobr> | 94 | 96.40 | 98 | 98.80 | 98 | 98.80 | 97 ±2 | 98.00 ±1.20 |
|
| 90 |
+
| <nobr>solve_equation</nobr> | 0 | 0.00 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.00 ±0.00 |
|
| 91 |
+
| <nobr>sort_nesting_dolls_by_size</nobr> | 20 | 20.00 | 20 | 20.00 | 14 | 14.00 | 18 ±3 | 18.00 ±3.00 |
|
| 92 |
+
| <nobr>stack_blocks</nobr> | 36 | 43.50 | 44 | 50.00 | 46 | 52.00 | 42 ±5 | 48.50 ±4.25 |
|
| 93 |
+
| <nobr>stack_blocks_by_language</nobr> | 4 | 4.80 | 8 | 9.20 | 4 | 6.00 | 5 ±2 | 6.67 ±2.20 |
|
| 94 |
+
| <nobr>stack_bowls</nobr> | 48 | 55.50 | 52 | 58.90 | 46 | 54.10 | 49 ±3 | 56.17 ±2.40 |
|
| 95 |
+
| <nobr>store_laptop_and_headphones</nobr> | 36 | 49.20 | 40 | 51.20 | 34 | 47.20 | 37 ±3 | 49.20 ±2.00 |
|
| 96 |
+
| <nobr>store_tools_in_toolbox</nobr> | 0 | 0.50 | 0 | 0.00 | 0 | 0.00 | 0 ±0 | 0.17 ±0.25 |
|
| 97 |
+
| <nobr>swap_T</nobr> | 22 | 22.00 | 22 | 22.00 | 32 | 32.00 | 25 ±5 | 25.33 ±5.00 |
|
| 98 |
+
| <nobr>swap_blocks</nobr> | 18 | 18.00 | 32 | 32.00 | 32 | 32.00 | 27 ±7 | 27.33 ±7.00 |
|
| 99 |
+
| <nobr>sweep_blocks</nobr> | 10 | 10.00 | 4 | 4.00 | 8 | 8.00 | 7 ±3 | 7.33 ±3.00 |
|
| 100 |
+
| **Avg (per seed)** | 29.7 | 36.56 | 30.2 | 37.45 | 30.4 | 37.52 | 30 ±0 | 37.18 ±0.48 |
|
| 101 |
+
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/unscored-episodes.csv
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed,suite,task,episode,layout_id,record_type,attempt_count
|
| 2 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder--0028,28,original,4
|
| 3 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder--0035,35,original,4
|
| 4 |
+
0,long_horizon,fill_egg_holder,fill_egg_holder--0037,37,original,4
|
| 5 |
+
1,memory,imitate_sorting_sequence,imitate_sorting_sequence--0047,47,original,2
|
| 6 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder--0013,13,original,4
|
| 7 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder--0018,18,original,4
|
| 8 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder--0040,40,original,3
|
| 9 |
+
2,long_horizon,fill_egg_holder,fill_egg_holder--0051,51,reserve_candidate_failed,1
|
| 10 |
+
0,precision,deposit_coin,deposit_coin--0031,31,original,4
|
| 11 |
+
0,precision,deposit_coin,deposit_coin--0036,36,original,4
|
| 12 |
+
0,precision,deposit_coin,deposit_coin--0037,37,original,4
|
| 13 |
+
0,precision,deposit_coin,deposit_coin--0041,41,original,4
|
| 14 |
+
0,precision,deposit_coin,deposit_coin--0043,43,original,4
|
| 15 |
+
0,precision,deposit_coin,deposit_coin--0050,50,reserve_candidate_failed,3
|
| 16 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0004,4,original,3
|
| 17 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0008,8,original,3
|
| 18 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0017,17,original,3
|
| 19 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0029,29,original,3
|
| 20 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0036,36,original,3
|
| 21 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0048,48,original,3
|
| 22 |
+
0,memory,imitate_sorting_sequence,imitate_sorting_sequence--0049,49,original,3
|
| 23 |
+
0,open,general_pickup,general_pickup--0029,29,original,3
|
| 24 |
+
2,precision,deposit_coin,deposit_coin--0024,24,original,4
|
| 25 |
+
2,precision,deposit_coin,deposit_coin--0033,33,original,4
|
| 26 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0016,16,original,3
|
| 27 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0024,24,original,3
|
| 28 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0032,32,original,3
|
| 29 |
+
2,memory,imitate_sorting_sequence,imitate_sorting_sequence--0043,43,original,3
|
| 30 |
+
1,long_horizon,fill_egg_holder,fill_egg_holder--0015,15,original,
|
| 31 |
+
1,long_horizon,fill_egg_holder,fill_egg_holder--0020,20,original,
|
| 32 |
+
1,open,general_pickup,general_pickup--0005,5,original,
|
inference/policy/Bagel_Robotics_1/evaluation/seeds-0-1-2/verification.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model": "Bagel-Robotics-1",
|
| 3 |
+
"checkpoint": 1000,
|
| 4 |
+
"huggingface_repository": "bageldotcom/Bagel-Robotics-1",
|
| 5 |
+
"huggingface_revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 6 |
+
"seeds": [
|
| 7 |
+
0,
|
| 8 |
+
1,
|
| 9 |
+
2
|
| 10 |
+
],
|
| 11 |
+
"reported_outcomes": 6300,
|
| 12 |
+
"original_selection_outcomes": 6271,
|
| 13 |
+
"later_replacement_outcomes": 28,
|
| 14 |
+
"repaired_variants": 1,
|
| 15 |
+
"preexisting_seed1_precision_selected_substitutions": 4,
|
| 16 |
+
"unscored_originals": 29,
|
| 17 |
+
"failed_replacement_candidates": 2,
|
| 18 |
+
"checkpoint_hashes": {
|
| 19 |
+
"action.pt": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
|
| 20 |
+
"video.pt": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
|
| 21 |
+
"compact_architecture.json": "a7f6696ab9a9beca785531c2402b81ccd1550745cb6e9ccaad67f658630f6b32",
|
| 22 |
+
"dataset_stats.json": "2649d4e3ff3d6d400c9953b7c20c79541474f9c38a8a9916b9caacecd8578f16"
|
| 23 |
+
},
|
| 24 |
+
"source_verification": {
|
| 25 |
+
"primary_archive_manifest_sha256": "b825a5418e7746e3e44d6c25b1dbbeb891353542a7b2f32542064cec850e612f",
|
| 26 |
+
"primary_archive_files_verified": 41025,
|
| 27 |
+
"primary_native_receipts_checked": 3150,
|
| 28 |
+
"seed1_overlap_verified_and_deduplicated": 750,
|
| 29 |
+
"normalized_episode_source_sha256": "b12ed6dd73e4525b3b857431bfb167de1464488164635fb8870a286fc110c4cb",
|
| 30 |
+
"previous_report_verification_sha256": "a438594b0de99cf5300a251d76c8aad50893faf47136e44391bf3163266d6063",
|
| 31 |
+
"complementary_node2_export_sha256": "9cc1fff72c51a69fcbe04d28164187f92d67311648acd5aa3788f1aacc5af409",
|
| 32 |
+
"campaign_protocols_verified": 34
|
| 33 |
+
},
|
| 34 |
+
"summary_commit": "f57e1633ffed59761bc8e568cfe7530c4b33381c",
|
| 35 |
+
"summary_variation": "population standard deviation in overview; half-range in native per-task tables",
|
| 36 |
+
"benchmark_base_commit": "08b7ee46034c3d0c8a389b2b7bfc138f4d55ee4d",
|
| 37 |
+
"local_runtime_patches": true,
|
| 38 |
+
"reported_suite_scores": [
|
| 39 |
+
{
|
| 40 |
+
"seed": 0,
|
| 41 |
+
"suite": "precision",
|
| 42 |
+
"n": 400,
|
| 43 |
+
"success": 144,
|
| 44 |
+
"fail": 256,
|
| 45 |
+
"sr": 36.0,
|
| 46 |
+
"score": 43.5
|
| 47 |
+
},
|
| 48 |
+
{
|
| 49 |
+
"seed": 0,
|
| 50 |
+
"suite": "memory",
|
| 51 |
+
"n": 300,
|
| 52 |
+
"success": 161,
|
| 53 |
+
"fail": 139,
|
| 54 |
+
"sr": 53.666666666666664,
|
| 55 |
+
"score": 53.86666666666667
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"seed": 0,
|
| 59 |
+
"suite": "generalization",
|
| 60 |
+
"n": 600,
|
| 61 |
+
"success": 158,
|
| 62 |
+
"fail": 442,
|
| 63 |
+
"sr": 26.333333333333332,
|
| 64 |
+
"score": 33.516666666666666
|
| 65 |
+
},
|
| 66 |
+
{
|
| 67 |
+
"seed": 0,
|
| 68 |
+
"suite": "long_horizon",
|
| 69 |
+
"n": 400,
|
| 70 |
+
"success": 145,
|
| 71 |
+
"fail": 255,
|
| 72 |
+
"sr": 36.25,
|
| 73 |
+
"score": 53.3875
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"seed": 0,
|
| 77 |
+
"suite": "open",
|
| 78 |
+
"n": 400,
|
| 79 |
+
"success": 15,
|
| 80 |
+
"fail": 385,
|
| 81 |
+
"sr": 3.75,
|
| 82 |
+
"score": 4.3625
|
| 83 |
+
},
|
| 84 |
+
{
|
| 85 |
+
"seed": 1,
|
| 86 |
+
"suite": "precision",
|
| 87 |
+
"n": 400,
|
| 88 |
+
"success": 126,
|
| 89 |
+
"fail": 274,
|
| 90 |
+
"sr": 31.5,
|
| 91 |
+
"score": 40.425
|
| 92 |
+
},
|
| 93 |
+
{
|
| 94 |
+
"seed": 1,
|
| 95 |
+
"suite": "memory",
|
| 96 |
+
"n": 300,
|
| 97 |
+
"success": 173,
|
| 98 |
+
"fail": 127,
|
| 99 |
+
"sr": 57.666666666666664,
|
| 100 |
+
"score": 58.0
|
| 101 |
+
},
|
| 102 |
+
{
|
| 103 |
+
"seed": 1,
|
| 104 |
+
"suite": "generalization",
|
| 105 |
+
"n": 600,
|
| 106 |
+
"success": 183,
|
| 107 |
+
"fail": 417,
|
| 108 |
+
"sr": 30.5,
|
| 109 |
+
"score": 36.825
|
| 110 |
+
},
|
| 111 |
+
{
|
| 112 |
+
"seed": 1,
|
| 113 |
+
"suite": "long_horizon",
|
| 114 |
+
"n": 400,
|
| 115 |
+
"success": 137,
|
| 116 |
+
"fail": 263,
|
| 117 |
+
"sr": 34.25,
|
| 118 |
+
"score": 53.0
|
| 119 |
+
},
|
| 120 |
+
{
|
| 121 |
+
"seed": 1,
|
| 122 |
+
"suite": "open",
|
| 123 |
+
"n": 400,
|
| 124 |
+
"success": 16,
|
| 125 |
+
"fail": 384,
|
| 126 |
+
"sr": 4.0,
|
| 127 |
+
"score": 4.475
|
| 128 |
+
},
|
| 129 |
+
{
|
| 130 |
+
"seed": 2,
|
| 131 |
+
"suite": "precision",
|
| 132 |
+
"n": 400,
|
| 133 |
+
"success": 139,
|
| 134 |
+
"fail": 261,
|
| 135 |
+
"sr": 34.75,
|
| 136 |
+
"score": 43.375
|
| 137 |
+
},
|
| 138 |
+
{
|
| 139 |
+
"seed": 2,
|
| 140 |
+
"suite": "memory",
|
| 141 |
+
"n": 300,
|
| 142 |
+
"success": 175,
|
| 143 |
+
"fail": 125,
|
| 144 |
+
"sr": 58.333333333333336,
|
| 145 |
+
"score": 58.81666666666667
|
| 146 |
+
},
|
| 147 |
+
{
|
| 148 |
+
"seed": 2,
|
| 149 |
+
"suite": "generalization",
|
| 150 |
+
"n": 600,
|
| 151 |
+
"success": 169,
|
| 152 |
+
"fail": 431,
|
| 153 |
+
"sr": 28.166666666666668,
|
| 154 |
+
"score": 35.266666666666666
|
| 155 |
+
},
|
| 156 |
+
{
|
| 157 |
+
"seed": 2,
|
| 158 |
+
"suite": "long_horizon",
|
| 159 |
+
"n": 400,
|
| 160 |
+
"success": 142,
|
| 161 |
+
"fail": 258,
|
| 162 |
+
"sr": 35.5,
|
| 163 |
+
"score": 52.7
|
| 164 |
+
},
|
| 165 |
+
{
|
| 166 |
+
"seed": 2,
|
| 167 |
+
"suite": "open",
|
| 168 |
+
"n": 400,
|
| 169 |
+
"success": 13,
|
| 170 |
+
"fail": 387,
|
| 171 |
+
"sr": 3.25,
|
| 172 |
+
"score": 3.9
|
| 173 |
+
}
|
| 174 |
+
],
|
| 175 |
+
"five_suite_scores_by_seed": [
|
| 176 |
+
{
|
| 177 |
+
"sr": 31.2,
|
| 178 |
+
"score": 37.72666666666667
|
| 179 |
+
},
|
| 180 |
+
{
|
| 181 |
+
"sr": 31.583333333333332,
|
| 182 |
+
"score": 38.545
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"sr": 32.0,
|
| 186 |
+
"score": 38.81166666666667
|
| 187 |
+
}
|
| 188 |
+
],
|
| 189 |
+
"three_seed_overall": {
|
| 190 |
+
"success_rate_mean": 31.594444444444445,
|
| 191 |
+
"success_rate_population_sd": 0.3266931206219816,
|
| 192 |
+
"partial_score_mean": 38.361111111111114,
|
| 193 |
+
"partial_score_population_sd": 0.461640258637678
|
| 194 |
+
},
|
| 195 |
+
"video_files_in_export": false,
|
| 196 |
+
"source_records_modified": false,
|
| 197 |
+
"public_export_verification": {
|
| 198 |
+
"episode_rows_checked_against_retained_source": 6300,
|
| 199 |
+
"successes_all_seeds": 1896,
|
| 200 |
+
"unique_scored_identities": 6300,
|
| 201 |
+
"complete_native_task_seed_cells_reported": 126,
|
| 202 |
+
"complete_native_task_seed_cells_original_only": 116,
|
| 203 |
+
"native_task_seed_values_independently_checked": 126,
|
| 204 |
+
"native_summary_byte_copy_verified": true,
|
| 205 |
+
"native_summary_overview_matches_independent_calculation": true,
|
| 206 |
+
"unchanged_seed1_files_verified": true,
|
| 207 |
+
"summary_sha256": "141e7d8fbad8efc115519510229ab10df762e3d88a213e2d3b1de92edf35d92f",
|
| 208 |
+
"original_summary_sha256": "c09a9a655c51fd8661618ea6bc46c8dc076d9e7a0db7206d754d9e14ab3a8cc5",
|
| 209 |
+
"source_ledger_sha256": "b12ed6dd73e4525b3b857431bfb167de1464488164635fb8870a286fc110c4cb",
|
| 210 |
+
"native_summary_script_sha256": "9ec361af587e1788d83d3cf8670e05b817ff252d33c33829a6337dc16ae370d2",
|
| 211 |
+
"native_launcher_sha256": "7008023f0fd27fc5f8f2adad30b1286a5e6cce432be981b82cf2ffa3498d6ea3"
|
| 212 |
+
}
|
| 213 |
+
}
|
inference/policy/Bagel_Robotics_1/evidence/pr-audit.json
CHANGED
|
@@ -16,8 +16,11 @@
|
|
| 16 |
"readiness_scope": "Opening a review PR; not maintainer acceptance or verified leaderboard publication",
|
| 17 |
"remaining_gates": [
|
| 18 |
"maintainer agreement on proposed eval-only integration and release timeline",
|
| 19 |
-
"assigned evaluator and private credential handoff",
|
| 20 |
"official multi-seed evaluation and leaderboard review",
|
| 21 |
-
"
|
| 22 |
-
]
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
}
|
|
|
|
| 16 |
"readiness_scope": "Opening a review PR; not maintainer acceptance or verified leaderboard publication",
|
| 17 |
"remaining_gates": [
|
| 18 |
"maintainer agreement on proposed eval-only integration and release timeline",
|
|
|
|
| 19 |
"official multi-seed evaluation and leaderboard review",
|
| 20 |
+
"scheduled technical report and training/pruning/export code release"
|
| 21 |
+
],
|
| 22 |
+
"public_checkpoint_access": {
|
| 23 |
+
"passed": true,
|
| 24 |
+
"evidence": "public-checkpoint-access.json"
|
| 25 |
+
}
|
| 26 |
}
|
inference/policy/Bagel_Robotics_1/evidence/public-checkpoint-access.json
ADDED
|
@@ -0,0 +1,187 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"checked_at_utc": "2026-11-07T14:51:23.788607+00:00",
|
| 3 |
+
"repo_id": "bageldotcom/Bagel-Robotics-1",
|
| 4 |
+
"private": false,
|
| 5 |
+
"gated": false,
|
| 6 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 7 |
+
"checkpoint_identity": "790bffc9464fa7a026aa43b217aec2033d186895cad643166daa0bcc01914d5c",
|
| 8 |
+
"authentication": "No Authorization header for API/GET probes; token environment removed, isolated empty Hugging Face home/cache, and implicit token disabled for the published metadata downloader.",
|
| 9 |
+
"metadata_downloader": {
|
| 10 |
+
"passed": true,
|
| 11 |
+
"huggingface_hub": "0.36.0 (submission-pinned version)",
|
| 12 |
+
"fully_downloaded_and_sha256_verified": [
|
| 13 |
+
"COMPLETE.json",
|
| 14 |
+
"compact_architecture.json",
|
| 15 |
+
"dataset_stats.json",
|
| 16 |
+
"validation.json"
|
| 17 |
+
]
|
| 18 |
+
},
|
| 19 |
+
"artifact_download_checks": [
|
| 20 |
+
{
|
| 21 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 22 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 23 |
+
"file": "COMPLETE.json",
|
| 24 |
+
"size": 2491,
|
| 25 |
+
"sha256": "eee062fbcb8c92a881117aa7f06b5cd039dfb9912d85571b59a20aa30837b0e1",
|
| 26 |
+
"remote_identity": "0afb4065d4e25d28c0ae3e6ca1bdae7c2cbd1ae5",
|
| 27 |
+
"anonymous_get_status": 206,
|
| 28 |
+
"bytes_read": 2491,
|
| 29 |
+
"content_range": "bytes 0-2490/2491"
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 33 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 34 |
+
"file": "action.pt",
|
| 35 |
+
"size": 4057916577,
|
| 36 |
+
"sha256": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
|
| 37 |
+
"remote_identity": "9b04836abe5a30972e69ee84e5f5e39c09bf53dd45865551e1d04b42ead90ec3",
|
| 38 |
+
"anonymous_get_status": 206,
|
| 39 |
+
"bytes_read": 4096,
|
| 40 |
+
"content_range": "bytes 0-4095/4057916577"
|
| 41 |
+
},
|
| 42 |
+
{
|
| 43 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 44 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 45 |
+
"file": "compact_architecture.json",
|
| 46 |
+
"size": 6563270,
|
| 47 |
+
"sha256": "a7f6696ab9a9beca785531c2402b81ccd1550745cb6e9ccaad67f658630f6b32",
|
| 48 |
+
"remote_identity": "d49d2e581a33844e824bac2396a6e08d8719a151",
|
| 49 |
+
"anonymous_get_status": 206,
|
| 50 |
+
"bytes_read": 4096,
|
| 51 |
+
"content_range": "bytes 0-4095/6563270"
|
| 52 |
+
},
|
| 53 |
+
{
|
| 54 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 55 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 56 |
+
"file": "dataset_stats.json",
|
| 57 |
+
"size": 100256,
|
| 58 |
+
"sha256": "2649d4e3ff3d6d400c9953b7c20c79541474f9c38a8a9916b9caacecd8578f16",
|
| 59 |
+
"remote_identity": "53027727e6b22c1817fe861b6a91c434d8d8473d",
|
| 60 |
+
"anonymous_get_status": 206,
|
| 61 |
+
"bytes_read": 4096,
|
| 62 |
+
"content_range": "bytes 0-4095/100256"
|
| 63 |
+
},
|
| 64 |
+
{
|
| 65 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 66 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 67 |
+
"file": "validation.json",
|
| 68 |
+
"size": 88781,
|
| 69 |
+
"sha256": "9806ad2b0b58f879351da58eede46507a79433ec9e4eb8f37f415cf58f2ef70e",
|
| 70 |
+
"remote_identity": "0b71b75d4730d4a4b9c3f856330fbf2e27fa8abc",
|
| 71 |
+
"anonymous_get_status": 206,
|
| 72 |
+
"bytes_read": 4096,
|
| 73 |
+
"content_range": "bytes 0-4095/88781"
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"repo": "bageldotcom/Bagel-Robotics-1",
|
| 77 |
+
"revision": "2651631c5d08819088475e8778676f5cf6d9afcf",
|
| 78 |
+
"file": "video.pt",
|
| 79 |
+
"size": 24937214042,
|
| 80 |
+
"sha256": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
|
| 81 |
+
"remote_identity": "fb62db0b3a74a53aeab60f4b1acc8a5be47274f5c9db166684043521fe2a40bb",
|
| 82 |
+
"anonymous_get_status": 206,
|
| 83 |
+
"bytes_read": 4096,
|
| 84 |
+
"content_range": "bytes 0-4095/24937214042"
|
| 85 |
+
},
|
| 86 |
+
{
|
| 87 |
+
"repo": "Haodong082399/VPP2",
|
| 88 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 89 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/LICENSE.txt",
|
| 90 |
+
"size": 11357,
|
| 91 |
+
"remote_identity": "261eeb9e9f8b2b4b0d119366dda99c6fd7d35c64",
|
| 92 |
+
"identity_algorithm": "git_blob_sha1",
|
| 93 |
+
"anonymous_get_status": 206,
|
| 94 |
+
"bytes_read": 4096,
|
| 95 |
+
"content_range": "bytes 0-4095/11357"
|
| 96 |
+
},
|
| 97 |
+
{
|
| 98 |
+
"repo": "Haodong082399/VPP2",
|
| 99 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 100 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/Wan2.1_VAE.pth",
|
| 101 |
+
"size": 507609880,
|
| 102 |
+
"remote_identity": "38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981",
|
| 103 |
+
"identity_algorithm": "sha256",
|
| 104 |
+
"anonymous_get_status": 206,
|
| 105 |
+
"bytes_read": 4096,
|
| 106 |
+
"content_range": "bytes 0-4095/507609880"
|
| 107 |
+
},
|
| 108 |
+
{
|
| 109 |
+
"repo": "Haodong082399/VPP2",
|
| 110 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 111 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/config.json",
|
| 112 |
+
"size": 250,
|
| 113 |
+
"remote_identity": "f5a12b285ec9c2497cafc4a4443fc6ba61b1a30b",
|
| 114 |
+
"identity_algorithm": "git_blob_sha1",
|
| 115 |
+
"anonymous_get_status": 206,
|
| 116 |
+
"bytes_read": 250,
|
| 117 |
+
"content_range": "bytes 0-249/250"
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"repo": "Haodong082399/VPP2",
|
| 121 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 122 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/special_tokens_map.json",
|
| 123 |
+
"size": 6623,
|
| 124 |
+
"remote_identity": "14855e7052ffbb595057dfd791d293c1c940db2c",
|
| 125 |
+
"identity_algorithm": "git_blob_sha1",
|
| 126 |
+
"anonymous_get_status": 206,
|
| 127 |
+
"bytes_read": 4096,
|
| 128 |
+
"content_range": "bytes 0-4095/6623"
|
| 129 |
+
},
|
| 130 |
+
{
|
| 131 |
+
"repo": "Haodong082399/VPP2",
|
| 132 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 133 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/spiece.model",
|
| 134 |
+
"size": 4548313,
|
| 135 |
+
"remote_identity": "e3909a67b780650b35cf529ac782ad2b6b26e6d1f849d3fbb6a872905f452458",
|
| 136 |
+
"identity_algorithm": "sha256",
|
| 137 |
+
"anonymous_get_status": 206,
|
| 138 |
+
"bytes_read": 4096,
|
| 139 |
+
"content_range": "bytes 0-4095/4548313"
|
| 140 |
+
},
|
| 141 |
+
{
|
| 142 |
+
"repo": "Haodong082399/VPP2",
|
| 143 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 144 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/tokenizer.json",
|
| 145 |
+
"size": 16837417,
|
| 146 |
+
"remote_identity": "6e197b4d3dbd71da14b4eb255f4fa91c9c1f2068b20a2de2472967ca3d22602b",
|
| 147 |
+
"identity_algorithm": "sha256",
|
| 148 |
+
"anonymous_get_status": 206,
|
| 149 |
+
"bytes_read": 4096,
|
| 150 |
+
"content_range": "bytes 0-4095/16837417"
|
| 151 |
+
},
|
| 152 |
+
{
|
| 153 |
+
"repo": "Haodong082399/VPP2",
|
| 154 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 155 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/google/umt5-xxl/tokenizer_config.json",
|
| 156 |
+
"size": 61728,
|
| 157 |
+
"remote_identity": "4e1cc1cd85599ce0b47fd0a746af188fe4043ff2",
|
| 158 |
+
"identity_algorithm": "git_blob_sha1",
|
| 159 |
+
"anonymous_get_status": 206,
|
| 160 |
+
"bytes_read": 4096,
|
| 161 |
+
"content_range": "bytes 0-4095/61728"
|
| 162 |
+
},
|
| 163 |
+
{
|
| 164 |
+
"repo": "Haodong082399/VPP2",
|
| 165 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 166 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth",
|
| 167 |
+
"size": 4772359047,
|
| 168 |
+
"remote_identity": "628c9998b613391f193eb67ff68da9667d75f492911e4eb3decf23460a158c38",
|
| 169 |
+
"identity_algorithm": "sha256",
|
| 170 |
+
"anonymous_get_status": 206,
|
| 171 |
+
"bytes_read": 4096,
|
| 172 |
+
"content_range": "bytes 0-4095/4772359047"
|
| 173 |
+
},
|
| 174 |
+
{
|
| 175 |
+
"repo": "Haodong082399/VPP2",
|
| 176 |
+
"revision": "04824ddb3aae11f13db50984ed7d540c3b77dd9f",
|
| 177 |
+
"file": "checkpoints/Wan2.1-I2V-14B-480P/models_t5_umt5-xxl-enc-bf16.pth",
|
| 178 |
+
"size": 11361920418,
|
| 179 |
+
"remote_identity": "7cace0da2b446bbbbc57d031ab6cf163a3d59b366da94e5afe36745b746fd81d",
|
| 180 |
+
"identity_algorithm": "sha256",
|
| 181 |
+
"anonymous_get_status": 206,
|
| 182 |
+
"bytes_read": 4096,
|
| 183 |
+
"content_range": "bytes 0-4095/11361920418"
|
| 184 |
+
}
|
| 185 |
+
],
|
| 186 |
+
"scope": "Anonymous GET access and pinned remote identity/size checked for six checkpoint files and nine encoder/tokenizer assets. Four checkpoint JSON files downloaded fully and SHA-256 verified with the published downloader. Large tensors were range-probed, not downloaded and rehashed in full; earlier full tensor verification remains recorded in VALIDATION.md."
|
| 187 |
+
}
|
inference/policy/Bagel_Robotics_1/runtime/LICENSE
CHANGED
|
@@ -22,3 +22,9 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
| 22 |
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 23 |
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 24 |
SOFTWARE.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 23 |
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 24 |
SOFTWARE.
|
| 25 |
+
|
| 26 |
+
Component attribution
|
| 27 |
+
VPP2/FastWAM source retains the MIT copyright and terms above.
|
| 28 |
+
Wan2.1/DiffSynth-derived components retain Apache-2.0 terms; see licenses/Apache-2.0.txt.
|
| 29 |
+
Source identities and packaging changes are recorded in the adapter's source-provenance.json.
|
| 30 |
+
Model weights and datasets retain their respective upstream terms.
|
inference/policy/Bagel_Robotics_1/runtime/THIRD_PARTY_NOTICES.md
DELETED
|
@@ -1,8 +0,0 @@
|
|
| 1 |
-
# Runtime source notices
|
| 2 |
-
|
| 3 |
-
Reused source components from [VPP2](https://github.com/roboterax/video-prediction-policy-2)/FastWAM retain the MIT notice in LICENSE.
|
| 4 |
-
Wan2.1/DiffSynth-derived components retain Apache-2.0 terms, included in
|
| 5 |
-
licenses/Apache-2.0.txt. Source attribution is retained in individual modules
|
| 6 |
-
where present. Source hashes and packaging changes accompany the adapter in
|
| 7 |
-
source-provenance.json. Model weights and datasets are separate artifacts with
|
| 8 |
-
their own upstream terms; this code license does not relicense those artifacts.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
inference/policy/Bagel_Robotics_1/runtime/pyproject.toml
CHANGED
|
@@ -8,7 +8,7 @@ version = "0.1.0"
|
|
| 8 |
description = "Isolated inference runtime for Bagel-Robotics-1 recovery checkpoint 1000"
|
| 9 |
requires-python = ">=3.10"
|
| 10 |
license = "MIT AND Apache-2.0"
|
| 11 |
-
license-files = ["LICENSE", "
|
| 12 |
dependencies = [
|
| 13 |
"torch==2.11.0", "torchvision==0.26.0", "numpy==1.26.4",
|
| 14 |
"transformers==4.49.0", "huggingface-hub==0.36.0", "hydra-core==1.3.2",
|
|
|
|
| 8 |
description = "Isolated inference runtime for Bagel-Robotics-1 recovery checkpoint 1000"
|
| 9 |
requires-python = ">=3.10"
|
| 10 |
license = "MIT AND Apache-2.0"
|
| 11 |
+
license-files = ["LICENSE", "licenses/*.txt"]
|
| 12 |
dependencies = [
|
| 13 |
"torch==2.11.0", "torchvision==0.26.0", "numpy==1.26.4",
|
| 14 |
"transformers==4.49.0", "huggingface-hub==0.36.0", "hydra-core==1.3.2",
|
inference/policy/Bagel_Robotics_1/runtime/src/bagel_cp1000/models/wan21_14b/vpp2.py
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
# See LICENSE
|
| 2 |
from contextlib import contextmanager, nullcontext
|
| 3 |
import os
|
| 4 |
from pathlib import Path
|
|
|
|
| 1 |
+
# See LICENSE for retained source notices.
|
| 2 |
from contextlib import contextmanager, nullcontext
|
| 3 |
import os
|
| 4 |
from pathlib import Path
|
inference/policy/Bagel_Robotics_1/source-provenance.json
CHANGED
|
@@ -125,7 +125,7 @@
|
|
| 125 |
{
|
| 126 |
"source": "models/wan21_14b/vpp2.py",
|
| 127 |
"source_sha256": "8c419af905a8ffa93d77c827a283b7aeff744f3a461c390783eef8a333c31756",
|
| 128 |
-
"packaged_sha256": "
|
| 129 |
},
|
| 130 |
{
|
| 131 |
"source": "models/wan21_14b/wan21.py",
|
|
@@ -188,7 +188,8 @@
|
|
| 188 |
"Recorded deterministic backend controls applied at startup",
|
| 189 |
"Remove unreachable adapter branches for unsupported action spaces; preserve EE16 math",
|
| 190 |
"Restore evaluated processor, image transform and action/state merger dependencies under the isolated namespace",
|
| 191 |
-
"Recursively map serialized Hydra targets to bagel_cp1000; preserve tensor keys and all non-target configuration values"
|
|
|
|
| 192 |
],
|
| 193 |
"adapter_files": [
|
| 194 |
{
|
|
|
|
| 125 |
{
|
| 126 |
"source": "models/wan21_14b/vpp2.py",
|
| 127 |
"source_sha256": "8c419af905a8ffa93d77c827a283b7aeff744f3a461c390783eef8a333c31756",
|
| 128 |
+
"packaged_sha256": "a1df6528f7f9536b2ff4ad3913fa718d40dfb1061dc35e9fa0e28e2e1fa3bf96"
|
| 129 |
},
|
| 130 |
{
|
| 131 |
"source": "models/wan21_14b/wan21.py",
|
|
|
|
| 188 |
"Recorded deterministic backend controls applied at startup",
|
| 189 |
"Remove unreachable adapter branches for unsupported action spaces; preserve EE16 math",
|
| 190 |
"Restore evaluated processor, image transform and action/state merger dependencies under the isolated namespace",
|
| 191 |
+
"Recursively map serialized Hydra targets to bagel_cp1000; preserve tensor keys and all non-target configuration values",
|
| 192 |
+
"Consolidate explanatory notices into LICENSE; update the license reference comment without changing executable code"
|
| 193 |
],
|
| 194 |
"adapter_files": [
|
| 195 |
{
|